WikiSummary, the Social Science Summary Database

 

Zaller and Feldman. 1992. A simple theory of the survey response. American Journal of Political Science 36: 579-616.

In Brief

We have multiple considerations (for and against) concerning policy issues. At any given time, different considerations are on our minds, and these salient considerations determine what we tell surveyors about our policy preferences. (An argument very similar to Zaller 1992.)

Main Argument

Three Axioms

A deductive model based on three axioms:

1. Ambivalence. "Most people possess opposing considerations on most issues, that is, considerations that might lead them to decide the issue either way." (585)
2. Response. "Individuals answer survey questions by averaging across the considerations that happen to be salient at the moment of response, where salience is determined by the accessibility axiom."
3. Accessibility: "The accessibility of any given consideration depends on a stochastic sampling process, where considerations that have been recently thought about are somewhat more likely to be sampled." (586)

Seventeen Deductions

From these three axioms, the authors deduce 17 hypotheses, of which they confirm 16 (see list on pg 608). Grouping these axioms into three headings (as in pg 607), the model generally explains all these phenomena:

1. "Dependence of attitude reports on probabilistic memory search." Thus, responses are unstable over time, but still centered around the mean of the underlying considerations (e.g. if 80% of my considerations are pro and 20% are con, I will answer "pro" 80% of the time).
2. "Effect of ideas recently made salient." This method can explain many surveying artifacts, such as question order effects, race-of-interviewer effects, endorsement effects, reference group effects, question framing effects, and TV news priming effects.
3. "Effects of thought on attitude reports" (607). When people are asked to think before answering, they give more stable answers. Similarly, more politically active/aware people give more stable responses. According to this model, these people have a larger number of considerations "accessible" (i.e. on their mind) when they answer a closed-ended survey question; since their response is an average of the accessible considerations (axiom 2), it follows that their answers are more stable (since they are averaging across a larger number of considerations).

Empirical Method

The authors administered two types of survey, asking similar questions as in earlier studies. Instead of simply asking the closed-ended response items, though, respondents were asked either to "stop-and-think" (before answering) or reflect (after answering), telling the surveyor what thoughts crossed their mind when thinking about the question. Each thought ("consideration") was coded (number of considerations, number of considerations in each direction, number of explicit expressions of ambivalence, etc.).

Implications

Although scholars worry about individual-level response instability, aggregate level data should nevertheless be okay. Since individuals are randomly sampling their considerations, you would expect bias to average out in the aggregate (unless, of course, something is biasing the way that respondents sample their considerations, like a poorly written survey or recent news coverage; see e.g. Iyengar and Kinder 1987).