Can of worms, I hereby declare you open.
OK. If you’re paying attention to the Presidential race, you probably have noticed a few controversies over election polling in the past few weeks. Here’s a short summary:
- RNC holds convention, picks Sarah Palin as VP.
- Various polls show McCain/Palin with pretty large surge.
- Various blogger types complain the polls are “rigged” because they have more Republicans in them then they should, and certainly have more Republicans in them then the previous round(s) of polls.
So, what’s going on? Are the polls being rigged?
Of course not. Look, polls may be a lot of things, including shoddy, sloppy, and/or hopelessly incompetent, but with very rare exceptions — I think there is a pollster or two out there who will put his thumb on the scale if the final poll before the election looks “off” — no one is going to risk their reputation to do something of such dubious value eight weeks out. Yes, the media loves that the race is so close right now, but I think the media would be having just as much fun if Obama was 20 points ahead, and really, probably the same if McCain was. A 20 point lead would be astounding, for either party, and would be representative of a campaign that really caught fire with the public in a way that just doesn’t happen anymore. In short, I think the McCain surge in the polls is largely “real,” and, yes, the polls do have “too many” Republicans in them, but that’s OK, because the polls are reflecting reality at this particular time, and the current reality is that Sarah Palin has invigorated a lot of conservatives who previously were not very happy with John McCain.
Different pollsters ask the party identification question in different ways. On one end of the scale, you can treat party ID as a largely immutable characteristic like gender, age, or race — if you were born in 1950, you’re 58 years old, and that answer isn’t going to change until your next birthday comes around. These pollsters, if polling a state that registers voters by party ID, may be weighting their data or managing their sample in such a way that the respondents in the poll closely match those party registration figures, or, possibly, closely match a different set of figures the pollster has decided will more accurately reflect the electorate this year. These pollsters (again, in the states where you register by party) are asking, essentially, “Are you registered to vote as a Republican, as a Democrat, or as a member of another party?”
On the completely opposite end of the scale, you have those who believe that party ID is a very changeable characteristic, and that it can be envisioned as a 10-point scale, with Democrats all the way at the left, Republicans all the way at the right, and independents dead center. A “1” might be mad at Barack Obama one day (FISA, perhaps?) and say she’s a “3.” A “5” might have liked Palin’s convention speech so much that he’s an “8” today, but maybe he’s a “6” a few days later, a “4.5” when you poll him a week later, and during the course of your too-long poll, where you randomly branch him down a path that asks critical questions about Palin, he decides you’ve gone too far and by the time you ask him his party affiliation, he’s on the other side of the hump, a “5.5” Pollsters who treat party ID as a changeable characteristic, as the way a respondent feels on a given day, are more likely to ask a question along the lines of “Do you generally consider yourself a Democrat, a Republican, an independent, or something else?”
There are polling outfits in the middle, too, who apply some weighting based on their past experience, past polling, whatever. There are a million ways to do it, and no one can say for sure which is right. If your personal worldview is that party affiliation is what your voter registration card says on it, and that the percentage of Democrats and Republicans from poll to poll should rarely or never shift, I’d suggest you look at West Virginia, where voter registration statatistics, as of April (PDF), are 56% Democratic and 29% Republican. Now, both US Senators from West Virginia are Democrats, as are two of its three US House members, and the governor, but I guarantee you this:
Take a random sample of West Virginia registered voters and split it in half. For half, ask them how they are registered, followed by four non-political questions — ask about movies, or college football, or something. The other half, reverse it — questions first, then registration. The results will generally line up with the registration data you’re making calls from, and the two halves of the sample will have pretty similar numbers — about 2:1 Democrats over Republicans, just like in the real registration data.
Now take another random sample, again split in half, and ask the same non-political questions; this time, though, ask the party ID question differently — try asking what respondents generally consider themselves to be. I’m betting that you’re going to get 10 to 15% more Republicans this time, and that the numbers will be similar for the two halves — the fact we’re asking “consider” instead of “registered” will help the Republicans, but there won’t be a difference between those asked the question before and those asked the question after the topical questions of the poll.
Next: ask questions about the 2008 race for Governor. Again, ask what respondents consider themselves. This time, I’m betting the question asked up front will be 10% to 15% more Republican than actual registration, and the question asked after the questions on the guberbatorial election wil be closer to the actual 2:1 figure. Joe Manchin is pretty popular, and after people say they’re going to vote for him, they’re going to consider themselves Democrats, which they technically are.
Finally, swap out the Governor race questions and ask about McCain/Obama. This time, I predict 10 to 15% more Republicans in the affiliation question asked up front, and 20 to 30% more Republicans in the question asked after the horserace question.
If party identification is able to swing so much based simply on where it’s asked in what type of poll, and I’m pretty sure it is, why isn’t it just as likely that political happenings — the introduction of a new face to the campaign, one who is hugely popular with the exact same conservative voters who previously were unenthusiastic about McCain — also change party ID numbers the same way?