Of the online polls I see regularly, Harris Interactive seems to be the best of the lot. They actually did something I thought was particularly good in a piece of research I saw today, but of course I’m going to post on the thing they did today that I didn’t like as much.
First, I was asked this:
Nothing horrible there; it’s a matrix, but it’s not too huge, and the answer choices are fine. But then they ask this as the very next screen:
I check my bank statements all the time, so I started clicking the right-most column, just like I’d done with the Facebook questions — and then I realized that the right-most column wasn’t “very likely” anymore, but had mutated into “not applicable.” I suppose you can make an argument that the column is necessary on this page — the Facebook questions were asked only of Facebook users, and it’s possible some people answering this screen might not have a 401(k), or might not have any credit cards — but I think it would have been better to either let the “not at all likely” column take care of those folks or to have added the “not applicable” onto the Facebook questions as well.
The more you can keep answer choices identical, the more your respondents can glide effortlessly through the research. Note that what Harris did here is really quite minor, and not at all as aggravating as what I’ve seen done elsewhere from time to time, where a painfully over-sized matrix runs across multiple pages, but with the answer choices randomly shifting to abuse the respondents make sure the respondents are paying attention. What I’m talking about here is nitpicking compared to that sort of thing — but the common element is that your respondent is going to click the wrong thing because they expect their answer choice to be somewhere else.