Don’t Fear “Not Sure.”

I think Polling Point does some interesting work; they’ve got the thing where they show about 16 buttons, each labeled with the name of a company, and ask you to click to select the ones you like, which turn green as you do; then they put up the same list and ask you to click the ones you don’t like, which turn red. Clever, though I think I’d try to integrate the corporate logos into the buttons as well.

They also have a cool way of asking follow-ups, where they’ll show them in a pop-up balloon-looking window that connects to your initially selected choice with a speech-bubble-looking-thing.

But then they ask me this:

The first choice — Limited — well, I’ve been in Limited a slew of times in my life; I’ve stood around and watched my wife buy various things. I’ve definitely been there, but I’ve never personally purchased anything there.

I guess “Did not purchase anything here in the last week” is the most appropriate option, and maybe it’s the obvious option to other people, but to me, that option implies I’ve bought things there occasionally.

I think the best solution is a “not sure” choice. “Not sure” on almost EVERY question gives everyone who doesn’t fit one of the other categories a place to put themselves. I’d definitely choose “not sure” on Limited and on Catherine’s Plus Store — Limited, because as I’vr said, been there lots but never bought anything; Catherine’s because the name is vaguely familiar, but I’m sure I’ve never been — so I can’t state authoritatively if it’s in my area or not.

Typically, when I offer “not sure” on a piece of research, the percentage of respondents selecting it is < 5%. If it’s much larger than that, there’s one of two things happening: either I’ve asked people to rate something that they’re unfamiliar with, or I’ve designed a defective piece of research.

Either way, we’ve learned something valuable. “Not sure” is a useful tool: it keeps respondents from feeling “trapped” by a question with options that don’t correctly apply to them, and it serves as a self-diagnosis tool for you.



Filed under answer choices, Polling Point

3 responses to “Don’t Fear “Not Sure.”

  1. Pingback: What Ever Happened to Yes and No? « Bad Research; No Biscuit.

  2. doug rivers

    Just noticed this post…

    Unlike most MR surveys, you can skip any time, so a respondent should never feel “trapped.” I’ve never understood why MR people feel the need to require responses. Most respondents skip very few questions (and they only skip these because the answers don’t make sense for them, i.e. they are trying to cooperate with you). A respondent who skips a lot of questions isn’t cooperating and forcing them to answer is more likely to produce meaningless responses. I’d rather be able to identify them (based on how many questions they skipped) and drop them from the sample.

  3. edaiiur hhipuih vfqbeuxt hudikyfeut lwoknkb laqnwjlmq vzlnisp

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s