Stop screwing up the conversation.

You’ve probably given a lot of thought to the scales you use in your research. You’re undoubtedly aware of the advantages and disadvantages of 5-point Likert vs. 4-point Likert scales, and you’ve probably even spent time arguing with colleagues or clients over whether a 7-point or 5-point scale would produce better learning in a given situation.

Here’s the part where I piss you off and tell you I believe you’re doing it wrong:

You’re doing it wrong.

This is the thing, and it’s a theme I will never stop hammering: market research is a conversation between the researcher and the respondent. The more the interaction feels like an actual conversation to the respondent, the easier and more natural the process is. You ask a question, you get an answer. I think fairly few people would say it’s difficult or onerous or boring to hold a conversation with a friend, and yet, those are probably the very first words many would use to describe the agony of taking a typical piece of research.

So what does this have to do with scales?

Which of these is a conversation you can imagine people having in real life?

  • “Bob, I think that woman over there is very attractive. Do you strongly agree? Somewhat agree? Neither agree nor disagree? Somewhat disagree? Or strongly disagree?”
  • “Bob, on a scale of one to seven, I think that woman over there is a perfect seven.”
  • “I’d give her a 10. What do you think?”

In other words, no one in the history of the world has ever described a beautiful woman (or an attractive man, or a perfectly executed Olympic gymnastic routine) as being “a perfect 5.” Frame your questions in terms that your respondent are already familiar with, so they don’t have to constantly keep translating your strange foreign language of research-speak into English, so that they can understand it, formulate an answer, and then translate it back again into research-speak so they can give you the response you’re forcing them to choose.

tl;dr: stop screwing up the conversation.



Filed under answer choices, bad user experiences, Market Research, quality of responses

4 responses to “Stop screwing up the conversation.

  1. Wow, I’m guilty of this one. Ziggy Zubric makes a similar case for different reasons: Less really is more when it comes to response scales. I’m having a tough time letting go of my traditional scales. Is there a 12-step program you can recommend?!

  2. That was a really interesting post. I’m not sure about switching the order to put the “middle” choice first, but I agree with the rest of it, to the point that I think I should have added a fourth example up there — “Wow, she’s cute. What do you think?”

    Switching styles is hard. I don’t know of any Likert Anonymous meetings, but I’d love to just see more researchers experimenting with non-traditional scales, possibly using split samples to see where the differences are. I think the key is for us to understand what the results are going to look like; once we do, we can explain the benefits and drawbacks of the different methodologies, which I think will result in clients deciding they’d rather go with the more “natural” style.

  3. Yeah, really good post. This will certainly change the way I think about surveys. Thanks 🙂

  4. Although I’m fully in agreement about the need to think about the conversation, I can’t help recalling the film ’10’, where the whole thing swung on the Bo Derek character being rated as a perfect 10 by the Dudly Moore character.

    Conversations vary. And they don’t always take predictable paths.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s