Here’s what I saw on the first screen of a survey Food and Wine magazine wanted me to take:
(I entered my actual age for the question; I changed it to 99 for the screengrab.)
There’s no “continue” button here. I was initially hopeful that it was one of those speedy flash-based things that would zip me to the next question as soon as I clicked a radio button, but no, nothing happened.
Temporarily disabling Adblock Plus– a Firefox extension that 8 million people used yesterday — makes the page (and its missing button) render properly, as does viewing it in Internet Explorer:
How is it possible that your testing didn’t catch this? How is it possible that you’ve managed to create the only survey I’ve ever seen that can be defeated by the most common advertisement blocking software on the planet? What vacuum are you working in where none of your staff is sophisticated enough to use Adblock?
About six clicks later, after a few questions about three different car brands, I hit a matrix, asking about one of the three brands. Surprisingly, I only got asked the (horribly redundant) questions about this one brand, which was refreshing:
Is it just me, or do orange juice brand managers seem more likely than just about anyone else to think their brands have the ability to come to life with distinct human personality traits and characteristics? Is it just one crazy person who keeps using Greenfield for this? Is it the whole industry? Someone help me understand this.
This is at least the third time I’ve seen one of these:
(Also, “warm?” You really want me to think about whether or not any of these refrigerated products could best be described as being “warm?” Because, ew.)
OH, COME ON. This is just ridiculous. I’m almost too fatigued looking at this to copy and paste it here, let alone fill it out. In fact, I think the only reason I’m continuing at all is so I can see what fresh horror awaits. How do they expect real people to answer things like this? Oh, right — because they promise us a sweepstakes entry in exchange for 35 minutes of our time. (No, really, they do:)
Anyway, here’s what I’m yelling about now:
Crap, I know I’ve been busy, but this is ridiculous.
Still fighting the good fight, but haven’t had time to write about (or even look at) much research lately. I did catch this grid a couple days ago, and I think it’s worth throwing up and looking at, not because it’s a particularly terrible example (it’s sadly just typical), but because I can imagine so many better ways to measure this:
Can’t you picture something with different carmaker logos (or, maybe even better, images of their most popular models) that you can drag up and down or left and right to indicate exactly how likely you would be to consider each of them? And that’s just my very first thought on this one.
Flash makes pretty much anything possible, but we’re still using virtual #2 pencils to fill in virtual scantron bubbles, aren’t we? What do you think?
From this story at The Consumerist. I’m sure it’s just a browser rendering issue that would have been solved with more testing, but right now, it’s just one of my all time favorite matrixes:
Well done, Best Buy!
“We know respondents don’t like grids,” Jackie Lorch, SSI VP, Global Knowledge Management, points out. “They’ve been telling us that for years in focus groups and feedback, but we’ve always thought of grids as a necessary evil in questionnaire design. Now, we’re beginning to learn that not only are grids frustrating for respondents – they actually produce inferior data.”
Here’s a link to the full report.
L0t there to digest. What do you all think?
Not in a good way, though:
It’s always possible this was a Firefox-only problem, but I (a) doubt it and (b) don’t think that makes this OK either, actually.
It’s not eggregious, of course, just … off, and as such, distracting. Don’t distract your respondents; don’t make them take time out from thinking about their answers to think about why your boxes won’t fit on your page.
You’ll have to excuse the low quality of the image here; I snagged it off a WebEx meeting where the presenter flashed it by as part of a PowerPoint he was sharing. I don’t know the source, I don’t know the rationale, I just know it’s probably going to be the worst matrix you ever see:
Click to see it full size. You might want to print it out and tape it up to remind you to never do this, though I suspect you wouldn’t be reading this if you would.