Greenfield: They’re Just Weird. (And why are they plagiarizing recipes from the BBC?)

A typical survey invite email from Greenfield Online:

I don’t get it. The recipe, the quotation from Aristotle, the mountain climber — what does any of this this have to do with orange juice? (I haven’t clicked to see what the survey is about yet, but I’m sure it’ll end up being about orange juice. Again. BTW, I just bought a 4-pack of Tropicana at Costco, because it’s what they have, not because I thought it was the most likely to be “outgoing” if it happened to “come to life as a person with distinct personality traits and characteristics.” And just scroll down a bit if you don’t know what I’m talking about.)

So anyway, that’s weird enough, and it was going to be a post on its own, but I figured I’d take a quick look through my in box and see what else Greenfield was using so I could mock them a little more. The very next invite I had included a recipe for monkfish pasta, and here’s where this starts to get pretty funny:

Greenfield is based in the US state of Connecticut. We don’t much use the metric system in America, and yet here’s a recipe that not only includes metric portions at all, but lists them first … and also refers to a type of fish (huss?) I’d never before heard of, as well as to  “double cream,” which despite sounding delicious, does not exist here. At all.

That all seemed odd enough that I copied and pasted the first line of ingredients into Google, and guess what the first hit was?

This:

I’m no lawyer, but it’s probably legal for Greenfield to include this recipe, especially as it seems to be coming from the “Sea Fish Authority,” which seems to be an official UK agency. It’s just deeply weird, no? Why would you do this? What is it adding to the respondent experience? Was there research done on what consumers want in a survey invite, and did the results actually end up ranking “Strange recipes from other countries” slightly higher than “trivial facts about adorable animals?”

(from another invite:)

(I can’t tell what the original source for this one is, though it’s surprisingly not Wikipedia, although that sentence about the British Isles does appear in the Red Fox entry there.)

Oh, Greenfield. Don’t ever change.

Adblock? Never heard of it!

Here’s what I saw on the first screen of a survey Food and Wine magazine wanted me to take:

(I entered my actual age for the question; I changed it to 99 for the screengrab.)

There’s no “continue” button here. I was initially hopeful that it was one of those speedy flash-based things that would zip me to the next question as soon as I clicked a radio button, but no, nothing happened.

Temporarily disabling Adblock Plus– a Firefox extension that 8 million people used yesterday — makes the page (and its missing button) render properly, as does viewing it in Internet Explorer:

How is it possible that your testing didn’t catch this? How is it possible that you’ve managed to create the only survey I’ve ever seen that can be defeated by the most common advertisement blocking software on the planet? What vacuum are you working in where none of your staff is sophisticated enough to use Adblock?

About six clicks later, after a few questions about three different car brands, I hit a matrix, asking about one of the three brands. Surprisingly, I only got asked the (horribly redundant) questions about this one brand, which was refreshing:

Meh.

Is it me?

Is it just me, or do orange juice brand managers seem more likely than just about anyone else to think their brands have the ability to come to life with distinct human personality traits and characteristics? Is it just one crazy person who keeps using Greenfield for this? Is it the whole industry? Someone help me understand this.

This is at least the third time I’ve seen one of these:

(Also, “warm?” You really want me to think about whether or not any of these refrigerated products could best be described as being “warm?” Because, ew.)

ETA:

OH, COME ON. This is just ridiculous. I’m almost too fatigued looking at this to copy and paste it here, let alone fill it out. In fact, I think the only reason I’m continuing at all is so I can see what fresh horror awaits. How do they expect real people to answer things like this? Oh, right — because they promise us a sweepstakes entry in exchange for 35 minutes of our time. (No, really, they do:)

Anyway, here’s what I’m yelling about now:

Whoa, Two Months?

Crap, I know I’ve been busy, but this is ridiculous.

Still fighting the good fight, but haven’t had time to write about (or even look at) much research lately. I did catch this grid a couple days ago, and I think it’s worth throwing up and looking at, not because it’s a particularly terrible example (it’s sadly just typical), but because I can imagine so many better ways to measure this:

Can’t you picture something with different carmaker logos (or, maybe even better, images of their most popular models) that you can drag up and down or left and right to indicate exactly how likely you would be to consider each of them? And that’s just my very first thought on this one.

Flash makes pretty much anything possible, but we’re still using virtual #2 pencils to fill in virtual scantron bubbles, aren’t we? What do you think?

Best Buy Has Some Issues

From this story at The Consumerist. I’m sure it’s just a browser rendering issue that would have been solved with more testing, but right now, it’s just one of my all time favorite matrixes:

Well done, Best Buy!

What are the Republicans Thinking?

For reasons completely beyond me, I’m on the Republican National Committee’s junk mail list. This morning, I received the strangely bad survey they’ve been taking to try to figue out what they’re doing wrong and what they can do to fix it.

Partisan feelings aside (for the moment), this isn’t a horrible idea; there’s nothing wrong with trying to survey your membership, or failing that, survey people you’ve misidentified as your membership (my hunch is that this dates back to my signing up for McCain’s email list back in 2004 when we all thought he was going to switch parties, but who knows). The problem, of course, is simply that the execution is possibly among the worst I’ve seen. You’d think they’d have asked a Republican pollster to come up with a professionally-written instrument, but thankfully, they didn’t: let’s make fun of it.

Random capitalization aside (more on that shortly), what the heck is up with the “Need to Lead in Congress” choice? If they’re trying to list perceived weaknesses, wouldn’t it need to be constructed differently, like “failure to lead in Congress,” or “lack of leadership in Congress,” or other words to that effect? “Standing up for Principles” isn’t much better — as worded, it’s impossible to tell if the poll is suggesting that Republicans are weak as a party because they stand up for their principles (which aren’t popular enough to help them win elextions) or if they’re saying Republicans are weak as a party because they don’t stand up for their principles enough. Either interepetation could make sense, but the pollster apparently wants us to figure it out for ourselves.

Also, take a look at the preceeding question, which asked what the party’s biggest strengths were:

I’m certain this would be an unpopular view at GOP Headquarters, but couldn’t the case be made that every single one of those items should also be in the weakness box? Isn’t it possible there are some actual Republicans who believe the party’s focus on “values” issues like gay marriage turned off moderate voters who might have been receptive to a message focused solely on fiscal responsibility?

I don’t even know where to begin with this mess, so let’s just leave it here as an example of what a biased poll question looks like.

Why are Those Things Capitalized In That Way?

Two separate problems here, and they’re both fairly common. First, obviously, the “scare quotes,” the use of “wasteful” to modify “government spending,” and the hilarious “Pelosi-Reid Democrats” label — is that supposed to be like “Reagan Democrats” or something? Who identifies with a Speaker of the House/Senate Majority Leader enough that it works as a label? Whatever. The other problem here is very common, and I see it all the time, and I think most people miss it: they’re setting up a question where the respondent has to choose a positive answer choice — “yes,” in this example — if they want to vote for a negative answer choice, “oppose.” This creates just enough of a moment of confusion in the respondent’s mind that it should be avoided, especially because there’s nearly ever a need for it. “Should Republicans in Congress support or oppose the new blah blah blah,” with “support” and “oppose” as the two answer choices — that’s much simpler and avoids that dangerous construction entirely.

This just amuses me — shouldn’t it be “instead of you and your doctor and the non-doctors who make the decisions at your insurance company?”

This was my favorite part, though. (It’s actually the first item in the survey, but I saved it for last.) The email address field was pre filled-in, by the way, and my email address was also in the survey’s URL. That’s one way to make sure no one fills in the “other” blanks with anything you don’t want to hear, I guess.

Pathetic, really.

Also Not About Research: Why Can’t Newspapers Figure The Internet Out?

Looks like I’m adding a couple hundred more words to the millions already written this year about Why The Media is Dying. This screencap from a New York Times article today about famous people hiring people to “ghost twitter” for them:

You’d think those links (or, as I’m sure they call them at the Times, hyperlinks), which appear throughout the story for just about ever well-known person they mention, would go to the corresponding Twitter accounts, right?

Of course they don’t. They go to pages the Times has built that contain links to all the other NYT stories about those people. (And, hilariously, both Obama links go to the same place.)

How can they possibly be this bad at the internet?

There is, as always, a lesson for researchers mixed in: there’s no point using new technology if you ignore the advantages it gives you. Don’t make people re-enter demographic information that’s already in your database; don’t make people select their state from a pull-down menu when you could just let them click on a map. Don’t make people hit a “submit” button when you could just auto-advance once they click their response. Recognize that Flash is pretty much universal by now and there’s really no reason not to take advantage of what it can do.

Don’t be a newspaper.

Meanwhile, in Facebook/NCAA News…

This has nothing to do with bad research.

Bad Facebook, that’s another story.

So one of the basketball pools I’m in is hosted at cbssports.com. I installed their Facebook application because I was hoping it would let me log into the main site through the application, which would let me attach my bracket to my profile. (I’m 8 for 8 so far, but Villanova is making me nervous.) That didn’t seem to be an option, so I went to delete it … but first, I screengrabbed this strange status updater thing that was built into the app:

So, last year, I told a different CBS Sports application that North Carolina was my favorite team; it pulled that info from somewhere and pre-wrote all these status updates for me. Six of the eight pre-written updates insert the name of my favorite team into the update:  all of the six were written as if the name of the team was singular. As a result, all of those updates come out looking like they were written by an idiot.

But here’s the thing:  I can’t think offhand of ANY sports team, college or otherwise, whose name is singular. Tar Heels. Wildcats. Orangemen. Huskies. Banana Slugs. Hoyas.  I might be wrong, but if I’m not, whoever developed this app for CBS has managed to ensure that six of eight of their status updates will never be right.

There’s definitely a lesson of some sort in here for researchers — beware of automated processes that might make you look stupid is probably it — but I’m not really going for the tie-in here — I’m just making fun of some bad, bad Facebook application design.

SSI: Matrixes Make Us Cry, Too.

“We know respondents don’t like grids,” Jackie Lorch, SSI VP, Global Knowledge Management, points out. “They’ve been telling us that for years in focus groups and feedback, but we’ve always thought of grids as a necessary evil in questionnaire design. Now, we’re beginning to learn that not only are grids frustrating for respondents – they actually produce inferior data.”

THANK YOU.

Here’s a link to the full report.

L0t there to digest. What do you all think?

What?

Just answered about four painfully bad screens of questions — a matrix asking me to rank 21 items on a 1-10 scale, and then multiple screens asking me to rank each of those items individually from 1st to 21st most important to me, which is impossible when you really only care about five of the items — it’s pretty impossible to decide in any meaningful way what should be #9 and what should be #19 when you don’t care about any of it — but I eventually got through it and was given this prize:

As far as I can tell, I didn’t do anything wrong. In any case, the “continue” button still existed, so I pressed it and went on — but what a terrible experience for respondents.