Category Archives: Greenfield

Straightlining vs. Answering Your Stupid Question Honestly

OK, this is something I hadn’t thought of before.

When I’m staring at a bad survey question — asking me to compare two absolutely identical companies in a matrix, for instance — my tendency is to do this:
straightline

They’re equal. There’s no difference between Visa and MasterCard in my mind. Discover and American Express, those are different, both from one another and from these two brands, but Visa and MasterCard might as well just merge, as far as I’m concerned. Of course, there’s no way to provide that answer in the framework provided here, so I decided to simply give each company a score of “5” for each item. That seemed to get the message across, as far as I was concerned. Of course, as soon as I clicked the button, I got booted, with the same generic non-qualified message you get when you tell them you don’t have kids or haven’t seen a movie in the past two months or whatever it is. We all know the truth: they booted me for straightlining.

Which I wasn’t.

At the very least, wouldn’t it be smarter to keep me in and see what the rest of my answers looked like? With the amount of amply-documented badly designed questionnaires out there, shouldn’t we maybe consider that a respondent will occasionally need to do something to get around a poorly framed question, or an item that simply doesn’t apply to them?

Simply ending the survey as soon as someone gives all items on a page the same value seems both too simplistic and too drastic a solution to me.

5 Comments

Filed under answer choices, bad user experiences, data quality, Greenfield, Market Research, matrixes make me cry, web research

Another Fine Matrix

First, look at this full-size. See how there are 14 brands of cat food going across the top? I already told it I’d never heard of five of them, and yet here they all are again. It’s one thing to ask me if I’ve ever heard of a brand and to then, even if I haven’t, show me an ad for the brand and ask if I’ve seen that ad — I very well could have forgotten about it, or misremembered what brand it was for.

This is just stupid…

matrix from hell big

Worse, though, it’s endless. Here’s a reduced-size capture so you can see how long it is:

matrix from hell

This is what I’m referring to in the comments on Gary Langer’ post here — what the hell sort of non-representative person is going to sit through this? This kind of garbage really is the cancer that is killing market research. Stop pulling this crap, and then go and worry some more about probability samples.

3 Comments

Filed under bad user experiences, Greenfield, Market Research, matrixes make me cry, The cancer that is killing market research, web research

Is the panel research business model creating a gold farming problem?

Greenfield must be having trouble getting panelists to complete research these days. Maybe it’s the summer blahs, with respondents too hot, too sweaty, or just too on vacation to be bothered.

Then again, maybe it’s something to do with people just getting sick of trying to imagine their orange juice has come to life and is displaying personality traits.

In any case, I’m sure this is the answer:

50 cents

50 cents per survey! At 20 minutes per survey, that’s like, $1.50 an hour! This will totally solve all of Greenfield’s problems, and can only lead to amazing data quality.

Right?

But let’s turn this repetitive Greenfield mockery into a real question:  what are the odds that this sort of incentive (and incentives in general, really) has already led to or soon will lead to the market research version of “gold farming?”

Gold farming, if you don’t want to bother reading the Wikipedia entry, is an exploit carried out within massive online role-playing games, like World of Warcraft. I’m no expert in it, but as I understand it, people hire low-wage workers (this has apparently been an issue in China) to sit in front of multiple computer terminals logged into the online game. The workers don’t actually play the game as it’s intended to be used, but they instead perform repetitive actions, generally using automated scripts, to earn (or, colloquially, to farm for) in-game cash — virtual money, essentially, that can be spent on in-game items like better weapons and the like. The folks behind the operation then sell the virtual currency online, to actual players of the game who want to buy a really cool sword or whatever but who can’t be bothered spending weeks building up the in-game cash to buy it.

So, since Greenfield is paying 50 cents for 20 minutes worth of human labor here, it occurs to me that someone has probably already figured out that they write some scripts to blast through these things in (let’s say) five minutes each — 12 per hour, as opposed to 3 per hour. And that’s per computer. So you sit a guy in front of five screens, each logged in on a different Greenfield account, each earning $6 an hour — so $30 an hour across those five screens — you know, if your labor only needs to make around $3 an hour, that’s $216 a day in pure profit for the guy in charge. And that’s assuming he’s only got one guy doing this on only five accounts at once.

Now, I’m sure I can’t be the first person this sort of thing has occurred to, and I’m sure Greenfield and the other panel outfits are trying their hardest to make this impossible, limiting the number of surveys one respondent can complete in a day, maybe checking for a total elapsed time and invalidating surveys that move too quickly — but, I don’t know, that strikes me as sort of being similar to making the roads near the bars really wide and straight instead of outlawing 24-hour happy hours, or some similarly goofy comparison.

If we want honest answers from real people, maybe we should rethink this entire insulting “we’ll pay you fifty cents to answer 120 repetitive questions about the minute differences between four brands of orange juice” business model.

9 Comments

Filed under Greenfield, incentives/compensation, Market Research, quality of responses, The cancer that is killing market research, web research

Greenfield: They’re Just Weird. (And why are they plagiarizing recipes from the BBC?)

A typical survey invite email from Greenfield Online:

greenfield is weird

I don’t get it. The recipe, the quotation from Aristotle, the mountain climber — what does any of this this have to do with orange juice? (I haven’t clicked to see what the survey is about yet, but I’m sure it’ll end up being about orange juice. Again. BTW, I just bought a 4-pack of Tropicana at Costco, because it’s what they have, not because I thought it was the most likely to be “outgoing” if it happened to “come to life as a person with distinct personality traits and characteristics.” And just scroll down a bit if you don’t know what I’m talking about.)

So anyway, that’s weird enough, and it was going to be a post on its own, but I figured I’d take a quick look through my in box and see what else Greenfield was using so I could mock them a little more. The very next invite I had included a recipe for monkfish pasta, and here’s where this starts to get pretty funny:

monkfish green

Greenfield is based in the US state of Connecticut. We don’t much use the metric system in America, and yet here’s a recipe that not only includes metric portions at all, but lists them first … and also refers to a type of fish (huss?) I’d never before heard of, as well as to  “double cream,” which despite sounding delicious, does not exist here. At all.

That all seemed odd enough that I copied and pasted the first line of ingredients into Google, and guess what the first hit was?

This:

bbc2

I’m no lawyer, but it’s probably legal for Greenfield to include this recipe, especially as it seems to be coming from the “Sea Fish Authority,” which seems to be an official UK agency. It’s just deeply weird, no? Why would you do this? What is it adding to the respondent experience? Was there research done on what consumers want in a survey invite, and did the results actually end up ranking “Strange recipes from other countries” slightly higher than “trivial facts about adorable animals?”

(from another invite:)

greenfield fox

(I can’t tell what the original source for this one is, though it’s surprisingly not Wikipedia, although that sentence about the British Isles does appear in the Red Fox entry there.)

Oh, Greenfield. Don’t ever change.

4 Comments

Filed under ethics, Greenfield, Market Research, web research

Is it me?

Is it just me, or do orange juice brand managers seem more likely than just about anyone else to think their brands have the ability to come to life with distinct human personality traits and characteristics? Is it just one crazy person who keeps using Greenfield for this? Is it the whole industry? Someone help me understand this.

This is at least the third time I’ve seen one of these:

another juice grid

(Also, “warm?” You really want me to think about whether or not any of these refrigerated products could best be described as being “warm?” Because, ew.)

ETA:

OH, COME ON. This is just ridiculous. I’m almost too fatigued looking at this to copy and paste it here, let alone fill it out. In fact, I think the only reason I’m continuing at all is so I can see what fresh horror awaits. How do they expect real people to answer things like this? Oh, right — because they promise us a sweepstakes entry in exchange for 35 minutes of our time. (No, really, they do:)

35 mins

Anyway, here’s what I’m yelling about now:

minute maid COME ON

6 Comments

Filed under answer choices, bad user experiences, Greenfield, incentives/compensation, Market Research, matrixes make me cry, web research

Sweet Fancy Matrix

This one, from the nameless partner of a nameless partner of Greenfield the Matrix King, may just take the cake.

Maybe it’s just that it’s late at night and my glasses are dirty and my eyes are tired, but this is incredibly off-putting to me:

badmatrix

I especially enjoy the “100% Fruit Juice In Any Flavor” column, and the way each word is on its own row.

I’m going to try to persevere and get through this one, mainly because I think it’s sure to get worse as I go, but if it weren’t for you, my loyal readers, I swear I’d just to to bed.

In all seriousness, though: who decided that this was the way to determine what consumers truly think?

3 Comments

Filed under bad user experiences, Greenfield, Market Research, matrixes make me cry

Greenfield Solves the Wrong Problem

So, I see Greenfield Online has gone and added a “security” question to their matrices to weed out people who pay no attention whatsoever. This, of course, is a predictable response to the somewhat misplaced industry obsession with the quality of online survey responses.

Oddly, though, they’ve gone and inserted this into a survey where I’m personally finding the questions — even in this matrix format — to be pretty interesting. For once, I’m not feeling like this is a tedious waste of my time:

greenfield_mortgage

Leave a comment

Filed under Greenfield, Market Research, matrixes make me cry, quality of responses