Category Archives: what

LA Times: What the What?

So for a couple of weeks now, I’ve been getting emails from the Los Angeles Times about how my email newsletter subscriptions are about to end. I’ve been ignoring them, because I don’t think I actually get any emails from the Los Angeles Times. I suppose I must have registered with a real email address on their site to read a story once, years ago, before BugMeNot and their Firefox extension made such things unnecessary. In any case, I don’t care, fine, whatever, stop sending me those newsletters you’re not actually sending me, I’ll find a way to survive, despite the longing I shall forever feel in my heart.

Just now, though, I got this brilliant piece of email from them:


It boggles the mind.

Leave a comment

Filed under abject stupidity, bad user experiences, non-research, silly nitpicking, what

Excuse me while I vomit.


Protip: just because you’ve somehow managed to combine some letters to form some words and you’ve placed radio buttons alongside them doesn’t mean you’ve created a poll.

Bonus protip: you can see research that’s just as bad (but generally a lot less offensive) on the home pages of most every local TV station, newspaper, and radio station in America.


Filed under Public Opinion Polling, web research, what

Meanwhile, Deep in John Zogby’s Secret Underground Lair/Science Lab…

I’m not the first one to see this, but that never stopped me before; everyone loves a good dogpile.

The Borg Collective fine people at Zogby Interactive just asked me this interesting set of questions:

zogby chip

What the hell are they working on up there in Utica?


Filed under Market Research, web research, what, Zogby

What are the Republicans Thinking?

For reasons completely beyond me, I’m on the Republican National Committee’s junk mail list. This morning, I received the strangely bad survey they’ve been taking to try to figue out what they’re doing wrong and what they can do to fix it.

Partisan feelings aside (for the moment), this isn’t a horrible idea; there’s nothing wrong with trying to survey your membership, or failing that, survey people you’ve misidentified as your membership (my hunch is that this dates back to my signing up for McCain’s email list back in 2004 when we all thought he was going to switch parties, but who knows). The problem, of course, is simply that the execution is possibly among the worst I’ve seen. You’d think they’d have asked a Republican pollster to come up with a professionally-written instrument, but thankfully, they didn’t: let’s make fun of it.


Random capitalization aside (more on that shortly), what the heck is up with the “Need to Lead in Congress” choice? If they’re trying to list perceived weaknesses, wouldn’t it need to be constructed differently, like “failure to lead in Congress,” or “lack of leadership in Congress,” or other words to that effect? “Standing up for Principles” isn’t much better — as worded, it’s impossible to tell if the poll is suggesting that Republicans are weak as a party because they stand up for their principles (which aren’t popular enough to help them win elextions) or if they’re saying Republicans are weak as a party because they don’t stand up for their principles enough. Either interepetation could make sense, but the pollster apparently wants us to figure it out for ourselves.

Also, take a look at the preceeding question, which asked what the party’s biggest strengths were:


I’m certain this would be an unpopular view at GOP Headquarters, but couldn’t the case be made that every single one of those items should also be in the weakness box? Isn’t it possible there are some actual Republicans who believe the party’s focus on “values” issues like gay marriage turned off moderate voters who might have been receptive to a message focused solely on fiscal responsibility?


I don’t even know where to begin with this mess, so let’s just leave it here as an example of what a biased poll question looks like.


Why are Those Things Capitalized In That Way?


Two separate problems here, and they’re both fairly common. First, obviously, the “scare quotes,” the use of “wasteful” to modify “government spending,” and the hilarious “Pelosi-Reid Democrats” label — is that supposed to be like “Reagan Democrats” or something? Who identifies with a Speaker of the House/Senate Majority Leader enough that it works as a label? Whatever. The other problem here is very common, and I see it all the time, and I think most people miss it: they’re setting up a question where the respondent has to choose a positive answer choice — “yes,” in this example — if they want to vote for a negative answer choice, “oppose.” This creates just enough of a moment of confusion in the respondent’s mind that it should be avoided, especially because there’s nearly ever a need for it. “Should Republicans in Congress support or oppose the new blah blah blah,” with “support” and “oppose” as the two answer choices — that’s much simpler and avoids that dangerous construction entirely.


This just amuses me — shouldn’t it be “instead of you and your doctor and the non-doctors who make the decisions at your insurance company?”


This was my favorite part, though. (It’s actually the first item in the survey, but I saved it for last.) The email address field was pre filled-in, by the way, and my email address was also in the survey’s URL. That’s one way to make sure no one fills in the “other” blanks with anything you don’t want to hear, I guess.

Pathetic, really.


Filed under bad user experiences, election polling, Market Research, what


Just answered about four painfully bad screens of questions — a matrix asking me to rank 21 items on a 1-10 scale, and then multiple screens asking me to rank each of those items individually from 1st to 21st most important to me, which is impossible when you really only care about five of the items — it’s pretty impossible to decide in any meaningful way what should be #9 and what should be #19 when you don’t care about any of it — but I eventually got through it and was given this prize:


As far as I can tell, I didn’t do anything wrong. In any case, the “continue” button still existed, so I pressed it and went on — but what a terrible experience for respondents.


Filed under bad user experiences, Market Research, web research, what

Another Strange Facebook Poll

I’ve edited this image to remove the names and images of the people who posted the items around the “poll.” I left them in just to show the context — the thing just appears right smack dab in the middle of all the normal content generated by my friends.


Weird, right? Also, I live something like 900 miles from Kohler, Wisconsin, and don’t think I’ve ever been closer than 400 miles to it — and that was just to change planes in Detroit.  It’d be one thing if they were asking me about Milwaukee, or Madison, or some other city in Wisconsin that I’d possibly heard of at some point in my life, as opposed to tiny little Kohler, population 1,926, according to Wikipedia.

The results, by the way, seem to suggest that someone is wasting their money on this apparently completely untargeted piece of research:


Very strange. I’d be really curious to know if there were any follow-ups to the 7% who said yes, though I think I’d be surprised if there were.

Leave a comment

Filed under Market Research, social media, what

Wow. Just wow.

You’ll have to excuse the low quality of the image here; I snagged it off a WebEx meeting where the presenter flashed it by as part of a PowerPoint he was sharing. I don’t know the source, I don’t know the rationale, I just know it’s probably going to be the worst matrix you ever see:


Click to see it full size. You might want to print it out and tape it up to remind you to never do this, though I suspect you wouldn’t be reading this if you would.

1 Comment

Filed under bad user experiences, Market Research, matrixes make me cry, quality of responses, The cancer that is killing market research, what