Zogby Motorcycle Emptiness

What fresh hell is this? John Zogby has discovered the matrix? And somehow made it uglier than any other matrix in the history of research? All true, and more on that in a moment, but first, this unusual use of the ticky box:

zogby motorcycles

That’s probably illegible unless you click on it, but the gist is I’m being asked, in a pretty wordy fashion at that, which of the following things I’m interested in or knowledgeable about; nothing inherently wrong with that setup, except this: I’m then given exactly one box to check (or not check, as the case may be) : “motorcycles.”

Let’s type it out and count, OK?

John’s Way: (53 words)

“Now for some questions about consumer goods categories. Please choose the categories in which you feel you have a particular interest and knowledge. This means you actively seek out information on these products and services (for example you watch TV shows/read magazines and websites/attend exhibitions/discuss with friends and colleagues etc.)

My Way: (6 words)

“How interested are you in motorcycles?”

Now, it’s possible the reason I only saw the single checkbox for motorcycles is because my answers to the previous series of questions disqualified me from everything else, although I don’t really think my responses would have pointed in the direction of motorcycles,  but who am I to judge? Anyway, as promised, here’s what the first screen of these looked like:

zogby matrix

Maybe I shouldn’t say this is uglier than any other matrix — maybe it’s just that it’s simpler, in that it uses less newfangled HTML and is therefore, I don’t know, easier to access via mobile browsers, which isn’t a bad design goal to have — it’s just very strange looking to me, and pretty hard to take in at a glance. By the time you get down to the last button on the right, it’s not immediately obvious to me if that “1” radio button is for “Pessimistic” or for some other word that I perhaps need to scroll down for, or that’s just not appearing for some reason.

Are mobile/degraded browsers a big factor in the panel research industry? Are there a lot of folks on the Greenfield panel using Netscape 2.0 on Mac IIci’s or something? Because I seriously don’t get why this hasn’t all been replaced — and I’m talking about everyone here, not just Zogby — with some well-designed Flash code. Seems to me a freshman design student could pretty quickly mock up something vastly superior to anything being used in the industry today, no?

One other thing I found interesting: on some (but not all) screens of this survey, when I clicked the final radio button, I was automatically advanced to the next page. Despite the pages having a “continue” button on them, by the way. I don’t have a problem with auto-advancing in general, though I think it needs to either consistently happen 100% of the time or 0% of the time — but I’m curious what y’all think about it. Is the auto-advance, which would make it difficult or impossible to go back and fix an error you made, a good thing or a bad thing?



Filed under answer choices, bad user experiences, Market Research, matrixes make me cry, the web is a visual medium, web research, Zogby

8 responses to “Zogby Motorcycle Emptiness

  1. My guess is that you only got Motorcycles, because it was a multiple category screener and that was the only category left open. But wouldn’t the researcher still ask ALL categories to asses incidence or at least to understand more about YOU, as in, you are interested in Motorcycles AND Sailboats or something?

    Auto-advance. I love it from an aesthetic perspective but yeah, it could irritate me as a respondent to not back up. Odd that they have the auto-advance but DON’T use Flash in any other way…..

    By the way, Flash is tricky to a point in that load times need to be monitored if using it heavily in a survey. You could take a 5 minute survey totally tricked out in Flash and for the respondent, it becomes a 10-minute nightmare of load-time waiting. Just need to use Flash smartly and do testing.

  2. I am not sure that Flash is the best solution, either. Like Brian, I Flash on computers with low memory capacity can be a bear to load — and, likely, people will simply give up.

    The second is that Flash needs to be installed in some browsers (at least Firefox) as an add-on. If someone is on a networked computer or doesn’t have administrative privileges, then they will not be able to take the survey. Sophisticated JavaScript can do just about everything that Flash can do.

    That does not, however, excuse the awful design. HTML can be used elegantly and (I suspect) can provide greater cross-platform support. This is just horrendous.

    BTW, this is my first time to your blog; I really enjoy it.

  3. ubu.roi

    They should have added at least a None option to the motorcycles question. What happens if you are not interested in motorcycles? Can you just pass on?

    As for the design, it’s all because of survey generation software and mass production. You cannot pay undergraduate students or even graduates for that matter, to spit out Flash surveys like if you’re a big company doing hundreds or thousands of surveys per year. At least not today. You need a piece of software that would create surveys based on templates and that is easy to use so that an undergraduate can use it (undergraduates are cheaper than graduates).

    As far as I understand Flash (I’m not a techie), some sort of software that can compile Flash surveys would have to be written in Flash as well. As far as my knowledge goes, there isn’t such a thing commercially available. This is why we don’t have surveys in Flash now. It’s why we don’t have much fancy Javascript either. It’s all about cost cutting nowadays.

    But hey, next year or next month or next week for all I know, someone will come up with a way to put Flash to work on the survey production line and you’ll have your Flash surveys. Until then. the primitive looking HTML surveys will do just fine.

  4. Flash has it’s problems but I’m pretty sure you could come up with some flash generating script in for example PHP. Not exactly sure what the possibilities are for that. (This sounds like an “Ed, this is something you should look in to.”…)

    Problem is though, I guess, that an awful lot of people are still working with IE6, with even JavaScript disabled. I don’t have any data on it myself but I’m guessign restrictions apply even more for surveys and the like because peopel might do it at work where their sysadmin is either lazy or really restrictive in the use of browser-options.

  5. ubu.roi

    Of course you could, the question is: how much does it cost vs the benefits? Right now, the thing that costs the least is doing nothing and carrying on with the old templates/scripts/layouts/whatever.

    So Ed, this is something you should look into…

  6. Hehe, yea, I should.

    I think research showed that dropout ratios and completion times rise slightly when using Flash. But I figure you could counter this a bit by providing a nicer”experience”, especially compared to the HTML example we have here.

  7. Glenn

    …and what about people who are blind or who have poor eyesight?
    They use screen-readers which can cope fairly well with straightforward HTML, but for which Flash content is a meaningless blob.

    Researchers should try to be careful that they don’t marginalize a particular group of people by the mechanics of the way their questions are asked.

  8. Some Guy

    “…and what about people who are blind?”

    I don’t think blind people buy that many motorcycles … 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s