A beginning is a very delicate time.


Where and how do we begin?

I’ve been working in the research industry for about fifteen years. My career path did not travel a straight line, and my undergraduate degree is in another field. I have no graduate degree. My professional experience comes under the tutelage of others who similarly were trained outside of the ivory towers of academic polling. I’ve learned and improved by doing. I’m not in any way a math or statistics expert. I haven’t done regression analysis or chi-square tests since college.

To some of you, I’ve just disqualified myself as a researcher and critic. That’s a shame, because you’re probably the most in need of the following advice:

Hire someone who knows how to use words effectively, and let them edit your surveys down to their core. (That person, by the way, probably isn’t currently a researcher. That’s OK. It’s much easier to teach someone how to write good questions than it is to teach them to write in the first place.)  Stop asking the same question six different ways; ask it once — correctly — and move on. Stop abusing your respondents by subjecting them to telephone surveys that go on forever and web panel studies that present page after page of matrices that practically beg respondents to click random choices just to get it over with.

It drives me crazy that so much of the research I see is so very bad. The reasons vary, but the outcome is the same — frustrated respondents and unreliable results.

My purpose here is simple — to display instances of what I consider to be bad research in the hope that we all can learn by example and improve. Some of what we’ll see is flat-out awful; other examples will be less egregious. There also are, of course, researchers who are doing great, innovative work, and I hope to hold some of that up as examples of what we all should be aiming for.

The better we make our questions — the better job we do of survey design — the better our results will be. I hope this blog can become a home for what I see as a movement — the movement to make our research instruments more friendly to respondents, which in turn will make our data better.



Filed under administrivia, general

2 responses to “A beginning is a very delicate time.

  1. Hi

    recently came across your blog and can’t praise you enough for daring to say that the emperor is butt naked. I’ve a similar amount of experience to you myself and a similar background (BA in Geography, learned my craft sitting with Nellie, don’t know a chi squared from a cronbach’s alpha) but am equally incensed by much of what I see and read. While I work for a company that is not immune to this sort of junk I’m in the fortunate position of not personally having clients to kow-tow to and I get the opportunity to speak out at conferences on such issues. Would you mind if I took the liberty of quoting you and the examples you have shared here if and when I can?

    best regards


  2. Absolutely, Pete, please do.

    I’m trying to figure out a way to be a little more open about this without causing any problems at work for myself — I’d love to be able to sit on an AAPOR panel someday on this sort of thing, for instance — but I haven’t figured out exactly how to do that just yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s