Probability Sampling vs. Web Panel Sampling vs. Interviewing People Completely By Accident

A reader sent me this a number of months ago, and embarrassingly, I’m just getting around to posting it now, as I’ve been reminded by something @Lovestats posted the other day.

Another matrix with multiple problems — “I felt expected,” really? But the focus here actually isn’t the matrix, believe it or not: it’s that my reader has never actually stayed in the hotel that sent him the survey, with the familiar “please rate your recent stay with us at our such-and-such location” sort of email you tend to get after any interaction these days. Not only had he not stayed at that location — he’d never stayed at any hotel in the chain.

Now, I’m assuming this was a glitch of some sort — an actual customer with a similar email address had sloppy handwriting, or a friend of our reader has been giving out the reader’s information instead of his own, or the hotel company is a conglomerate and they actually meant to ask our reader to rate his experience buying a shirt at their clothing store  — but however it came about, it made me think — with all the ongoing discussion about probability samples versus non-probability samples, what about starting by just making sure you’re not interviewing people who are totally outside your frame?


Filed under Market Research, matrixes make me cry, web research, worrying about the wrong thing

2 responses to “Probability Sampling vs. Web Panel Sampling vs. Interviewing People Completely By Accident

  1. I quite like the grammar of this question. 🙂

    “Thinking about when you arrived at the hotel, how satisfied are you with the staff made sure the room type booked met my expectation.”

    Awesome, high quality survey writing here! Good for a chuckle, bad for the market research industry.

  2. I love the way it just totally falls apart after the second item. It’s as if once they locked themselves into the 1-10 scale with that introduction, they were powerless to change anything following to something that made more sense and used half the words, like “Did the staff make sure you liked your room?”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s