Obscure AND Potentially Personally Identifying? Let’s Ask It!

Sent in by a reader; click to embiggen:

Bad enough they’re asking for something few people would know offhand — and who wants to go fetch a piece of mail to get the answer  — but I think there’s an equally bad issue here regarding respondent confidentiality, at least theoretically.  A quick search of census data for some five-digit zip codes chosen at random from among those I’m familiar with around the country shows between about 8,500 and 16,000 occupied households in each. (I wouldn’t call that an average, as it’s practically anecdotal, but it’ll do for now, since I can’t find exactly what I’m looking for.)  A zip+4, though, is designed to be reflective of a much, much smaller geography. According to the US Postal Service:

The 4-digit add-on number identifies a geographic segment within the 5-digit delivery area, such as a city block, office building, individual high-volume receiver of mail, or any other unit that would aid efficient mail sorting and delivery

How small are those “geographic segments?” You can use this USPS lookup tool to get a sense of it. I live on a suburban street; my house is on a corner. My immediate neighbor around the corner has a different zip+4; the people across the street have a different zip+4; the house immediately behind me has a different zip +4. The house next door to me, though, and the two houses that follow it going down to the end of the block — those all have the same zip+4 data. Apparently, my personal zip+4 will narrow you down to one of four homes.

Now, presumably, you gave your full mailing address when you signed up for this panel, so it’s not as if the research company) doesn’t already know exactly who you are and where you live — and it’s not as if telephone research doesn’t contain your even more personally identifiable phone number right there in the data — but still, this makes me uncomfortable. Rather than using back-end databases to append that information in post-production (which, for the millionth time, would be the ideal way to deal with this situation), we’re instead outright asking for something that both makes your data pretty easy to tie back to you and which you don’t know in the first place. (I actually thought I knew mine, and I don’t, though I was fairly close.)

All in all, this strikes me as a really bad question. What do you think?

Adblock? Never heard of it!

Here’s what I saw on the first screen of a survey Food and Wine magazine wanted me to take:

(I entered my actual age for the question; I changed it to 99 for the screengrab.)

There’s no “continue” button here. I was initially hopeful that it was one of those speedy flash-based things that would zip me to the next question as soon as I clicked a radio button, but no, nothing happened.

Temporarily disabling Adblock Plus– a Firefox extension that 8 million people used yesterday — makes the page (and its missing button) render properly, as does viewing it in Internet Explorer:

How is it possible that your testing didn’t catch this? How is it possible that you’ve managed to create the only survey I’ve ever seen that can be defeated by the most common advertisement blocking software on the planet? What vacuum are you working in where none of your staff is sophisticated enough to use Adblock?

About six clicks later, after a few questions about three different car brands, I hit a matrix, asking about one of the three brands. Surprisingly, I only got asked the (horribly redundant) questions about this one brand, which was refreshing:

Meh.

R-E-S-P-E-C-T.

I feel like Seth Godin here, talking about permission-based marketing. (Which is worth learning about, if you’re unfamiliar.)

Everything we do, as researchers, is dependent on the goodwill granted us by our respondents. If we abuse that goodwill — if we contact them too often, if we keep them on the phone/online too long, if we ask them too many questions that make them groan in frustration — well, that all costs us goodwill.

Whether our research is product-based or political, we cannot depend on the results telling us how to proceed if our respondents aren’t being truthful with us. If we disrespect them, they’ll disrespect us. If we don’t care enough to make the process of taking the survey pleasurable, why should we expect them to be patient and honest when we tie them up for half an hour asking redundant questions?

If you view your research as something the respondent will endure in order to get their guaranteed $2 or their SweepLand points or whatever the incentive is, you will not only produce sub-par results, but you will continue to contribute to the overall problem of over-used, disengaged respondents.

Shorter surveys. Smarter questions. Respect.

Get to it.

Redundancy is Stupid.

This is such a minor thing, in the grand scheme of everything, but I might as well start somewhere.

The Harris Poll web panel studies are actually generally pretty well done, but they’re by no means immune to bad practices. Recently, I’ve noticed they tend to ask your country of residence … followed by your state … (sometimes) followed by your county … followed by your zip code.

I know I’m just a cantankerous pain in the ass with things like this, but that drives me nuts. If this level of detail is important, why not simply ask me just my zip code, with the option to enter “00000” if I currently live outside the United States? The (presumably) few who live in other countries can then get the country picker (which, incidentally, is correctly implemented, placing the United States at the top of the list. Hopefully, they ocalize this, putting the country the respondent’s IP address suggests they’re most likely located in up at the top of the list, and not always displaying America first.)

Assuming that I do enter a zip code, congratulations: you now know my state and county as well, without having had to ask me three additional questions.

Alternatively, the web being a distinctly visual media, you could always display a world map, allowing me to click on the general vicinity of where I live and then zooming in to let me pick a  more specific area if needed, though I think the single zip code question would be more efficient.