An interesting article by Will Jennings and Christopher Wlezien in today’s Times Red Box pointed me in the direction of their recently published paper in Nature Human Behaviour on the accuracy of pre-election polling. Their conclusion, in a nutshell, is that polls are becoming better at accurately predicting the outcome of elections.

This gave me pause for thought: are polls designed and intended to reflect the outcome of an election? Or are polls about reflecting the views of the population at a point in time?

My hunch is that they are more often designed for the latter purpose. Most polls ask how people would vote if there were an election today. I’m not aware of any polls that attempt to correct for the typical post-election “honeymoon” nor the typical midterm “slump” in their efforts to better predict the next election result.

If my hunch is right, then it’s probably unfair to talk about poll “error” when the results of polls conducted well before elections do not match the election results. More importantly, it puts a different spin on their findings.

Assuming all other things are equal (which they are most emphatically not), then late polls better reflecting the outcome of an election suggests that they are better reflecting the views of voters. Assuming that this increased “representativeness” carries across the election cycle and that polls are measurements rather than predictions, then mid-cycle polls more accurately reflecting the final outcome suggests that the population’s views are becoming more intransigent. (In truth, I’ve no idea whether or not this fits their data, it just seems like it might.)

I don’t know whether that is true or not, but it certainly feels like it might be. I feel like things are reaching a point where people are no longer willing to engage with alternative political views, let alone change their own view. On social media, in particular, I see people who didn’t have had a clearly defined political view five years ago now suggesting that those with differing political views necessarily have malintent. This goes for both sides of the political debate. This never seems a particularly good strategy to me – I don’t think many people have their views changed through the hurling of insults!

The picture at the top is by RachelH_ on Flickr, used under Creative Commons licence.

It’s said that so-called ‘shy Tories’ swung the General Election: those people who are unwilling to admit to pollsters that they vote Tory for fear of a negative response, but who put their cross next to a Conservative candidate’s name in the privacy of the polling booth.

‘Shy Tories’ is basically a shorthand for a particular form of social desirability bias. When asked questions by pollsters, we may be unwilling to admit views that we think might upset pollsters. After all, most people like to please people.

Online polls, so the theory goes, should be less susceptible to social desirability bias because they remove the imposing pollster from the equation. Peter Kellner, President of YouGov suggests:

One of virtues of online research is, or should be, that it allows respondents to submit their views with complete anonymity, as there is no stranger watching over them or listening to their answer.

Or should be. An important caveat given that YouGov, online pollsters, were as wrong in their General Election predictions as telephone pollsters.

So perhaps social desirability bias exists online just as much as elsewhere. Perhaps. And there’s an intriguing piece of evidence from, of all places, Netflix, which described this week putting much greater emphasis on recommending shows based on what people actually watch, as opposed to how they rate individual shows. Todd Yellin, Netflix‘s VP of Product Innovation reports:

Most of our personalization right now is based on what they actually watch, and not what they say they like, because you can give five stars to An Inconvenient Truth because it’s changing the world, but you might watch Paul Blart: Mall Cop 2, three times in a few years … so what you actually want and what actually say that you want are very different.

Is rating something highly because it’s changing the world rather than because you actually enjoyed it a form of social desirability bias? I reckon it probably is. I reckon it’s an attempt to make ourselves feel more educated and ‘worthy’.

And if we fake our Netflix ratings, conning only ourselves, why wouldn’t we equally fake our responses to online polls?