Polls – and the Harris v Trump Contest

Some critical thinking is always called for when reading about opinion polls.

In the straw poll of readers on my last post, a majority have said that things will get worse (not better) in Aotearoa New Zealand – although the poll is still open for a few days.

That, of course, was hardly a representative sample. Roy Morgan opinion polls in recent months, however, have reported more people (ranging from 45 to 52 percent) saying the country is heading in the wrong direction than in the right direction. Meanwhile, oddly enough, two recent Taxpayers’ Union–Curia polls showed more people (45 to 47 percent) saying it’s heading in the right direction than those saying the wrong direction.

Obviously there’s also a good number who say they don’t know. Well, it’s not a simple question. The ways in which people respond to it can be subjective, and sometimes they’re influenced by who’s asking or by the questions that have preceded it. Some people may respond to a poll question to make a political point rather than to make a considered judgement. What’s to stop them?

Opinion polling isn’t like counting daisies and dandelions in a clearly marked patch of lawn.

The American polling industry is much bigger and more sophisticated than New Zealand’s. And yet they sometimes get it wrong. As discussed in an excellent article by Pew Research, in the last two weeks before the 2020 presidential election:

“Polls suggested that Biden’s margin over Trump was nearly twice as large as it ended up being in the final national vote tally.”

The polls rightly predicted the winner, but they overestimated the margin between the two contenders. There are two plausible reasons for this:

1. The turnout of voters was not as predicted, so the survey samples didn’t match the actual population of voters.

2. Trump supporters are more likely than others not to respond to surveyors, due to distrust of media and pollsters.

As a general issue, point 1 also explains why polls were inaccurate ahead of New Zealand’s 2020 election with its unexpectedly high turnout. These inaccuracies are openly acknowledged by the American (but not the NZ) polling industry, and people are working on fixing them, but no one promises that this time around they’ll get it right.

As I write, polls are showing Harris ahead of Trump on a national head-to-head basis – by 3.2 points according to 538 and by 1.9 according to RCP. But results are very close in the crucial battleground states, so it’s anyone’s guess as to who will win the majority in the electoral college. For all I know, the polling averages may be inaccurate again.

None of this means “you shouldn’t trust the polls”. Polls are just numbers, and we trust or distrust people, not numbers. It’s in the pollsters’ interests, however, to be as accurate as possible, and so I don’t consider them to be any more or less trustworthy than you or me. The point is that their job isn’t easy, and yet political opinion polls (for better or worse) have become embedded in the culture of democracies.

A problem often arises from journalists misinterpreting or over-hyping poll results. Sometimes this is wilful – for click-bait – and sometimes it’s because a reporter hasn’t studied statistics.

I always urge people to read opinion polls with critical caution. If you want to learn more, I recommend the Pew Research Center article, as it’s written in plain English.

To conclude, here’s a quote from another good analysis of American polls on Brookings:

“Taken as a whole, the current evidence suggests that the 2024 presidential election is likely to resemble those of 2016 and 2020. The winner’s national vote margin will be modest, and the race will be decided by a handful of votes in a few swing states.”