Skip to content

The problem with political polls

The fact that a number of political opinion polls are completely contradictory to each other seems lost on journalists reporting on many of them.

The fact that a number of political opinion polls are completely contradictory to each other seems lost on journalists reporting on many of them. Since it’s unlikely the breathless coverage of polls won’t end during this federal election campaign, here is some advice when consuming those “news” stories:

A poll is trying to gauge the opinion of 100 per cent of adults (including those who don’t have an opinion). But remember that 100 per cent of people don’t vote, and voting turnout hovers around 60 per cent nationally. But it gets even trickier than that. People of different ages vote in vastly different numbers. Most older (50+ years) people cast ballots; most young people do not. So if a poll’s sample (how many older people versus how many younger ones) doesn’t reflect true voting patterns, have a huge grain of salt sitting nearby. If pollsters won’t release tables that show voting intentions by age, gender, income and geography, be skeptical of what they claim to have found.

Ignore, for the most part, provincial poll results gleaned from a national poll. This is a rule that many journalists don’t know or choose to ignore because it can ruin a good story. The problem with producing provincial “findings” from a national poll is the sample size is so small it is almost worthless to base any detailed analysis on it. The hallowed “margin of error” can be higher than 10 percentage points, which means a party leading by five points in a provincial sample may actually be losing when the margin of error is factored in. Stay away!

Try to ascertain a pollster’s methodology. Is it a telephone poll? If so, is it a computerized one or does it involve an actual human being phoning someone? Or is it an online panel, put together by a pollster who recruits participants?

Faced with new challenges in polling, pollsters have changed their methodology by various degrees in recent years. The days of pure randomness based on telephone numbers – the basis of polling for decades – are long gone. None of this is to say that political polling is pointless or not valuable in some way. Top pollsters such as Ipsos-Reid, Insights West, Angus Reid, Ekos and Nanos provide powerful insights into what’s “trending” in public opinion.

They have uniformly picked up on what appears to be significant growth in support for the NDP nationally, and a decline in Liberal support, with the Conservatives holding fairly steady. But going much beyond those generalizations – such as making a big fuss when a party’s apparent support goes from 33 per cent to, say, 35 per cent in subsequent polls – is a fool’s errand. Yet too many people continue to do just that.

Keith Baldrey is the chief political reporter for Global B.C.