It’s not exactly the best time to be releasing an opinion poll. In the wake of the 2019 election, there are fair questions about why we poll any more. But today we launch the Lowy Institute’s annual poll and it is still deeply revealing about Australian attitudes on foreign policy.
Before we look at some of the findings, first on polls in general.
The election polls were wrong. And not just within the margin of error wrong, as some have defended, which is true in particular cases but not true across the country.
What went wrong? There will be a review, and it will be hard to say – but we know that polling is getting harder. Here are a few (non-exhaustive) thoughts.
1. The samples weren’t sufficiently representative
Traditional polling has been living on borrowed time since the decline of the landline telephone. Polling 101 says this: you need a representative sample, and you need the people in the sample to tell you the truth. A sample is truly representative if all members of the population have a chance of being included in the sample and that their chance of inclusion is known.
Now that a White Pages isn’t representative of most of the country, pollsters are using a variety of methods including blending and “weighting” data from potential voters that they reach via mobile, fixed line telephones, and online. The confidence we have in the representativeness of the sample is therefore reduced significantly. Because fewer people answer their phones, it is cheaper to use robo-polling (computers asking questions by telephone), but this may introduce additional biases. Pollsters are constantly tweaking methodology, and traditional weightings protocols can struggle to keep up.
2. Polls are at risk of being contaminated by conventional wisdom
Some commentators suggest that election polls are affected by “herding”. That is, pollsters tend to not publish results that differ too much from what the other pollsters are saying.
There is safety in numbers – and conventional wisdom said that the Coalition could not win the election. To the extent that no one wants to be an outlier, this can lead to polls all being wrong in the same direction.
3. Election polls do not generally publish the undecided votes
When the two-party preferred is published as 49–51, the data is actually something closer to 44–46 with 10% undecided. Pollsters that don’t otherwise account for the undecideds, assume that they will vote with a similar split to the rest of the poll. That is a significant proportion of the electorate about which to make assumptions, when elections are won and lost by a few points.
So there are challenges that election pollsters are dealing with. But here’s the thing. Issues polls – such as the Lowy Institute Poll – are different.
For starters, it is more affordable to ensure your sample is representative when you only poll once a year, rather than every few weeks. We use the only probability-based (equal chance of being represented) panel, Life in AustraliaTM, at significant cost but with confidence in the methodology and consequently, the result. Most of the large international firms also use panels of this nature.
There is less incentive for issues polls such as ours to align their results with other findings in the field. The Lowy Institute Poll is currently the only opinion poll focused solely on foreign policy in Australia. There is little in the way of conventional wisdom for us to adjust to, and no attempt to do so.
The Lowy Institute Poll repeats many questions from previous years, allowing us to track trends over time, rather than relying on a single result in a single year.
And possibly most importantly, we disclose all of the “don’t know” responses. We do this not only to be transparent and accurate, but also because for our purposes, “don’t know” can point to a lack of awareness, knowledge or engagement – all of which are important for policymakers to be aware of.
For us, it’s the trends that matter most.
The Lowy Institute Poll repeats many questions from previous years. This allows us to track trends over time, rather than relying on a single result in a single year. Small differences in results (49:51) don't change the interpretation of trends in the way they change the outcome in an election poll. So 49% of Australians say this year that foreign interference in Australian politics is “a critical threat” to Australia's interests, and 51% say the same thing about a severe downturn in the global economy. The two percentage points between those propositions are insignificant because they are within the margin of error: what really matters is that concern about foreign interference has risen eight points in one year, whereas concern about the global economy is stable.
Polls still have interesting stories to tell. People are asking what happened to the “climate election”. The 2019 Lowy Poll shows the highest levels of concern about climate change in the past decade. But it also shows falling levels of optimism about our economy. We can see how this may have played out in different electorates.
The Lowy Institute Poll gives Australians a voice on these pressing issues that will have an effect on them. In this sense, the Poll is democratic: and this poll finds 65% of Australians think democracy is preferable to any other kind of government. To dismiss public opinion polling entirely would be to dismiss public opinion.
