Polls apart on climate change

by zebra on 7 December 2010 · 4 comments

Regular Stubborn Mule guest James Glover (@zebra) turns his statistical expertise on some apparently contradictory polls gauging opinions on climate change.

Two polls came out today on the question of whether people believed climate change is real and if so whether it is caused by human activity. The first was a Newspoll published in The Australian and the second was by Essential Media and was commented upon by Essential’s Peter Lewis on ABC’s The Drum. Intriguingly, the Newspoll suggests 73% of Australians believe in climate change with a significant human contribution (so called Anthropogenic Global Warming or AGW). Now The Australian has copped a bit of flak lately for its alleged anti-climate change agenda, but leaving that aside this poll suggests that AGW should be practically a closed book politically as an overwhelming majority believe in it. Essential Media describes themselves as a research tool for progressive political campaigns. Essential’s poll indicates that only 45% of people believe in climate change caused by human activity. In the accepted narrative of such things the results would have been around the other way and the tweetsphere would be apoplectic accusing The Australian and News Corp of once again distorting Newspoll results for their own right-wing political agenda. So what is happening here?

First a note on sampling error. Essential polled 1896 people while Newspoll contacted 1,123 people. For polls where the expected split is approximately 50% a good rule of thumb for margin of error (MoE) is 1/√sample size. In the two polls here this gives MoEs of 2.2% and 3.0% respectively. MoE represents two standard deviations from the sample average so differences of 25% are extremely unlikely (like 10-10 probability unlikely) to be explained by a unfortunate random choice of sample from the general population.

The most likely explanation is that one or both of these polls suffer from an underlying sampling bias. This would be easy enough to generate artificially—just poll people in Newtown if you want to get more people who believe in AGW or in Bob Katter’s seat for the opposite result (is this a little glib? Maybe, but you know I am right). But legitimate pollsters like Essential and Newspoll rely on the rigour of their sampling technique. Especially as every time you publish a controversial result, a large section of the population who disagree with it will accuse you of bias. There are a number of techniques to reduce bias—one is to ask coquestions whose population statistics are well accepted. For example if in your poll you found that 46% of the respondents were female and 54% were male you can readjust the result to reflect the actual population average of 51:49. I assume both polling organisations follow standard methodologies to minimise bias. Often though their actual methodologies are proprietary so question marks remain. A famous political polling agency was well know to always come up with polling results that reflected the political opinions of its founder after “adjustment for bias”.

Some indication that there isn’t an overwhelming bias are some additional questions about voting intention. Here are the results:

Essential Newspoll
Coalition 45% 41%
Labor 38% 34%
Green 11% 14%
Other 6% 11%



I would have to say that the differences in the numbers are on the borderline of being consistent with the MoEs I estimated. In any event the Newspoll which has a higher number believing in AGW has less Coalition voters (though about the same Labor+Green votes). It seems unlikely that the votes for Independents and other alone could account for the 28% difference in the polls on the question of AGW.

So that leaves us with the polls themselves. I have assumed so far that they asked the same questions, but there are major differences. Here are the actual questions and results:

Essential

Climate change is happening and is caused by human activity 45%
We are just witnessing a normal fluctuation in the Earth’s climate 36%
Don’t know 19%



Newspoll

No climate change 18%
Climate change solely caused by human activity 18%
Climate change partly caused by human activity 55%
Climate change not caused by human activity 3%
Believe in climate change but don’t know cause 2%
Don’t know if climate change is real 5%



Now what appears at first to be a headline difference between the polls is more subtle. It is quite hard (I tried) to map the answers between them exactly to compare the results. For example Essential doesn’t ask if the respondents directly if they don’t believe in climate change at all (18% in Newspoll) so presumably the climate skeptics get lumped under “Don’t know” (19%) which will also include those who believe in climate change but don’t know if it is caused by human activity or don’t know if climate change is real. That Newspoll total of “don’t knows” and skeptics is 23%, a bit higher than Essentials “Don’t know” of 19% but within the MoEs as reflected by the voting intention results.

However we can try to compare the two main results which boil down to “Believe climate change is real and human activity is significantly affecting it” of 73% (Newspoll’s headline result combining “solely” and “partly” caused by human activity) vs Essential’s “Climate change is happening and is caused by human activity” of 45%. The difference appears huge. The only thing I can think to explain this is that when not offered the choice of “solely” vs “partly” caused by human activity the Essential respondents threw their lot in with “caused by a normal fluctuation in the Earth’s climate”. In other words the results are consistent if most people who believe that “climate change is real” but don’t believe it is “solely caused by human activity” believe it is “partly caused by human activity” but mostly due to “natural fluctuations in the Earth’s climate”.

What is clear here is that the wording of polls is important and that both polls failed to tease out the subtle distinctions in people’s views on climate change (though Newspoll did a better job of this than Essential). There are also question marks about the sampling bias as shown in the voting intention results. But the headlines of both polls will superficially look like totally different results. And that is a problem when the results are used to support political rather than scientific views on anthropogenic global warming as fact or fiction.

Possibly Related Posts (automatically generated):

{ 3 comments… read them below or add one }

1 Steve December 9, 2010 at 6:27 pm

I agree with your explanation, Sean. I believe climate change is partly caused by human activity (>50% of the change), but I would hesitate to choose Option 1 in Essential’s poll because it seems to force me into agreeing that climate change is caused ONLY by that. I’d probably lean towards Option 3 in that poll as the only option that truthfully reprsents my view.

2 Pertinax December 10, 2010 at 11:07 am

The Australian, following Murdoch’s line, has always strong believed in AGW and that action should be taken.

However, the also follow Murdoch’s line on free market and individual freedom and believe in a rational market based approach and oppose most of the Greens apocalyptic claims and regulatory approaches.

It is quite clear that there are both natural and man-made causes of climate change. The Essential poll is just creating a false dichotomy that it is all natural or all man-made.

3 Graham December 12, 2010 at 8:16 am

“Intriguingly, the Newspoll suggests 73% of Australians believe in climate change with a significant human contribution (so called Anthropogenic Global Warming or AGW).”

Without assuming what the poll “suggests”, consider the actual question.
“Do you personally believe that climate change is partly caused by human activity?” Ambiguous in the extreme. Even if your assumption of “significant” is correct, that term itself is wholly ambiguous. Presumably, even the respected sceptic Professor Bob Carter (author of Climate: The Counter Consensus) may be tempted to answer “yes” or, at the very least to ask “What activity?”, “By how much?” and “Where?”. Even so, this question comes closer than the others to the reality. No wonder the response was so high. (By the way, the figure was 71%, not73%.)
http://www.newspoll.com.au/cgi-bin/polling/display_poll_data.pl

You did not address another Essential poll result that is most pertinent. That is, 76% of Greens believe that “Climate change is happening and is caused by human activity”. And 24% of Greens don’t believe that? With apologies to Abe Lincoln
“You can fool some of the people all of the time, and all of the people some of the time, but you can not even fool all of the lunatics all of the time.”

Leave a Comment

 

{ 1 trackback }

Previous post:

Next post: