Summary
Based on the ethics application, Lewandowsky’s known views on skeptics and comments he made about “the pause” it appears this project was set up with the expectation that it would show that skeptics changed their views depending what they thought the graph showed. It was probably expected that skeptics would demonstrate a bias by changing their estimation of future trend when informed the graph showed global temperature and it might have been expected that many would say it was cooling.
To test this idea, the groups were randomly split into those told the graph was the share price and another told it was global temperature. They were also asked their views on climate and Lewandowsky then compared those within each group to see how their beliefs on climate was related to their average prediction of the future trends.
It turns out the survey showed the complete opposite effect to that we believe was expected. Skeptics in the group told it was temperature and those told it was share prices had almost the same prediction of the future trend. So irrespective of whether it was shares or global temperature skeptics estimate of future trends were very similar. In contrast the expected trend given by global warming believers differed dramatically between those who were led to believe the graph showed share prices and those led to believe it was global temperature.
This is a very significant finding. It appears to show that those who believe in warming are very suggestible and that skeptics base their views very largely on the evidence and not what they are told the evidence shows.
But the final paper avoided this conclusion by focussing on another point saying that: “even skeptics predict it will warm”. This focus only appeared after the results were available.
To put it bluntly, the paper strongly suggests that people who believe in global warming are more gullible and skeptics are very resistant to false suggestions.
Background
Steve McIntyre has been investigating the ethical application for the Lewandowky NASA conspiracy paper which was originally drawn up for another project. But this project is also dynamite. (also see WUWT)
The original application was submitted on or about 14 December 2009, with the title:
Understanding Statistical Trends
The aims of the project were described thus:
The project seeks to explore people’s understanding of statistical trends in time series data. If we are monitoring a stock price, what do we think will happen to it in the future?
Participants will be shown simple graphs of time series (samples enclosed) and will make predictions about the future trends.
The procedures are expanded upon in a further part of the application (section 9):
Subjects will be shown a number of statistical graphs (3 or 4 at most) that contain time series data. Subjects will be asked to extrapolate the visible trend into the future by indicating their guess of the next most likely values (see enclosed sample).
Some of the trends will be upward, some downward, and most will be presented as fictitious stock prices. The actual data will either be generated randomly or will be the world’s temperature (climate) data collected by NASA (NASA GISS data set).
For some subjects, the climate data will be identified as such whereas for other subjects (chosen at random) they will be presented as stock prices.
And finally:
Upon completion of the graphical task, subjects will be presented with 3 – 4 questions about their impressions of scientific certainty. For example, people will be asked how certain they think sceintists are about the association between emissions and climate change, HIV and AIDS, and tobacco smoke and lung cancer (using a scale from 0 – 100%)
The aim is very clear. The project aimed to understand how participants background affected how they viewed statistical trends. This was going to be tested by asking them to predict the future trend of a plot after one group was told it was stock market data and other that it was global temperature. Then this would be tested to determine whether participants views on issues like climate change affected their perception of the likely future trend.
Results & change in name
But was not what was reported in the results. Instead the result was a paper named:
Popular Consensus: Climate Change Set to Continue
This is a very odd name for a paper intended to find any link between people’s views on issues like climate science and their expectation of future temperature. And there was nothing in the ethical assessment about “contrarian claims about global warming having stopped”. But according to the report this is suggested as the focus of the project:
This study,… suggests that presentation of climate data can counteract contrarian claims about global warming having stopped.
This was clearly not the aim of the study, so what could have led to such a dramatic change in focus. The answer appears to come from the following paragraph:
Notably, although extrapolations overall differed little between presentation formats, people’s perceptions were related to their attitudes only when the data were identi fied as temperatures; however, even for those few individuals in that condition who explicitly rejected AGW, extrapolations were still (just) significantly positive.
Like most people would, when I read this I understood this to be a statement about “contrarians” to the effect that contrarians were more likely to change their view depending whether the data was said to be climate or stock markets.
But this conclusion is clearly false. Although the paper does not give all the relevant figures, there is enough figures in the paper to estimate how the future trend of “contrarians” compares to the predicted trend of other people as shown by the table below:
| Predicted temperature trend | Predicted Share trend | |
| r(96)=0.09 | R(96) = 0.21 | |
| All | 5.51 | 3.66 |
| Neutral or disagree with AGW |
3.75 |
~3 (estimated |
| Disagree (6) | 3.77 | Not Stated |
Discussion
This shows that whilst “skeptics” tended to interpret a temperature graph as having a slightly higher gradient (3.75) than that of shares (3), those who believed in global warming dramatically changed their view of the graph because when everyone was included there was a dramatic change between when told it was temperature (5.51) and when share trend (3.66).
As skeptics would have reduced this figure, it is fairly safe to say that whereas skeptics showed almost no change in the predicted trend irrespective of what they were told the data was, believers must have doubled their predicted future trend depending only on what they were told the graph portrayed.
But this obvious conclusion that those believing in global warming are very subjective and change their interpretation to fit their beliefs in sharp contrast to skeptics who tended to base their views only on the data and not what they were told the data showed, was not only omitted from the paper, but instead it was replaced with a conclusion very strongly suggesting the opposite.
Conclusion
As such not only does this paper show that members of the public who believe in global warming change their perception of the global temperature graph to fit what they believe it shows, but it is also strong evidence that at least some academics are so strongly influenced by their beliefs regarding global warming that (to put the best possible interpretation on their actions) they are “blinded” to obvious conclusion that do not fit their world-view.
skience 
Good job. I think that you are correct that this result is not what Lewandowsky expected. I managed to get a copy of the actual article and it is identical as far as I can see to what you linked to. This is supposed to be a flagship journal! The reporting of results is appallingly poor. Can you digitize the charts and check to see whether the actual numbers are remotely close?
Did you notice the ages of respondents? Did you notice that of the 200 pedestrians approached in a pedestrian mall, “all 200 agreed to complete the task.” I have been involved in surveys for my entire professional life and this response rate is extraordinary. He references another study, did you manage to locate it?
Thanks. To put it in context my area of research at the time I came across this was trying to understand how relative expertise might change the interpretation of graphical plots. Unfortunately this (and a few others which were equally unhelpful like: http://www.psych.utoronto.ca/users/spence/Lewandowsky_Spence_1989.pdf) was all I found. So, whilst I would have preferred to have had a comparison of predictions between the general public and academics I did have groups with differing views on climate which seemed worth examining as it might have given some insight. (But thinking about it I vaguely recollect that a comparison with experts in some form was mentioned in the proposal but isn’t in the report. This may be how I found the paper.)
It was only when reading the paper carefully that I noticed the conclusions didn’t make sense with the reported results.
However, as this paper didn’t focus on my area of research, having understood what I could from the data, whilst I noted the conclusions didn’t match the results, my priority was to continue the research.
It was only when I read McIntyre & other articles that I noticed the ethics proposal he was referring to with regard to the “NASA conspiracy” paper was written for this project. I then realised that the original project proposal was not on the question of whether people expected a rise (as the paper’s title suggested) but instead the project focus had been changed so that it was focussed precisely on the area where the results did not match the conclusion. As the intention had been to look at this area, I realised that omission and/or carelessness could not be an explanation and so this meant that Lewandowsky would have had to act pro-actively in some way to write a conclusion that did not match the results.
At present, there appears to me to be only two possible scenarios:
The first is that Lewandowsky deliberately hid this conclusion and then later knowing skeptics were the group who drew conclusions from the data rather than what they were told the data meant, he attacked them for being “motivated” in their “denial”. There is no way given the results in this paper that one could draw the conclusion that skeptics come to their conclusion through “motivated” denial as the evidence shows exactly the opposite: that skeptics are most likely to base their interpretations on the data.
The second is perhaps even more interesting and more profound in terms of general research: that there is some kind of psychological or perceptual block akin to the “invisible gorilla” such that Lewandowsky was mentally unable to see this conclusion even though the data backing this conclusion must have been very obvious.
skience:
Interesting. Have you tried asking Lewandowsky for his actual data? Also, it struck me that the way the data was presented to the respondents influenced the size of the slope of their projections. An expanded x-axis would produce a visually gentler slope. This obviously would not affect any relative differences in projections of AGW and non-AGW respondents but, I would guess that it would reduce the size of the slope. Any thoughts, since I would assume that the visual way the data is presented is an issue of relevance to this research area?
Given that the purpose of the questionnaire was to test whether there was a difference in future projection depending on what people thought the graph showed, one might have expected a random graph to be compared with the global temperature graph.
It is clear the assumption Lewandowsky was working under was that skeptics were “motivated” to change their view based on what they believe the the graph to show.
So the choice of a share price graph, which is known to have a general upward trend and is associated with capatalist (i.e. energy using) industries seems very important.
He probably assumed skeptics would be “favourable” to capital markets and “hostile” to global temperature, and so he predicted skeptics would predict growth in one and decline in the other.
This would produce a very strong signal and picking a known upward trend graph to compare with global temperature tends to support the idea that Lewandowsky expected skeptics to predict cooling – as this effect would be strongest when compared to an overall rising stock market prediction.
The fact that both group predicted warming explains his title – one suggesting he thought he had found something worth reporting in that “even skeptics” thought that on average it would warm.
In contrast, he could have as easily said: “even believers predict stock markets will rise” – it’s equally meaningless and irrelevant to the stated purpose of the project.
Nice dissection.
PS: “Mac”-worthy!
bernie1815
skience:
Interesting. Have you tried asking Lewandowsky for his actual data?
No, but hopefully as a result of this article someone will.