See Editor’s Note Introducing this Feature
On the heels of the Copenhagen climate talks – whose scant accomplishments reveal that climate change science may be no match for international politics – the Intergovernmental Panel on Climate Change (IPCC) finds itself in a scientific controversy of its own making.
The IPCC Fourth Assessment Report’s malformed paragraph on Himalayan glacier melt has prompted intense, and warranted, criticism of the IPCC review process. This criticism has come not only from climate science skeptics or contrarians. It’s generally clear that the ungrammatical, internally contradictory two sentences – which reproduce errors found in improperly cited sources – shouldn’t have made it into the first draft of the report, much less the final.
The IPCC now has recanted the paragraph in question. Though the widely quoted claims were in print for nearly three years, the IPCC’s admission does indicate that scientific errors can be publicly identified and corrected.
But the errors don’t end – or begin – with the IPCC report. A careful look shows a complex set of conflations and misquotations begun by some science journalists more than a decade ago, transmitted and compounded by members of the IPCC Working Group II writing team, and hopelessly muddled by hasty, confused press coverage.
As Though A Pervasive Curse Haunts Accurate Coverage
Dozens of articles and analyses of this situation, whether dashed-off blog posts or New York Times coverage, exhibit a curious consistency. Not a single article or analysis appears to include all relevant issues without introducing at least one substantial error. It’s as though the original documents contained a curse which has spread to infect every commentator and reporter. The curse seems to stem from not reading sources carefully (or at all), which, ironically, was the IPCC Working Group II’s central failing, and also a major issue in the documents that were the basis of the defective paragraph.
For a good, brief, but cursed summary of these events, see Canadian geographer Graham Cogley’s letter to Science. Cogley, who was an important early investigator of this issue, gets a lot right, but – the curse! – he does not mention that an Indian environmental magazine was likely the source that the IPCC copied and pasted.
Having identified “the curse,” one runs the risk of falling prey to it. The authors here vow to respond quickly to correct any error in this account.
Considering the sources carefully and in context clarifies some aspects of the story and complicates others. An extended investigation of this controversy leads to a more nuanced understanding of the tremendous – and frequently unacknowledged – challenges of reading and reproducing electronic information in our globalized world. Heightened awareness of these fault-lines may enhance the ability of scientists, policymakers, and journalists to communicate about climate change. This analysis also reveals that IPCC scientists and science journalists can learn from each others’ best practices. Indeed, such cross-pollination could play a vital role in strengthening both communities.
The Defective Paragraph and its Foundational Errors
Here is the relevant paragraph from the IPCC report:
Glaciers in the Himalaya are receding faster than in any other part of the world (see Table 10.9) and, if the present rate continues, the likelihood of them disappearing by the year 2035 and perhaps sooner is very high if the Earth keeps warming at the current rate. Its [sic] total area will likely shrink from the present 500,000 to 100,000 km2 by the year 2035 (WWF, 2005).
Media coverage has focused largely on the lack of scientific support for these claims. But three major errors can be spotted immediately, without consulting the IPCC’s sources:
1. The first sentence predicts disappearance (a 100 percent loss) by 2035. The next sentence predicts an 80 percent loss. Nonetheless, the first prediction is made using more confident language.
2. The second sentence begins with “Its,” ungrammatical if it is referring to “glaciers” and unclear otherwise. It’s as if the two sentences were simply copied and pasted from different sources.
3. The approximate area of the Himalayan glaciers is 33,000 km2, so the 500,000 km2 starting figure in the second sentence is off by a factor of 15, and the decreased area predicted in 2035 – 100,000 km2 – is three times greater than the current Himalayan glacier area.
David Saltz, an IPCC reviewer, spotted the first two errors before publication (as discussed below), but they were not corrected. The third error has been under-discussed. But where did it come from? Quick source checking reveals that the 500,000 km2 claim does not appear in the WWF report, discussed below, which the IPCC cites. A more elaborate search reveals that the sentence originated in V.M. Kotlyakov’s paper Variations of Snow and Ice in the past and at present on a Global and Regional Scale (published in 1996 but apparently written in 1991), which states:
The extrapolar glaciation of the Earth will be decaying at rapid, catastrophic rates – its total area will shrink from 500,000 to 100,000 km2 by the year 2350.
As numerous sources have pointed out, Kotlyakov was referring to all extrapolar glaciers, and suggesting that they would decline by the year 2350, not 2035.
Climate Science Watch succinctly shows how this error – indeed, how the whole IPCC paragraph – reproduced two fundamental errors introduced in “Glaciers Beating Retreat,” Mridula Chettri’s April 30, 1999, article for the reputable Indian environmental magazine Down to Earth. Climate Science Watch republished Chettri’s paragraph and highlighted the identical IPCC text in bold. However – the curse again! – it neglected to bold a key phrase. We have underlined what it should have bolded.
Chettri’s article, which, again, is from Down to Earth magazine, April 30, 1999, reads:
Glaciers in the Himalaya are receding faster than in any other part of the world and, if the present rate continues, the likelihood of them disappearing by the year 2035 is very high,” says the International Commission for Snow and Ice (ICSI) in its recent study on Asian glaciers. “But if the Earth keeps getting warmer at the current rate, it might happen much sooner,” says Syed Iqbal Hasnain of the School of Environmental Sciences, Jawaharlal Nehru University, New Delhi. Hasnain is also the chairperson of the Working Group on Himalayan Glaciology (WGHG), constituted in 1995 by the ICSI.
The glacier will be decaying at rapid, catastrophic rates. Its total area will shrink from the present 500,000 to 100,000 square km by the year 2035,” says former ICSI president V M Kotlyakov in the report Variations of snow and ice in the past and present on a global and regional scale (see table: Receding rivers of ice).
Chettri’s writing gives the impression that she is quoting Syed Iqbal Hasnain and the ICSI separately. But Hasnain was not just the chairperson of the WGHG, as Chettri notes: he also was the sole credited author of the ICSI report she mentions. One source has been split into two. The report does not contain anything resembling the quotation that Chettri pulls from it. Moreover, it was never published, suggesting that Chettri credited to the report something she thought she heard in the interview with Hasnain.
The report was apparently scanned and put online by glaciologist Georg Kaser at the request of Graham Cogley, the Canadian geographer mentioned above. (According to metadata in the PDF, it was scanned on 11/23/2009.) Kaser, a lead author for IPCC Working Group I, says he alerted the IPCC about the erroneous paragraph in 2006, before the Fourth Assessment Report was published. Left unclear is why WG II was consulting popular science magazines instead of top IPCC glaciologists? Working Group I chapter 4.5, for example, contains detailed documentation on world-wide glacier melt.
It is unclear where Chettri got the “Receding rivers of ice” table for her article in Down to Earth (see below). No such table appears in Kotlyakov’s 1996 paper, and although Table I in Hasnain’s unpublished 1999 paper has the same goal – indicating the speed of glacier melt – it is substantially different. Of the eight glaciers mentioned by Chettri, only one (Ponting Glacier) has the same dates and retreat figures as in Hasnain. The other glaciers either do not appear at all or appear with different dates or figures. Chettri gives no source for her data, and she clearly did not simply just mechanically copy from Hasnain’s unpublished paper or rounding his figures: for example, Pindari Glacier is given a retreat of 2,840 meters in Chettri and 3,000 in Hasnain over the same time period. Hasnain also mentions several glacier studies that Chettri does not.
However, the IPCC authors seem to have directly copied Chettri’s table from the Down to Earth magazine article. Compare Chettri’s table to Table 10.9 in IPCC Section 10.6.2:
The IPCC’s table is almost identical to Chettri’s – a fact which has been largely missed in the discussion so far. The glacier names, the headings (except for capitalization), the dates, the figures, and even the (arbitrary) order are the same, with only one exception: a second study on Gangotri Glacier has been added. This study is credited in the IPCC text as Hasnain (2002).
As mentioned above, the final version of the IPCC document also cites a 2005 WWF report. The WWF report has been intensely discussed since this mistake became clear. Its Table 7 (on page 32), however, is quite different from the IPCC’s. It does not include total retreat figures, only average retreats per year. To its credit, it also includes sources for each claim. All but two glacier studies in the WWF report either have different dates in the IPCC table or do not appear at all. Small details confirm the difference: the WWF spells Bara Shigri as Bada Shigri (both transliterations from Hindi are permissible), and puts a hyphen in Chota-Shigri. The only data identical in both tables is Hasnain’s 1985-2001 study of Gangotri, although the WWF cites a 2004 paper, not the 2002 paper mentioned by IPCC.
More strikingly, the IPCC table contains a simple mathematical error found also in the Down to Earth article, but not in the WWF report. This error is like a radioactive tracer back to Chettri’s article, as it could not likely have been made twice. The 1845 to 1966 Pindari Glacier study appears in all three tables, but only the WWF report correctly divides the 2,840-meter retreat by 121 years to arrive at the actual yearly rate of 23 meters. Both the Down to Earth and IPCC tables divide 2,840 by 21 years instead of 121, leading to an incorrect, much faster retreat rate of 135 meters per year.
Quite a curse. But the January 24th Daily Mail article reporting the division error is itself cursed – it misattributes the error to the WWF report, which actually gives the correct figure for Pindari melt. (It’s possible that the error was silently corrected, but a January 19 copy of the WWF report, with the original March 9, 2005, creation date, has the 23 meters-per-year figure.) Basing the IPCC claim only on an advocacy group’s report would have been inappropriate even if the report had in fact been the source of the data. The IPCC’s citation error compounds the paragraph’s problems, confusing numerous reporters who haven’t looked at the sources.
So what do we know? Despite the incorrect citation to the WWF 2005 report, two numerical quirks – the 2350-to-2035 switch and the division error – and various other features of the language and presentation are lifted directly from Chettri’s Down to Earth article. Since, as we’ll see, the WWF 2005 citation did not appear until the final draft of the report, one possible scenario is this:
(1) An original author copied and pasted the text from Chettri’s Down to Earth article. Perhaps it was never intended for final publication; it might have been working notes that lost their citation and then got incorrectly placed in the draft document.
(2) When a second-round reviewer asked that the section’s citations be improved, a different IPCC author, processing the comments, noticed that this section was unattributed and tried to find the source.
(3) The citation to the WWF 2005 report appeared at this point. The WWF report, in turn, cites “Flooded Out,” a New Scientist article from 1999, which has received far more attention than Chettri’s piece. The author of that article, Fred Pearce, recently told the Australian that he had read the Chettri article, and though he did not cite it, he appears to have been substantially influenced by it. The second-round IPCC author, perhaps in an unprofessional rush, may have noticed the subject matter similarity between the WWF and IPCC tables and simply added the citation.
The scenario above is speculation. Much of the media coverage – including reporter Andy Revkin’s summary on DotEarth – has argued that the IPCC authors were drawing upon the WWF 2005 report and the New Scientist article. Some even claim the IPCC looked at the 1996 Kotlyakov paper that contained the 2350 figure. Neither explanation appears satisfactory, in light of the clear differences between the WWF and IPCC tables and the verbatim similarities between the Chettri article and the IPCC report.
Questions remain about the Pearce and WWF pieces, however, since they also reproduce the 2035 claim attributed to (and now disclaimed by) Hasnain.
In a January 13, 2010, reflection on the article he had written more than a decade ago, Pearce said he had interviewed Hasnain via e-mail. But almost every news story written since then claims that Pearce’s interview was by telephone. If still available, the record of that 10-year-old e-mail could resolve this inconsistency. Further, Pearce’s January 13th article is multiply cursed – it excoriates the IPCC for citing the WWF report but does not mention the Down to Earth article (even though Pearce had used it as a source himself), does not note that the 500,000 km2 figure was not actually in the WWF report, and does not report that the table in the WWF report does not match the table in the IPCC report or include the division error.
It seems that none of the journalists who have cited Pearce’s account of the controversy have read the original sources in question. If papers like The New York Times and the Sunday Times had adequately source-checked, they would have realized that the story was more complicated than Pearce’s account suggests.
Mistakes Made During Review, and Echoed in Media
Despite what has been regularly reported in the media, IPCC’s formal policies do not prevent it from quoting non-peer-reviewed literature in certain cases. As the IPCC has acknowledged, the sequence of steps required to do this was not followed for the paragraph in question.
However, another corrective process was not only followed but is documented online. The entire IPCC Fourth Assessment Report underwent a formal review, and with sufficient attention to detail, one can reconstruct several elements of how this process failed in the case of section 10.6.2.
There were no comments at the first draft stage related to the errors in 10.6.2. One reviewer said that the glacier retreat table discussed above should be removed, but gave no explanation; the writing team said this was an “irrelevant editorial comment” and did not remove it.
The second draft, however, generated twelve comments from experts and four comments from governments. [Click here for a PDF analyzing every comment made on section 10.6.2 and of how the IPCC writing team responded.]
That analysis reveals that this section was as poorly revised as it had been written. The writing team responded to eight comments by simply indicating that revisions had been made. However, in five of those cases, the final text was unchanged, and in another case one issue was corrected but another was not. Twice, reviewers asked that unclear terms be explained; the writing team did so in their responses to the comments, but did not change the terms in the actual text.
For example, Section 10.6.2 cited Table 10.9, discussed above, to support the claim that “[g]laciers in the Himalaya are receding faster than in any other part of the world.” Table 10.9, however, contained only data on Himalayan glaciers, and therefore could not support this claim. One reviewer noticed this discrepancy and asked for tabular data on non-Himalayan glaciers. The writing team responded with “Revised the section,” but no changes were made for the final draft.
Other comments deal with citation issues. Hayley Fowler, a reviewer from Newcastle University, suggested that some glaciers in the western Himalaya may actually be expanding as a result of climate change, which both IPCC Working Group I and another section in Working Group II discussed. Fowler suggested adding three references to 10.6.2. She also strongly recommended a 2005 paper from Nature which underscored the urgency of glacial melting in the Himalayas and worldwide: “[i]t appears that some areas of [the Himalaya-Hindu Kush region] are likely to ‘run out of water’ during the dry season if the current warming and glacial melting trends continue for several more decades.” Although this paper would have substantially strengthened Section 10.6.2, the writers did not include it either. Their comment about all three papers: “Was unable to get hold of the suggested references will consider in the final version.” (Working Group I, and the authors of WG II Section 10.2.4.2 must not have had the same trouble, as both contain at least one of Fowler’s suggested cites on glacier expansion.)
Notably, another reviewer complained that 10.6.2 had only one reference. The writing team responded to this with “More references added,” and the final draft of 10.6.2 did contain two more. One of them was the incorrect WWF 2005 citation mentioned previously.
The government of Japan had two comments questioning the probability of glacier melt. These were important points, since the IPCC uses tightly controlled language to indicate its authors’ assessments of probability and certainty. (This is a major aspect of the Fourth Assessment Report, which grapples with the problem of presenting uncertain probabilities to policymakers who demand certainty. Thus, the phrase “virtually certain” means that the IPCC authors believe what they are describing has a 99 percent or greater probability of occurrence, “very likely” indicates an assessment of a 90-99 percent probability occurrence, and so on.)
The writing team responded to both comments with the statement “appropriate revisions and edits made.” One “appropriate revision” involved sticking the word “likely” before the word “shrink” in the second sentence of the paragraph – language which does indicate a probability assessment. However, the phrase “likelihood … is very high” in the first sentence, which, unlike “very likely,” does not indicate a probability assessment, was unchanged. It may not appear obvious to some that “very likely” and “likelihood … is very high” have such different meanings, but this is a detail of IPCC procedure missed by both Elizabeth Rosenthal of The New York Times and Jonathan Leake and Chris Hastings of the Sunday Times. Rosenthal’s article improperly quotes Section 10.6.2 as containing the phrase “very likely,” an error the newspaper has not chosen to correct. Leake and Hastings get the language right but the analysis wrong. They write: “When finally published, the IPCC report did give its source as the WWF study but went further, suggesting the likelihood of the glaciers melting was “very high”. The IPCC defines this as having a probability of greater than 90 percent.” They were wrong – the IPCC in that case doesn’t use the term “very high” but rather “very likely.”
Some comments were even more detailed. As mentioned above, one reviewer, Saltz, questioned the ungrammatical use of “Its” and wondered what it referred to. The IPCC team responded “glaciers” but did not update the final text. Saltz also pointed out the glaring internal contradiction in the paragraph: “100,000? You just said it will disappear.” The IPCC team responded “Missed [sic] to clarify this one” and didn’t revise.
While these two concerns were ignored, a third suggestion of Saltz’s – that an alarmist-sounding sentence be cut – was accepted, so the questionable paragraph did undergo at least some correction.
Learning Lessons from the Himalayan 2035 Curse
All in all, both the IPCC and media coverage seem to have fallen off the log when it came to important details.
The offending paragraph about Himalayan glacier retreat was widely quoted in the media, without their first having adequately checked the sources used by IPCC. Perhaps this same tendency explains why most blogs and newspapers have reported incorrect and fragmentary versions of the IPCC’s errors in this case.
We expect a great deal from the IPCC process, and, at least as far as Section 10.6.2 goes, those expectations were not met. How much can we also reasonably expect from serious science journalism?
The curse can be vanquished. Perhaps both scientists and science journalists can learn from the best practices of each other’s professions. IPCC professional copy-editors could double- and triple-check facts, especially when expert comments raise questions. Moreover, IPCC could better append corrections to the original text where needed. (While the IPCC has retracted the paragraph in question, one can still download 10.6.2 without any indication that it contains an erroneous paragraph.)
Finally, experienced journalists are comfortable talking with both natural and social scientists. This controversy reveals a disconnect between IPCC Working Group I (which got the glaciology right), and Working Group II (which allowed the erroneous paragraph to slide).
Furthermore, science journalists and their editors, notwithstanding the economic pressures facing the media, need to resist inevitable temptations to base major conclusions on single sources of information without sufficient verification, which Chettri, Pearce, the WWF, the IPCC, The New York Times, the Sunday Times, and many papers quoting them have all done in different ways.
Ultimately, there is a common lesson for both scientists and the media: the need to drill down to original sources. This extra effort is vital in reporting on such complex and critical issues: It could help avoid future runaway quotations – like the claim that the Himalayan glaciers would disappear by 2035 – and enable science, the media, and society to focus on real environmental problems, such as glacier melt which continues around the world.
The IPCC’s Himalayan glaciers mistake in the end can encourage stricter editing, closer scrutiny, and more transparency in the review process. In that case, the mistake will have served a valuable function.
Increased attention to primary scientific literature may help avoid future errors and also serve as a reminder that the IPCC process often tends towards conservative statements. It’s also important to remember that the science is constantly being updated. Consider, for example, the recent finding that the IPCC models may have systematically overestimated the ability of the biosphere to grow in response to increased carbon. If this research proves right, the IPCC’s long-term temperature projections for the world may be a full degree Centigrade too low, and controversies now commanding headlines will recede into history.
When the IPCC errs, it’s not always in the same direction.
Bidisha Banerjee (bidisha@yaleclimatemediaforum.org) is a Masters candidate at Yale School of Forestry & Environmental Studies. George Collins (george@yaleclimatemediaforum.org) is a joint degree candidate at Yale Law School and Yale School of Forestry & Environmental Studies.
From the Editor
Drilling Down: An In-Depth Examination
Of IPCC’s Himalayan Glaciers Mistake
The IPCC, and by extension the world’s responsible climate science community these letters have come to symbolize, is under attack like never before.
The criticisms this time result from mistakes, errors, and shortcomings of IPCC’s own doing. The difference now is that the criticisms aren’t coming solely from the vocal, but comparatively small, legion of climate contrarians and skeptics. This time, the criticisms are coming from within and from beyond IPCC, and from many who could fairly be considered impartial observers.
Color them now trending toward skeptical, in the best sense of that word.
IPCC’s allegiance to the established scientific method has made it a bastion of credibility and respect in the field. It’s just that standing that some now are calling into question.
Some see this episode as an example of science righting itself through its ongoing self-correcting process. In that case, the events surrounding the Himalayan-glaciers-gone-by-2035 fiasco might in the end prove to be a worthwhile learning experience, something from which positives – and not just negatives – can be gleaned.
When, after all, is the last time you heard of the most determined contrarians publicly rethinking and correcting their positions?
That said, timing is everything. In basketball, ballet, music, politics, and climate science. And the timing in this case could hardly have been worse.
The appropriate, understandable, and necessary brouhaha over the IPCC’s handling of the melting Himalayan glaciers conclusion comes in the wake of the comparatively far less serious (in terms of impact on actual climate science, if not popular perceptions) hacked e-mails fiasco. And of the disappointments with the Copenhagen climate summit. And of the shrinking prospects for significant legislative action on climate change from a weakened congressional leadership still reeling from the loss of its “veto-proof” 60-vote majority, a testy electorate, and nagging economic and unemployment woes.
But all these eventually can and do have a cumulative impact, certainly on the public perception of things, and therefore also on political and policy responses.
It’s just what any serious doctor would not prescribe, just when an already ailing political process needed it least.
How the climate science community – IPCC and the thousands of scientists on whose shoulders its work depends – responds is critical. First, we must fully understand exactly how and why the mistakes about the Himalayan glaciers occurred, keeping in mind that there is still no doubt that the world’s glaciers indeed are at heightened risk in our warming world.
Bidisha Banerjee’s and George Collins’s comprehensive analysis helps to answer the important “How did this happen?” question.
The next question involves how IPCC addresses flaws in its procedures to prevent recurrence of such a mistake. In doing so, it can maintain its standing as the science community’s, and indeed the world’s, most authoritative voice on climate change.
The world is watching. IPCC must act quickly to repair the damage and return the world’s attention and focus to the real story – the causes and consequences of, and potential solutions for, anthropogenic climate change.
The world’s glaciers, after all, aren’t sitting by idly waiting. For them, the clock is ticking.