Dean's World

I wish I'd noticed this sooner: a PDF copy of a The Wegman committee's report on the 'Hockey Stick' analysis on recent global climate change.

This entire report needs to be read in full, but here is an important section, with some highlights in bold by myself:

"The Chairman of the Committee on Energy and Commerce as well as the Chairman of the Subcommittee on Oversight and Investigations have been interested in an independent verification of the critiques of Mann et al. (1998, 1999) [MBH98, MBH99] by McIntyre and McKitrick (2003, 2005a, 2005b) [MM03, MM05a, MM05b] as well as the related implications in the assessment. The conclusions from MBH98, MBH99 were featured in the Intergovernmental Panel on Climate Change report entitled Climate Change 20013: The Scientific Basis. This report concerns the rise in global temperatures, specifically during the 1990s. The MBH98 and MBH99 papers are focused on paleoclimate temperature reconstruction and conclusions therein focus on what appear to be a rapid rise in global temperature during the 1990s when compared with temperatures of the previous millennium. These conclusions generated a highly polarized debate over the policy implications of MBH98, MBH99 for the nature of global climate change, and whether or not anthropogenic actions are the source. This committee, composed of Edward J. Wegman (George Mason University), David W. Scott (Rice University), and Yasmin H. Said (The Johns Hopkins University), has reviewed the work of both articles, as well as a network of journal articles that are related either by authors or subject matter, and has come to several conclusions and recommendations. This Ad Hoc Committee has worked pro bono, has received no compensation, and has no financial interest in the outcome of the report."

"In general, we found MBH98 and MBH99 to be somewhat obscure and incomplete and the criticisms of MM03/05a/05b to be valid and compelling. We also comment that they were attempting to draw attention to the discrepancies in MBH98 and MBH99, and not to do paleoclimatic temperature reconstruction. Normally, one would try to select a calibration dataset that is representative of the entire dataset. The 1902-1995 data is not fully appropriate for calibration and leads to a misuse in principal component analysis. However, the reasons for setting 1902-1995 as the calibration point presented in the narrative of MBH98 sounds reasonable, and the error may be easily overlooked by someone not trained in statistical methodology. We note that there is no evidence that Dr. Mann or any of the other authors in paleoclimatology studies have had significant interactions with mainstream statisticians. In our further exploration of the social network of authorships in temperature reconstruction, we found that at least 43 authors have direct ties to Dr. Mann by virtue of coauthored papers with him. Our findings from this analysis suggest that authors in the area of paleoclimate studies are closely connected and thus ‘independent studies’ may not be as independent as they might appear on the surface. This committee does not believe that web logs are an appropriate forum for the scientific debate on this issue.

It is important to note the isolation of the paleoclimate community; even though they rely heavily on statistical methods they do not seem to be interacting with the statistical community. Additionally, we judge that the sharing of research materials, data and results was haphazardly and grudgingly done. In this case we judge that there was too much reliance on peer review, which was not necessarily independent. Moreover, the work has been sufficiently politicized that this community can hardly reassess their public positions without losing credibility. Overall, our committee believes that Mann’s assessments that the decade of the 1990s was the hottest decade of the millennium and that 1998 was the hottest year of the millennium cannot be supported by his analysis.

Recommendation 1. Especially when massive amounts of public monies and human lives are at stake, academic work should have a more intense level of scrutiny and review. It is especially the case that authors of policy-related documents like the IPCC report, Climate Change 2001: The Scientific Basis, should not be the same people as those that constructed the academic papers.

Recommendation 2. We believe that federally funded research agencies should develop a more comprehensive and concise policy on disclosure. All of us writing this report have been federally funded. Our experience with funding agencies has been that they do not in general articulate clear guidelines to the investigators as to what must be disclosed.

Federally funded work including code should be made available to other researchers upon reasonable request, especially if the intellectual property has no commercial value. Some consideration should be granted to data collectors to have exclusive use of their data for one or two years, prior to publication. But data collected under federal support should be made publicly available. (As federal agencies such as NASA do routinely.)

Recommendation 3. With clinical trials for drugs and devices to be approved for human use by the FDA, review and consultation with statisticians is expected. Indeed, it is standard practice to include statisticians in the application-for-approval process. We judge this to be a good policy when public health and also when substantial amounts of monies are involved, for example, when there are major policy decisions to be made based on statistical assessments. In such cases, evaluation by statisticians should be standard practice. This evaluation phase should be a mandatory part of all grant applications and funded accordingly.

Recommendation 4. Emphasis should be placed on the Federal funding of research related to fundamental understanding of the mechanisms of climate change. Funding should focus on interdisciplinary teams and avoid narrowly focused discipline research.

There's a good deal more there, much of it very worth reading. They did far, far more than look at a couple of papers, which I see that some in the paleoclimatology community have tried to sugggest. No, they actually looked at those papers and at dozens of related papers, and found almost all of them wanting. They further did an extensive mathematical analysis of the social networks involved in those many papers.

Certainly, they slam Mann and his colleagues for misrepresenting data, sloppy math, and worse. Which is a big deal all by itself, since Mann is one of the most frequently cited researchers in the entire field. Another damning quote:

While the works do have supplementary websites, they rely heavily on the reader’s ability to piece together the work and methodology from raw data. This is especially unsettling when the findings of these works are said to have global impact, yet only a small population could truly understand them.

Making conclusive statements without specific findings with regard to atmospheric forcings suggests a lack of scientific rigor and possibly an agenda.

The social network analysis of authors' relations suggests that the 'independent reconstructions' are not as independent as one might guess.

At bare minimum, damning accusations have been levelled at Dr. Mann, and by extension, just about everyone associated with him--who turn out to be dozens of important people who've co-authored papers with him, or conducted peer review on his work.

This, again, from research that was a core part of the IPCC report telling everyone in the world--important politicians and the general public--that catastrophic global warming was real and probably human-caused and required extensive and very expensive public policy changes to address. All of it put together by the same tiny little social network of equally self-interested researchers, with two or three cliques pretty much at the center of everything (with "clique" being mathematically and precisely defined by the Wegman group, no less!)

I have to say that I found the response "RealClimate" guys (probably the most prominent bloggers representing the orthodox climate change position in the blogosphere) to be--well, disappointing, to put it charitably. I searched their entire web site for "Wegman," and found only this frivolous response. It's got to be the most thin-gruel defense against a damning indictment that I've ever read. They even fail to point to the actual Wegman group report, choosing instead to focus on Dr. Wegman's much more limited personal testimony to Congress.

Frankly, here's what it looks like to me: "Hey, what's that behind you?!" [zoom away while backs are turned.]

I was literally aghast at the Wegman group's report. It makes it clear that only a tiny handful of researchers are at the center of most research and most public policy recommendations on climate change, and that practically no one outside this tight little clique-ridden community is in charge of reviewing their work. They all simply review each other's work--and now literally dozens of papers in the field, along with general practices and procedures in the field, have been independently reviewed and found deeply flawed.

Worst of all, although the Wegman report does not say this openly, anyone who knows how taxpayer funding of science recognizes this (and it is all over the Wegman report by inference): Practically all the taxpayer funding for this climate research, much of it clearly shoddy, is controlled by this same Good Ol' Boy Network with practically no independent review, who simply "peer review" each other in a not particularly anonymous way while they dole out each other's grants and approve each other's papers.

Which is all, by the way, pretty much exactly how Dr. Richard Lindzen has described it. (More on Lindzen here.)

Worse, when I've seen the "RealClimate" blog guys try to answer these charges, they throw around words like "conspiracy" and try to laugh it off--changing the subject to "conspiracies" instead of acknowledging the real words: "Clique," and "Good Old Boy Network in the guise of peer review." Other useful terms instead of "conspiracy" might be "professional ego and reputation" and "conflict of interest among the closed group of people who 'anonymously' (wink, wink) control each other's funding and access to publication."

I happen to agree with the Wegman report that weblogs are not the place to hash out climate change research on the scientific merits. But you know what? On the issue of public policy? On the issue of how my tax dollars are spent? On the issue of demanding greater transparency and accountability? On asking for a thorough independent investigation of allegations of scientific bullying and isolation of dissenters in the so-called "peer-review process" that awards those government grants? Oh, I think those are all entirely fair discussions to have on weblogs, and among all members of the public.

I'd like to see which scientists in the Climate Change research community are willing to step forward and talk bluntly about those issues--and do more than assure us that "hey the system's not perfect but it works I assure you" or "no conspiracies here, haha, move along now."

These people not only take in millions of taxpayer dollars, but they're treated like high priests who are allowed to ask for some of the most massive and sweeping public policy changes on the globe. There is not a damned thing wrong with asking them to answer hard questions for the taxpayers and the general public. And not just about the research, but about ethics and standards and ethical practice and independent review and isolation of dissenting ideas that might be professionally inconvenient for some researchers.

I'd like to see a thorough response to the actual Wegman report, and not vague armwaving generalizations, "oh, that's boring, nothing to it, can we do something more interesting now?" No, boys and girls. If you don't want the dirty, dirty politics in your science, stop taking the dirty, dirty taxpayer money, and stop making public policy recommendations. Otherwise, it's time for a lot more questions to be answered.

I am now more convinced than ever that Richard Lindzen and others like him need to be listened to.