CSIRO and internal peer review

A “peer-reviewed” CSIRO study has apparently found “much stronger public support for wind farms than media coverage of the issue would suggest.”

Naturally a news report like that would demand scrutiny down at Southbank, where Andrew Bolt only yesterday gleefully reported a study from Spain that suggested Spanish wind farms “might” kill millions of birds annually.

Sure enough, Bolt seized upon a detail in that news item that a CSIRO scientist (and deputy director) was “one of the reviewers of the report.”

“Pardon?” Bolt asked. “CSIRO peer reviews its own work?”

Smitham may well be a meticulous reviewer, but having peer reviewers so close to the authors does not seem to me to be a wise way to guard against group-think.

That's a fair point — not withstanding Bolt’s motives in querying this CSIRO study are as predictable as his accepting, on face value, the Spanish bird-kill study.

So, come on Andy! You're a journalist, aren't you? You can contact them to... you know, find out the troooth!

It took me all of half a minute to google that CSIRO project team’s particulars, then a couple of minutes to shoot off an email to the designated contact person asking for clarification. By this afternoon, I had my answer.

Bolt, in an update to his post, seems to have tentatively settled on his own answer, suggested by one of his readers — it was simply a case of "misreporting".

He’s wrong.

I was informed that CSIRO reports are generally reviewed by CSIRO staff who aren't part of the specific project team, but who have expertise in the field, and who will frequently have had experience reviewing work by other scientists outside of the CSIRO. Further external review may be solicited if the work is to be published in external journals.

There it is, Andy — scooped ya! Thanks for the lead.

So yes, it's possible that such a process of ‘internal peer review’ (as my CSIRO contact frankly put it) could make the CSIRO’s work prone to something like “group think”. It would require a rigorous regime of internal checks and balances to address that risk.

If that isn’t already the case, then beefing up the review processes for their published work would certainly improve the CSIRO brand.