Saturday, March 31, 2007

On Her Majesty's Statistical Service

Long-time readers will recall the debate about the Burnham study (usually known as the Lancet study) that estimated excess violent deaths as a result of the US-led invasion of Iraq at around 600,000 as of 2006.. My summary of what I could figure out from reading the methodology and the various criticisms is here.

The British government (along with the Bush administration) was quick to denounce the Burnham study. Blair's office specifically disputed the figures, and British foreign officer minister Lord Treisman attacked its methodology.

Well, it appears that these statements were contrary to the advice Her Majesty's Government was getting from its science advisors. Quoth the BBC:

But a memo by the [Ministry of Defence]'s Chief Scientific Adviser, Sir Roy Anderson, on 13 October, states: "The study design is robust and employs methods that are regarded as close to "best practice" in this area, given the difficulties of data collection and verification in the present circumstances in Iraq."

The new line from the British government is quite a climbdown:

"The methodology has been used in other conflict situations, notably the Democratic republic of Congo.

"However, the Lancet figures are much higher than statistics from other sources, which only goes to show how estimates can vary enormously according to the method of collection.

"There is considerable debate amongst the scientific community over the accuracy of the figures."

Of course, anyone could say that much just from reading the Abstract of the Burnham study. The confidence interval ranged from 426,369 to 793,663. Even if the methodology were perfect, all that could be said is that it is highly likely (95%) that the true figure is between those two numbers.

The other "methods of collection" are really reported deaths. (Like that done by Iraqi Body Count.) That can only give a floor: no one thinks it provides a more accurate measure of actual deaths, and the British government certainly wouldn't in Darfur.

No comments: