28 Mar UK Science Advisor Backed Lancet Report’s Methodology
Last October, the Lancet released a report by the Johns Hopkins School of Public Health that estimated 655,000 Iraqis had died as a result of the Iraq war. Right-wingers immediately denounced the report, calling it a “fraud,” and even lefty types suggested that the report’s methodology was flawed, leading to inflated figures. Politicians — particularly those with a vested interest in minimizing the death toll — also dismissed the report’s findings. Bush said “I don’t consider it a credible report,” and Blair said through a spokesperson that the 655,000 figure “was not one we believe to be anywhere near accurate.”
As it turns out, the Lancet Report had an unusual — and unusually credible — defender: Sir Roy Anderson, the chief science advisor to the UK’s Defense Ministry:
Chief government advisers accepted as “robust” research that put the death toll from the Iraq war 10 times higher than any previous estimate, new documents have revealed.
[snip]
[A]ccording to papers obtained by the BBC World Service’s Newshour programme under the Freedom of Information Act, senior officials warned the methods used in the survey were “robust” and “close to best practice”.
The survey came up with its findings by comparing mortality rates before and after the invasion. Researchers surveyed 47 randomly-chosen areas across 16 provinces in Iraq, speaking to nearly 1,850 families, comprising more than 12,800 people.
One of the documents obtained by the BBC is a memo by the Ministry of Defence’s chief scientific adviser, Sir Roy Anderson, dated October 13 2006, two days after the report was published.
“The study design is robust and employs methods that are regarded as close to ‘best practice’ in this area, given the difficulties of data collection and verification in the present circumstances in Iraq,” he says.
Another item is an exchange of emails between officials in which one asks: “Are we really sure the report [in the Lancet] is likely to be right? That is certainly what the brief implies.”
Another replies: “We do not accept the figures quoted in the Lancet survey as accurate.” Later in the same email, the same official writes: “However, the survey methodology used here cannot be rubbished, it is a tried and tested way of measuring mortality in conflict zones.”
Anderson’s memo was sent one day after Blair’s spokesperson rejected the Lancet report, so his initial statement was not made in bad faith. Later statements, however, most certainly were. Even after the memo was released, Blair’s advisors instructed him to say “the overriding message is that there are no accurate or reliable figures of deaths in Iraq.” And Blair’s spokesperson rejected the report entirely.
This whole exchange is missing the point. The revealing aspect of the whole Lancet report (plural really, there were two of them of interest) was that they methodology (which was commonly used prior) was shown to be rubbish.
The only reason they could get away with using it in earlier conflicts is there were no body counts to detract from their inflated numbers.
Probably the best part of the entire second Lancet report was where they happily gloss over the fact that their newest analysis completely changes the body count for their previously analyzed period.
The multitude of flaws in the two Lancet reports on Iraq provides sufficient material for several thick articles on the subject… the only mercy of the more recent one is it did at least attempt some minor verification of the individual claims, although the sampling methodology itself was still hopelessly flawed.
Matthew,
Considering the claims you are making and the report in concern, it would be wise to include some links or attribution to to your claim that the methadology was “rubbish.” Seeing as how the Chief Science Advisor to the UK Defense Ministry, and countless statistical experts, would not agree with you, it’s somewhat bold to make a claim without providing any factual basis whatsoever.
I’ll see if I can provide some excerpts regarding it when I return home today. I actually have copies of the two Lancet reports (from prior arguments on this topic) but they’re not on my work computer. There have been a fair number of criticisms of the paper before, one of the more commonly read being Mr. Kaplan’s: From Slate To summarize my (personal) objections to the paper: 1) The methodology is obviously invalid as demonstrated by the necessary omission of the Fallujah data points. The fact that data points had to be removed solely because they gave a physically impossible result demonstrated the random sampling method itself was simply incapable of accurately gauging the casualties. 2) The original report abandoned any verification measures whatsoever, a change that went against the original intended methodology. The results should have simply been discarded at this point, and if the authors weren’t operating almost entirely on a political agenda, they would have. 3) Lancet argued that their results from this method were the actual numbers, when in fact they have almost never matched factual body counts: “Aside from Bosnia, we can find no conflict situation where passive surveillance [used in the IBC] recorded… Read more »
Oops… walked out of the house without it. Looks like this article is about to fall off the main page, but I’ll see if I can post on the matter tomorrow.
I second Matthew’s observations and think the key controversy center’s around the contention of these days attributable “as a result of the Iraq war.” 650,000 bodies makes a big mound in the aggregate, even disbursed over geography and time. The evidence of same has yet to be truly and dispassionately produced.