300: What’s In A Number?

Ira Glass

In her two years in Baghdad, Hannah Allam lost over 10 close friends. She was dragged from her hotel in the middle of the night by Iraqi police. She was caught in an attack on a shrine. And things got so bad and lasted so long that she called her mom on her cell phone, thinking she might not make it out of there.

Then a few weeks ago she took a break from her job as Baghdad bureau chief for the Knight-Ridder Newspapers and game home to the Midwest.

Ira Glass

It's hard though, soldiers are risking their lives, contractors are risking their lives, she and her staff of reporters are risking their lives, and it's not even for something people are paying attention to? Captain Chuck Ziegenfuss has seen some of the same things now that he's back in the states. Here's the commander of Charlie Company 234 armor in Iraq. He was blown off the road by an IED in June 2005. Shrapnel in his legs, arms, and face. He woke up two weeks later at a hospital in the states. He says that Fort Riley in Kansas where he lives, so many people have been sent overseas. It's a military community, there's no escaping the reality of the war.

Ira Glass

Take the body count. It's been in the news all this week because the number of American soldiers dead in Iraq passed 2,000.

Ira Glass

It seems like every day we hear these numbers and facts come and go, car bombings, more deaths. All really fast, without a lot of explanation. And today on our radio program, we just try to slow down, and understand better one particular number from Iraq, and what it means, really. From WBEZ Chicago it's This American Life, I'm Ira Glass.

In act one of our show we're going to talk about a number and a quick news story that nobody really seemed to take very seriously at the time that it happened and what was really behind it. In act two, the Air Force drops a bombs on the wrong target in Sunni Iraq and accidentally kills 12 people, including children. An army captain has to go apologize and try to make things right. A journalist was with them and recorded this very difficult moment. Act three, what do we do with these numbers anyway? Stay with us.

Ira Glass

Act one, Truths, Damn Truths, and Statistics. The number we're going to talk about right now is not how many Americans have died in the Iraq war, but how many Iraqis have. The fact is we have no idea how many civilians have died as a result of the war. Nobody counts. Not the military, not the State Department. The Iraqi Ministry of Health for a good while early on in the war, was compiling morgue figures from across the country, and making them public every week. But that practice was stopped. These days, the place that most people go when they need a figure is a privately run, nonprofit website called Iraqbodycount.net. It gets its figures by going through newspaper articles and other press accounts and simply counting the number of people reported dead in those articles.

Even the people who started the site and who run it today freely admit that this method gives you a huge undercount. At best it's a minimum. The true number could be much, much higher. One of the producers of our radio show, Alex Blumberg, started looking into all of this. And he found something surprising and disturbing about the death figures, and what we know about them. Here he is.

Alex Blumberg

Eveyone will tell you, counting civilian casualties in wartime is hard. First of all, you need to do something called a large-scale mortality study. And second of all, you need to do it in the middle of a war zone. To date in Iraq, there's only been one attempt. It was a Johns Hopkins University Study published in The Lancet, a British medical journal, in late October 2004, a couple of days before the US presidential election.

It concluded that probably 100,000 Iraqis had died as a result of the war. This figure was astonishingly high. 10 times higher than any casualty estimates at the time. Even today, a year later, with all the extra deaths that have happened in that year, no other estimate comes close. Just this week, the New York Times ran a story based on the Iraq body count website that's estimating civilian casualties at a fraction of that number, just 30,000.

Since The Lancet study's figures were so high and the study itself got almost no traction in the press, I remember thinking at the time it came out, that it's probably bogus and slanted. I'm guessing a lot of people, if they even heard about the study, felt the same way. But recently, in trying to figure out how many civilians have died in the war, I've learned more about The Lancet study. And the more I learned about it and the remarkable story of how it was done, the more likely it seems that the 100,000 is actually the best estimate. And if anything, low.

Alex Blumberg

This is Les Roberts, the lead author on The Lancet study and one of a handful of scientists in the world who could be called, an expert in counting war dead. In the Congo study, he found that 1.7 million civilians had died from the war, a figure cited by Colin Powell when he was secretary of state, and Tony Blair on the floor of the British parliament. Les has also done studies in Burundi, Rwanda, and Sierra Leon.

To a guy in Les Robert's line of work, the war in Iraq had a number of unique and interesting things that deserved study. The main thing that distinguished this work was that the military took unprecedented care to avoid civilian casualties. Almost 2/3 of the bombs dropped were precision guided, as compared to just 8% in the first Gulf War, and 0% in World War II. They limited daytime strikes and avoided civilian infrastructure, like power and sewer plants. Compare that to World War II where American forces fire-bombed entire cities as part of the military strategy, killing up to 100,000 people in Tokyo alone, and upwards of half a million civilians in Europe, and you can see why George Bush called the Iraq War one of the most quote, "humane military campaigns in history."

But Les knew that often it's not bombs and bullets that kill people in war, it's the other things that happen when society falls apart. Clean water and medical supplies get scarce. In a lot of the studies he did in Africa, diarrhea killed more people than weapons did. Women can't get to the hospital to deliver their babies, so infant mortality rates go up as well.

It took Les a long time to get to Iraq and see if the same things were happening there. First, an Iraqi doctor who he'd plan to work with died in an auto accident, another social ill, by the way, that tends to increase during wartime. And then insurgent violence spiked. It wasn't until August of 2004, five months after he'd originally planned to go, that he finally landed in Amman, Jordan. He had $24,000 in foundation money in his pocket, his passport, and a letter of invitation from the Iraqi Ministry of Education. He found a driver, a retired Iraqi army officer named Wahid, who agreed to take him to Baghdad.

Problems started at the very first checkpoint, on the border between Jordan and Iraq.

Alex Blumberg

Fortunately for Les, Wahid was something of a Han Solo figure, an unenthusiastic, but talented smuggler who didn't look for trouble, but didn't run away from it once it found him. He talked his way through that first checkpoint.

Alex Blumberg

Les finally made it to Baghdad, where he met for the first time in person, his Iraqi co-researcher, the man with whom he'd be working for the next month. His name was Riyadh Lafta and he was a doctor of community medicine at Al-Mustansiriya University in Baghdad. Riyadh had hired a team of researchers, mostly doctors from his university. All of them native Iraqis, but fluent in English.

Let's pause here a moment to talk about their methodology. Because when their study came out later, a lot of people wanted to believe that it was flawed or biased. In fact, the survey team used a standard methodology for measuring health and mortality over a geographic area. It's called a cluster sample survey and it works like this.

Using the most recent census figures available in Iraq, the team made what was essentially, a map of the population. They then used a random number generator to pick 33 points on that map. Baghdad was the biggest population center, so it got several points by itself. But the other points were spread all over the country, from the Kurdish north to the Shiite south, from small towns to big cities.

Once they'd picked a town though, the team still had to figure out who to interview there. Here again, they worked hard to leave everything to chance. Using GPS units, they would drive around the outskirts of the town and store the coordinates, creating a rough outline of the town border. They would then generate a random point within that border, drive to it, and interview the 30 nearest households. It was such a commitment to random sampling that the first few times the team did it, even the researchers Les and Riyadh were working with, found it obsessive.

Alex Blumberg

The survey went smoothly, at least for the first couple of days. People, it turned out, were much more willing to answer questions. Even to provide death certificates as verification, than the researchers had initially thought they'd be. In fact, the trouble, when it came, came from Les himself.

Alex Blumberg

Les watched from the car as the police took the two researchers-- both doctors, one a dignified man in his 50s and the other a single mother-- and drove them away. He was terrified that somehow the police would find out that they were working with him, an American. But he could do nothing but sit in the parked car and hope no one discovered him.

Alex Blumberg

Riyadh and Les decided that for everyone's safety, he should lay as low as possible. So for the next 16 days straight he didn't leave his hotel. To pass the time he crunched the numbers that the survey teams were calling into him every night.

The surveyors were getting basically two pieces of information from each household. How many people in that household had died in the 14 months before the invasion, and of what, and when? And how many had died in the 17 months after the invasion, and of what, and when?

By the time the teams had completed their 32nd out of 33 clusters, over 900 households, and over 7,000 people, the results were pretty shocking. The death rate itself had gone up about 60%. Large increase, but one that Les had expected from his other surveys. The shocker was how people were dying. For the first time in any of his surveys, the leading cause of death wasn't disease, it was bombs and bullets. In the 32 of the 33 clusters sampled, 21 people died of violence, as compared to just 1 violent death in the period before the war.

There was a second shocker. Of those 21, 2 people died in firefights where it was unclear where the bullet came from. 3 were killed by insurgents or Saddam loyalists. 7 died from criminal violence, carjackings, revenge killings, that sort of thing. And the biggest number, 9, were killed by the American-led coalition.

Alex Blumberg

At the end of three weeks, there was only one more cluster to survey. The team had saved it for the end because it was the most dangerous one: Fallujah.

Remember, this is September 2004, insurgents control the city and it's basically under siege from the coalition. They're shelling it regularly.

Alex Blumberg

Only one other interviewer agreed to go to Fallujah with Riyadh, a doctor who had relatives there he wanted to check up on. Their car was stopped three times on the way into the city. Heading to their random spot, they saw devastation everywhere. Houses were bombed, rubble lay in the streets. The block they stopped on was no different. They had to visit 52 households to get the requisite number of interviews. 23 homes were either temporarily or permanently abandoned. Neighbors said that in the abandoned houses, most people had died. But this data couldn't be substantiated, so it wasn't even included in the survey results.

In the 30 households they did survey, there were 53 deaths. 52 of these were violent deaths, all but one caused by coalition weapons. 24 of the people killed by coalition bombs and bullets were children under 12 years old. And with that, the survey was over.

Alex Blumberg

This is Fox Evening News October 28, 2004, on the day the results of Les' survey that just shy of 100,000 Iraqis had died as a result of the war were released. Les had not even considered the Fallujah data in coming up with this number. Fallujah had so many deaths, it was too much of a statistical outlier to even include.

Fox never mentioned the study, neither did ABC or CBS. The only national network that carried the story was NBC for 21 seconds.

Alex Blumberg

Morning Edition and All things Considered on NPR, devoted 45 seconds to the story. And it didn't make the front page of the New York Times, the Washington Post, or any national newspaper. The Iraq study had provided information about the war that up until that point, no one had been able to provide. The number it gave was much higher than anyone would have expected. It was just as accurate as Les' previous studies in Africa that he'd done using the exact same methods and which were widely reported in the press and quoted by lawmakers. His Congo study was page one in the New York Times.

The only differences with the study were that he'd risked his life to do it and it was about Iraq. Which, if anything, should have made it more interesting to the media. So why didn't it get any press?

Partly, it was the timing. The study came out five days before the US election, and so the media was pretty preoccupied. Plus, there was a suspicion that the team had timed the release of the survey specifically to influence the election, a suspicion that Les didn't really help dispel. He said to an AP reporter about the study quote, "I emailed it on September 30 under the condition that it come out before the election. My motive in doing that was not to skew the election. My motive was, that if this came out during the campaign, both candidates would be forced to pledge to protect civilian lives in Iraq. I was opposed to the war and I still think the war was a bad idea, but I think that our science has transcended our perspectives. As an American, I'm really, really sorry to be reporting this."

One desk editor at a national news organization told me that when the study came out he sent an email to one of his colleagues saying, "The Lancet had in the past, published some studies with a political slant, but that this study seemed sound and maybe they should report on it." Then he saw Les' comments and he didn't follow up. This is exactly the type of story that those who believe the media has a liberal bias love to pounce on. And so, in essence, if the research turns out to be flawed, this desk editor's organization gets the heat for it. He had a very small window of time during a very busy news cycle to decide whether the study was legit or just an angry and easily debunked researcher pushing an agenda. And Les' comments seemed to be all the evidence he needed.

And there was one other thing that made it easy for the media to dismiss the report. A researcher at Human Rights Watch, who himself had done studies of civilian casualties during wartime said he didn't believe the study. The researcher's name was Marc Garlasco, and he told a reporter for the Washington Post quote, "The number seems high to me." And quote, "It seems like a stretch."

Alex Blumberg

This is Marc Garlasco. He said he told the reporter from the Post that he hadn't read the study, but the reporter said he really needed a quote and could he just respond to the number. Garlasco's quote was cited elsewhere and he appeared on CNN, although none of the study's authors were interviewed on CNN, or any of the major networks. Here's what Marc Garlasco says now.

Alex Blumberg

OK, let's just stop here for one minute. You heard what he said. He'd taken part in the targeting for the war. Get ready, because this story's about to take a turn. Marc Garlasco isn't your typical human rights advocate.

Alex Blumberg

Marc doesn't see moving from the Pentagon to a human rights nonprofit as the 180 degree flip most people might. He says all he's ever wanted to do was fight bad guys, and both organizations do that, just in different ways. He'd been thinking about leaving the military before the war began and he hadn't supported the war himself. But he stayed through the fall of Baghdad because he knew the targets better than anyone else. And he figured if there was going to be a war anyway, might as well be him targeting the bombs rather than someone else who might not know or care as much as he did.

The thing that finally prompted him to leave, he says, didn't have anything to do with the war. His wife got a great job offer at the Bronx Zoo and they'd always wanted to move back to New York. When Marc went with Human Rights Watch to Iraq, it wasn't to get a comprehensive count of civilian casualties. His mission was to look at specific attacks and see which kinds of attacks caused high civilian death tolls. Because Marc had planned many of the strikes he was now going to investigate, it was a little complicated.

Alex Blumberg

The attack had hit the intended buildings, but it also destroyed the two neighboring buildings. That's where the man's family had died. Also, Chemical Ali hadn't been in the targeted building anyway. It's unclear who died there.

Marc went to lots of places in Iraq he'd studied on maps and aerial photographs and heard about from defectors. And there's no way around this, after all those years of imagining these places, what they must be like, it was exciting to actually be there.

Alex Blumberg

The military denied my request to talk on the record about civilian casualties. But Marc Garlasco says that civilian casualties are one of the primary factors he and his colleagues considered when planning the war. Say he had a target he wanted to take out. The headquarters of the Iraqi Secret Service maybe, or one of Saddam's palaces. He'd work with the weaponeering guys to figure out how many and which bombs to use. And then--

Alex Blumberg

How was that magic number arrived at, do you know?

Alex Blumberg

A lot of times when the collateral damage assessment came back too high, they would try to get it lowered. For example, a strike Marc planned early on in the invasion. An Iraqi division was holed up in a big, multi-building convention center in Baghdad. Which unfortunately, was right across the street from a hospital.

Alex Blumberg

What got Marc thinking about civilian casualties in the first place was a battle damage assessment he did after the war in Kosovo. He targeted the bombs for that war and then afterwards the military sent him over to see how well he'd done. He measured how often the bombs hit their targets, whether they'd destroyed what they were supposed to destroy. Pretty much the only thing he didn't check the accuracy of were the collateral damage assessments.

Alex Blumberg

In talking to people in the military off the record, I heard a couple of arguments against counting civilian deaths. First they say, it's not the military's job. If what you're trying to do is win a battle, it could be a dangerous, and in the long run, counterproductive distraction to worry about counting all the civilians you accidentally kill along the way. Second, and perhaps more persuasively, they say no one would believe them anyway. Just ask Les Roberts.

Even though Les' study didn't get much mainstream attention, it did provoke, like so many things these days, a bitter debate on the internet. The attacks came mainly, though not exclusively, from right wing blogs. Several charges leveled that the study were simply untrue and seemed designed to willfully muddy the waters.

For example, there was a claim repeatedly made, both in the press and online, that the data weren't random because the researchers had been blocked from going certain places, or had decided against certain places because it was too dangerous. This is simply false.

A couple people suggested that the researchers had gone to Fallujah on purpose to boost their numbers, even though exactly the opposite was true. Les had wanted to skip Fallujah altogether and they hadn't even included the data in their final casualty estimate.

Several objections had merit, though. First of all, the study makes no distinction between combatants and civilians. Les actually acknowledged this in the study itself and went to great lengths not to claim, as others did on his behalf, that the study was a measure of civilian mortality. Certainly, some of the people the coalition killed they intended to kill. But half of all the casualties were women and children, so even in the unlikely event that 50% of the men who died were actually fighting us, it's still a large number of innocents.

The critique that got most traction on the internet though, has to do with something called the confidence interval. Let's take an election poll as an example.

If candidate X is projected to receive 55% of the vote, what that really means is that he's projected to receive some percentage within two numbers. Let's say 52% and 58%. That range is called the confidence interval. The confidence interval in Les' survey was very wide, between 8,000 and 194,000. It was this wide for a lot of reasons, but mainly because the sample is relatively small relative to the population, and because violent death, unlike death due to malaria or diarrhea, isn't very uniformly distributed. So you have Kurdish areas where mortality actually went down during the war versus Fallujah, which averaged almost two violent deaths per household. Such a wide confidence interval means that statistically speaking, Les' estimate of 100,000 dead isn't very precise. The number could be thousands or tens of thousand smaller. Or equally likely, bigger.

But a lot of people made wrong conclusions from the wide confidence interval. They interpreted it to mean it was just as likely that 8,000 people had died as it was that 100,000 had. The online magazine, Slate, wrote "This isn't an estimate, it's a dart board." In fact, the likelihood follows a bell curve, with 98,000 being at the top of the bell, the most likely number. So actually, there's only a 2.5% chance that the number is 8,000 or below, but a 90% chance that it's 44,000 or above. Here's Les.

Alex Blumberg

Clearly, the people on talk radio weren't attacking the study out of a commitment to experimental rigor. They were attacking it for the same reason that the news media was hesitant to report it. Because the very act of counting civilian casualties is political. The moral logic of war is this. We're willing to undergo X number of costs in lives, money, resources, to accomplish some goal. The goal we hope will be worth it in the end. So assuming the goal in Iraq is good, is it wrong to kill 100,000 civilians?

Saddam, himself, probably killed 230,000 of his own people. A number, by the way, no one seems to go out of their way to dispute. If you add the million or so lives he lost in the futile war he launched against Iran, 100,000 seems like a bargain in comparison. Maybe he would have gone another killing spree and this 100,000 is insurance against the later, far worse death toll. Or Maybe 100,000 lives is worth it if in the end, democracy does blossom through the Middle East. After all, we killed far more Japanese with just two atomic bombs than, according to Les, we did in a year and a half in Iraq.

If we don't count civilian casualties, we don't have to get into this kind of horrible math. And most of us don't want to. So instead, we leave it to the professionals. The military are the only ones who even try to come up with a formula, the collateral damage assessment. 30 dead civilians for 1 bad guy.

For Les, he doesn't really care who counts, just so long as someone does.

Alex Blumberg

One of the most surprising things Les discovered in Iraq is that despite what everyone says about the difficulty of counting civilian casualties during wartime, it's actually not that hard. The survey teams got participation rates that most American pollsters would kill for. Only 5 of the 988 households the team surveyed refused to answer the questions. And people were able to provide death certificates over 80% of the time. That confidence interval? Les is sure that based on the results of the first survey and with a little more money-- remember, this whole thing cost only $40,000-- he could design a follow-up survey that would narrow that interval way down.

We can count civilian casualties in wartime, we just have to want to.

Ira Glass

Alex Blumberg. Coming up, so a military that's trying as hard as it can not to kill civilians makes a mistake. What do they do then? Well, we have a recording to play for you of what they do then. In a minute, from Chicago Public Radio and Public Radio International, when our program continues.

Ira Glass

It's This American Life, I'm Ira Glass. Each week on our program, of course, we choose some theme, bring you a variety of different kinds of stories on that theme. Today's program, What's in a Number? In act one of our program we heard about the number of Iraqi deaths that occurred in the first year and a half of the war there. And how the majority of those deaths, according to this Johns Hopkins study, were caused by US led forces. Now we're going to move to act two, where we hear US forces trying to cope in the aftermath of some of those deaths. This is act two, Not just a Number.

The civilian deaths in this particular story came at a unusually bad time for captain Ryan Gist. He was the American officer in charge of the US army presence in a section of the Nineveh and Salahuddin Provinces in Iraq, a Sunni area. And this meant that he dealt with everything from seeing that the region had water and fuel, to dislodging insurgents and winning over the local population. He faced all sorts of problems.

On November 10, 2004, for example, insurgents blew up most of the police stations in his area. And the next day, Iraqi army and police simply stopped showing up for their jobs. One town called Aitha was especially troublesome. US had not made many allies there.

Ira Glass

So the guy gives him names and locations of people that he identifies as insurgents, and the army launches an operation to get those people. And this is where things went wrong. As part of this operation, US forces were supposed to drop a bomb into a nearby field just as a show of force. But instead, the bomb was dropped onto the house that the army was just about to raid, killing 12 people inside, including children. And nearly killing the US troops who were about to go in. That's according to an American photojournalist, Sheryl Mendez, who was there.

In addition to the human tragedy of these deaths, for Captain Gist this could not have been worse.

Ira Glass

The photographer Sheryl Mendez tape recorded him as he went to the sheiks who were the local leaders and to the police, as he went around trying to make things right. Here's one of those recordings. He's in a police station.

Ira Glass

This is pretty much the standard way that he'd operate in these villages. He said, sure, there were some villages and local leaders who were sympathetic with the insurgents. But in other places, they were being bossed around by them.

Ira Glass

In Aitha, he says, people waited to see who was going to win-- the US led forces or the insurgents. Until there came a turning point.

Ira Glass

Captain Ryan Gist and the 65 men that he commands returned safely to their base in Fort Lewis, Washington, at the end of last month.

Ira Glass

Act three. What do we do with these numbers anyway? So if in fact, 100,000 Iraqis died because of the war, and that number is a year old, or at least like Alex said in his report, there's a 90% probability that at the time it was over 44,000 dead, what do we do with that number? You know, the problem at the heart of the whole thing is that it instantly brings you to all of these imponderable questions about what is worth it. What is worth so many people dead?

And in a way, once I understood this number and believed this number, I wasn't even sure if it's a helpful thing to think about that. What do you do with that? So I called a woman named Nancy Sherman, who has the unusual set of jobs of being a philosophy professor at Georgetown University. And also, she's taught ethics at the US Naval Academy. She interviews military people, she writes about how they think in books like Stoic Warriors, and she thinks you do want to know how many civilians die in a war, so you can understand the trade-offs.

Ira Glass

Nancy Sherman from Georgetown University.

Ira Glass

Well, our program was produced today by Lisa Pollak and myself with Alex Blumberg, Diane Cook, Jane Feltes, Sarah Koenig and Amy O'Leary. Our senior producer is Julie Snyder. Elizabeth Meister runs our website. Production help from Sam Hallgren and Chris Ladd.

[ACKNOWLEDGEMENTS]

You can download today's program at our archives at audible.com/thisamericanlife.

This American Life is distributed by Public Radio International.

[FUNDING CREDITS]

WBEZ management oversight for our program by Mr. Torey Malatia. You know, whenever he finishes a budget or correctly orders lunch, he calls me into his office, leans back in his chair--

Ira Glass

I'm Ira Glass, back next week with more stories of This American Life.

Announcer

PRI, Public Radio International.