The Science of Prediction: Losing the Super Bowl

I am a little late to the party, but I just finished reading Nate Silver’s The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (Penguin: 2012). On the whole, it is an excellent book that makes the fields of statistics and probability accessible to lay readers. Silver builds on his own work in baseball and in predicting election results to explore why it is so challenging to make predictions about the future across many fields, from meteorology to earthquakes to national security.[1] One of the chapters I found most interesting concerned the behavior of “pundits,” or the people you might see on television on Sunday morning making predictions about political outcomes. Silver analyzed a particular group of pundits and found that their predictions were no more accurate than a coin toss—and yet they are still asked to return to the show every week.

Silver draws on fascinating work by Phil Tetlock to explore a number of reasons why political outcomes are difficult to predict and why television pundits in particular are so bad at making predictions. Silver highlights some major pitfalls that pundits (and we) fall into when making predictions about the future: failing to think probabilistically and failing to update predictions when we receive new information about the world. No one can be 100% certain about what the future holds. Making good and useful predictions requires us to be explicit about the amount of uncertainty attached to a particular prediction. For example, as Silver points out in a later chapter, we are much more accustomed to seeing uncertainty expressed in weather forecasts (40% chance of rain) than in forecasts of political phenomena (20% chance this intervention will succeed). Making good predictions also requires that we be willing to change our predictions in the face of new information. Political candidates, for example, are often criticized for changing their policy positions (“flip flopping”), but an unwillingness to update our assessments and predictions in the face of new information inhibits the development of sound predictions and sound policy. So, for example, if we predict that a certain football team is likely to win the Super Bowl (60% chance) and then the team’s star quarterback sustains a career-ending injury, we would be foolish not to update our assessment of the likelihood that the team will win. We should be applying the same logic to our public policy making, particularly in the realm of national security.

The authors of an open letter to President Obama published in the National Interest on June 3 commit both of these cardinal sins of prediction. They urge the president to freeze American troop levels in Afghanistan at 10,000, barring “emergency conditions” that might favor a modest increase. The authors identify themselves as the Ambassadors to Afghanistan and invoke the ghosts of 9/11 and the threat of IS to justify their prediction that keeping troop levels this high will make the United States safer. They do not include any estimate of how likely their recommendation is to achieve the desired outcome (which they do not specify with any clarity). Nor do they seem to have updated their prescriptions after fifteen years of failure for our current policies. Daniel Davis published a great rebuttal two weeks later. As he argues, “This open letter to the president on Afghanistan is like a group of NFL coaches who have led teams to last place finishes ten straight years while trying to convince an owner to take their guidance on how to run his team. Their advice would be immediately rejected.”

In my last post on accountability, I expressed anger about the fact that a generation of pundits and politicians and advisors who have been wrong repeatedly about the wars in Iraq and Afghanistan are still invited to give advice about the future direction of American policy. There is one very good reason why these individuals are allowed to keep making public (failed) predictions: certainty sells. That is, it is much easier to sell books and make appearances on Meet the Press by overstating the confidence of one’s predictions than it is to be explicitly honest about the limits of one’s knowledge of a given issue and about the uncertainty attached to a policy prescription. I am tired of listening to people who brought us the failed wars in Iraq and Afghanistan tell us that we need to keep troops there, that we need to send just a few more Special Operations Forces, and that we need to stay just a few more years. The Super Bowls of Afghanistan and Iraq are over, and we lost. It’s not clear that anyone else won, but we definitely lost. It’s time to fire the coaches.

[1] The last chapter, in which Silver attempts to model the frequency of terrorist attacks like earthquakes, suffers from being totally atheoretical. I don’t think that this approach to predicting the distribution of a human behavior like terrorist attacks works as well as it does for understanding the distribution of natural phenomena like earthquakes.

Accountability in Kunduz

I’m back! I can’t promise that I will be posting regularly just yet but I am planning to ease back into the blog over the next couple of months. Now that my two major projects for 2016 (my book and my newborn daughter) have come to fruition, I’m also planning to focus on my new manuscript about British decision-making in the spring of 1940. Over the past twelve weeks I have, however, managed to sneak in some reading and some time to think about the current state of American foreign policy. One theme to which I have found myself drawn over the past couple of months is accountability. I will be exploring this idea over the next few weeks, and today’s post concerns accountability and responsibility in relation to the war in Afghanistan.

A few weeks ago, the New York Times Magazine published a great piece written by Matthieu Aikins and illustrated with haunting and sometimes gruesome photographs by Victor J. Blue about an American air strike in October 2015 that destroyed a hospital run by Doctors Without Borders (known by its French initials MSF) in Kunduz, Afghanistan. The article does an excellent job of teasing out responsibility for the strike, which killed 42: was this a simple case of negligence on the part of American forces or was it something else? The article suggests—convincingly I think, although I am simplifying a bit here—that the Afghan security forces manipulated their American allies into destroying the hospital.[1]

Why would local forces want the United States to destroy a hospital? MSF touts itself as an impartial actor in local conflicts, and consequently its mission is to provide medical care to individuals regardless of affiliation. As a result, the hospital in Kunduz and its staff treated many Taliban fighters alongside civilians and those loyal to the Afghan government. Aikins’s reporting suggests that the local population and Afghan security forces had long suspected (probably erroneously) that the hospital was serving as a headquarters of sorts for the Taliban. On October 3, 2015, the Afghan security forces seized an opportunity during an American mission to dupe the Americans into firing on the hospital, incinerating the building and those unable to escape in time.

The destruction of the hospital and the confusion surrounding American actions in October 2015 illustrate some uncomfortable truths about America’s involvement in Afghanistan: after fifteen years on the ground, we still don’t have a clear idea of what we are trying to accomplish there, we don’t understand the currents of local resentment and loyalties among the population, and we cannot be confident in the goals and capabilities of the Afghan forces. The American combat mission formally ended in 2014, and since then American forces on the ground have been restricted to two roles: training and advising the Afghan security forces, and attacking al Qaeda and IS. We saw in Kunduz in October 2015, however, that it is very difficult in practice for American forces to operate within these restrictive mandates and that a mission that is supposed to be purely defensive in nature may in fact be sliding down a slippery slope to offensive engagement with the Taliban.

Who bears responsibility for killing the Afghan civilians who died when American forces attacked that hospital in Kunduz? Is it the Afghan security forces, for tricking the Americans into firing on the wrong target? The Taliban, for making Kunduz a war zone? The United States, for sending its forces there in the first place? The Americans who made the decision to fire in the midst of shaky intelligence? Osama bin Laden, for launching 9/11 and drawing the United States into war there in the first place? I wish I had a good answer to this question. It seems that the New York Times can’t make up its mind, either. The print version of this story was titled “Reading the Wreckage: Did Afghan Forces Mislead the United States into Destroying the Doctors Without Borders Hospital in Kunduz?” As you will see if you click through, the title for the online version of the same article is, “Doctors with Enemies: Did Afghan Forces Target the M.S.F. Hospital?” In other words, the online version omits the United States entirely from the title and places all the blame on our Afghan allies.

 

[1] This does not mean that the United States bears no responsibility for the attack. The Pentagon recently disciplined sixteen soldiers for their conduct during the strike. The Afghan forces may have prodded the Americans to strike the hospital, but the accident raises important questions about our intelligence on the ground in Afghanistan and the rules for conducting such strikes. As Aikins’s article details, these “rules” are often unclear because of the nature of the American mission in Afghanistan.

Evaluating Clinton’s Experience: The 2011 Libya Intervention

The front page of Sunday’s New York Times was dominated by a lengthy article (the first of two) about Hillary Clinton’s role in the 2011 decision to intervene in Libya. “As she once again seeks the White House, campaigning in part on her experience as the nation’s chief diplomat, an examination of the intervention she championed shows her at what was arguably her moment of greatest influence as secretary of state. It is a working portrait rich with evidence of what kind of president she might be.” Indeed, there is no case more important if we want to understand how Clinton would conduct her foreign policy if elected President, aside from her decision to vote in favor of the 2003 Iraq War.

If you haven’t read the piece, I encourage you to do so. It paints a fairly accurate picture of how the United States decided to intervene in Libya: very hastily, without a clear understanding of the conflict on the ground, at the urging of European allies, and with Clinton’s support playing a pivotal role in pushing for the intervention. Robert Gates (then Secretary of Defense) claimed that Clinton’s support tipped the vote “51-49” in favor of the intervention and was a key factor in overcoming President Obama’s hesitation about the operation.

It just so happens that I wrote an entire chapter on the Libya intervention in my new book, Cheap Threats. The Times’ description of the run-up to the intervention is fairly accurate, but it makes some glaring omissions: there was no convincing evidence that the impending massacre of civilians in rebel-held Benghazi, which was used as the pretext for the UN resolution and the launch of the no-fly zone, would have occurred. Political scientist Alan Kuperman wrote an extremely important and convincing article[1] for Foreign Affairs making exactly this argument.  Leaving that issue aside, the intervention was initially sold to the US public as necessary to protect Libyan civilians. But in the aftermath of Qaddafi’s overthrow, we did absolutely nothing to stop the rebel forces from liquidating entire villages of people that had been loyal to Qaddafi’s regime, or to stop the competing factions from seizing and destroying Libyan infrastructure.[2] (It’s not surprising that this gets little attention from the Times given that it and most American media outlets devoted very little coverage to the violence that unfolded on the ground in Libya after Qaddafi’s overthrow—one had to look to European and Middle Eastern news outlets for this information after America’s attention went elsewhere. I await the second installment in the series to see whether these omissions will be rectified.) Even more ridiculous in the context of this “we must save Libyans” crusade is the fact that, around this time in early 2011, Bahrain was actually mowing down protesters in the street, not simply threatening to do so, but we never debated intervening in Bahrain because we have important military bases there. The point is not that we should have sent more force to pacify Libya (or Bahrain)—the point is that we could have and should have foreseen that our interference would destabilize both the country and the surrounding region.

It is clear from the article and from Clinton’s insistence that she wants to put boots on the ground to expand the campaign against IS in Syria, however, that the lesson she took away from the 2011 Libya intervention is not that we should have stayed out of it, but that we should have done more. In the summer of 2011, when rebel forces were gaining the upper hand against Qaddafi, she praised the intervention as a model of “smart power”[3]: “This is exactly the kind of world that I want to see where it’s not just the United States and everybody is standing on the sidelines while we bear the cost, while we bear the sacrifice.”

Except that we didn’t really sacrifice anything to make that intervention happen (other than the stability of North Africa). This is exactly the point I make in my book: the 2011 Libya intervention is a prime example of how the United States has developed a model for the use of military force that relies on precise, standoff strike technologies to minimize the human and political costs of using force—with the result that we have actually undermined our ability to wield force effectively. The Libya intervention was conducted entirely from the air and involved the extensive use of drones. Furthermore, the Obama administration asserted that it did not need Congressional approval for the operation because it was so limited—the lack of boots on the ground meant that it did not “count” as a war and thus it did not need Congressional approval. This is very troubling: once the Executive has asserted a right to a particular power in foreign policy, it is unlikely to be taken away. The next President can point to the Obama administration’s assertions to justify all manner of international action without the need for Congressional oversight.

Reading between the lines of the article, Clinton comes across as anything but “experienced” in this case: she rushed to push intervention in the face of very limited and conflicting intelligence about the situation on the ground in Libya, she allowed herself to be swayed about the coherence of Libya’s opposition by a very small number of men claiming to represent this opposition (recalling how the George W. Bush administration allowed itself to be duped by a single human source who fabricated information about the Iraqi regime in the run-up to war in 2003), and she pushed (despite objections from military officials and members of the State Department) to funnel arms to the fractured Libyan opposition—arms that are now in the hands of al Qaeda and IS and helping to sustain insurgencies across North Africa. Make no mistake: this intervention was a disaster and one whose consequences we will continue to suffer in the coming years—if not the coming decades. It was a disaster not because we didn’t send enough forces to get the job done, but because we never should have gotten involved in the first place.

Anne-Marie Slaughter, then director of policy planning at the State Department, described exactly what type of president Clinton would be: “when the choice is between action and inaction, and you’ve got risks in either direction, which you often do, she’d rather be caught trying.”  How many Iraqs and Libyas are we going to “try” before we learn that regime change is a foolhardy endeavor that inflicts massive suffering on civilian populations (regardless of how precise our weapons are) and unleashes far-reaching consequences over which we have limited control?

 

 

[1] Alan J. Kuperman, “Obama’s Libya Debacle: How a Well-Meaning Intervention Ended in Failure,” Foreign Affairs (March/April 2015).

[2] Anne-Marie Slaughter, who was a major proponent of intervention in 2011, admits that “We did not try to protect civilians on Qaddafi’s side,” but the article does not follow up on this reality.

[3] A term totally devoid of meaning, despite its buzzword status around the Beltway.

Special Operations and War on the Cheap

My book Cheap Threats:  Why the United States Struggles to Coerce Weak States is now available for pre-order from Georgetown University Press and Amazon.  In Cheap Threats, I argue that the United States struggles to use threats[1] effectively against weak states (like Iraq, Libya and Haiti) because it has adopted many strategies that render the use of military force relatively cheap.  Because force is cheap, the threat to use force does not convince weak targets of the United States’ compellent threats that the United States is highly motivated.  These targets will resist in the face of a cheap threat–even one issued by the state with the world’s most powerful military–because the cheapness of U.S. military action suggests that the United States lacks the motivation to pursue a long and costly victory against them.  In other words, by making the use of force cheap, we have made threats against weak states less effective.[2] This is a problem because the United States often chooses to launch costly military operations when the threat of force alone is insufficient to change the targeted state’s behavior.

How can I possibly argue that the use of force is cheap for the country that spends nearly as much as the rest of the world combined on defense each year? The first chapter of the book explains in detail the strategies that the United States has adopted to minimize the human, political, and financial costs of employing force—including the use of deficit spending to fund major military operations like the wars in Iraq and Afghanistan. I won’t repeat these arguments here, except to highlight one of the major cost-minimizing strategies that I discuss in the book: the transformation of the American military since the 1970s.

Before 1973, the United States relied on some type of conscription, i.e., compulsory military service, to fight its wars. After the backlash against selective service in the Vietnam War, during which privileged young men were better able to escape the necessity of military service, the United States eliminated conscription and has since relied on an all-volunteer force (AVF). This has led to an increase in education, retention, and professionalization in the service, but the U.S. military is not a representative institution: new recruits are disproportionately drawn from the lower socio-economic classes; the southeastern states tend to be over-represented; minorities are over-represented in the enlisted ranks while whites dominate the officer corps, etc. As a result, the wealthy, educated individuals who are most likely to be members of the decision-making class (members of Congress, scholars at think tanks, i.e., those responsible for making decisions about when and where we fight wars) are insulated from the burdens of military service and increasingly unlikely to have any direct connection to the military themselves. In addition to opting for a volunteer force that shields the vast majority of the American public from the privations and sacrifice of military service, the United States has also relied increasingly on private military contractors to perform functions formerly reserved for the military. The casualties suffered by these individuals are not counted in official figures, which helps to further insulate the general public from the true human toll of America’s military conflicts.

Why does this matter? It matters because strategies that minimize the impact of the use of military force on the American population make it much, much easier for policy-makers to choose to use military force. (Can we imagine that the George W. Bush administration would have been able to sell the war in Iraq if conscription had been in place in 2003?) The AVF makes it much less politically costly to use force, and hence makes it more likely that force will be used. My book demonstrates, however, that such strategies that make the use of force cheaper also undermine our ability to successfully threaten weak states with the use of force.

An op-ed by former Army Captain Matt Gallagher in last Sunday’s New York Times highlights an important and related trend: the growing national obsession with special operations forces (Army Rangers, Delta Force operators, Navy SEALs, etc). “The mythos of Special Operations has seized our nation’s popular imagination, and has proved to be the one prism through which the public will engage with America’s wars…We like our heroes sanitized, perhaps especially in murky times like these.” In the midst of an overall downsizing of the U.S. military, the number of special operations forces continues to rise. Perhaps more importantly, these types of forces conduct their missions in the shadows, with limited Congressional and public oversight of where they are sent (139 countries in 2015, many of these for training missions). Out of sight, out of mind.

As the American public becomes increasingly enchanted with the myth of the commando running secret missions in bad neighborhoods while bathed in the green glow of night vision goggles, it is pushing the United States further down this path of seemingly cheap, low-commitment, sanitized military force. As Cheap Threats argues, however, this will continue to undermine our ability to wield threats of force effectively.

 

 

[1] For example, the United States might threaten a small state with air strikes if it does not admit weapons inspectors.

[2] The theory I advance in the book suggests that threats against more powerful states will be more effective because they are more costly to issue and to execute. See the book for a more thorough explanation.