The Science of Prediction: Losing the Super Bowl

I am a little late to the party, but I just finished reading Nate Silver’s The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (Penguin: 2012). On the whole, it is an excellent book that makes the fields of statistics and probability accessible to lay readers. Silver builds on his own work in baseball and in predicting election results to explore why it is so challenging to make predictions about the future across many fields, from meteorology to earthquakes to national security.[1] One of the chapters I found most interesting concerned the behavior of “pundits,” or the people you might see on television on Sunday morning making predictions about political outcomes. Silver analyzed a particular group of pundits and found that their predictions were no more accurate than a coin toss—and yet they are still asked to return to the show every week.

Silver draws on fascinating work by Phil Tetlock to explore a number of reasons why political outcomes are difficult to predict and why television pundits in particular are so bad at making predictions. Silver highlights some major pitfalls that pundits (and we) fall into when making predictions about the future: failing to think probabilistically and failing to update predictions when we receive new information about the world. No one can be 100% certain about what the future holds. Making good and useful predictions requires us to be explicit about the amount of uncertainty attached to a particular prediction. For example, as Silver points out in a later chapter, we are much more accustomed to seeing uncertainty expressed in weather forecasts (40% chance of rain) than in forecasts of political phenomena (20% chance this intervention will succeed). Making good predictions also requires that we be willing to change our predictions in the face of new information. Political candidates, for example, are often criticized for changing their policy positions (“flip flopping”), but an unwillingness to update our assessments and predictions in the face of new information inhibits the development of sound predictions and sound policy. So, for example, if we predict that a certain football team is likely to win the Super Bowl (60% chance) and then the team’s star quarterback sustains a career-ending injury, we would be foolish not to update our assessment of the likelihood that the team will win. We should be applying the same logic to our public policy making, particularly in the realm of national security.

The authors of an open letter to President Obama published in the National Interest on June 3 commit both of these cardinal sins of prediction. They urge the president to freeze American troop levels in Afghanistan at 10,000, barring “emergency conditions” that might favor a modest increase. The authors identify themselves as the Ambassadors to Afghanistan and invoke the ghosts of 9/11 and the threat of IS to justify their prediction that keeping troop levels this high will make the United States safer. They do not include any estimate of how likely their recommendation is to achieve the desired outcome (which they do not specify with any clarity). Nor do they seem to have updated their prescriptions after fifteen years of failure for our current policies. Daniel Davis published a great rebuttal two weeks later. As he argues, “This open letter to the president on Afghanistan is like a group of NFL coaches who have led teams to last place finishes ten straight years while trying to convince an owner to take their guidance on how to run his team. Their advice would be immediately rejected.”

In my last post on accountability, I expressed anger about the fact that a generation of pundits and politicians and advisors who have been wrong repeatedly about the wars in Iraq and Afghanistan are still invited to give advice about the future direction of American policy. There is one very good reason why these individuals are allowed to keep making public (failed) predictions: certainty sells. That is, it is much easier to sell books and make appearances on Meet the Press by overstating the confidence of one’s predictions than it is to be explicitly honest about the limits of one’s knowledge of a given issue and about the uncertainty attached to a policy prescription. I am tired of listening to people who brought us the failed wars in Iraq and Afghanistan tell us that we need to keep troops there, that we need to send just a few more Special Operations Forces, and that we need to stay just a few more years. The Super Bowls of Afghanistan and Iraq are over, and we lost. It’s not clear that anyone else won, but we definitely lost. It’s time to fire the coaches.

[1] The last chapter, in which Silver attempts to model the frequency of terrorist attacks like earthquakes, suffers from being totally atheoretical. I don’t think that this approach to predicting the distribution of a human behavior like terrorist attacks works as well as it does for understanding the distribution of natural phenomena like earthquakes.

The Decline of Violence in Human Society

A recent and provocative opinion piece in the Washington Post by Ali Wyne is titled, “The World is Getting Better. Why Don’t We Believe It?” Indeed, it can be difficult when the evening news seems to be dominated by IS beheading videos and warnings about impending epidemics to believe that, on average, things are actually improving for most people around the world. Wyne cites a number of development indicators that suggest things are looking up for human civilization: the rate of extreme poverty fell from 37 % to 10% from 1990-2015; life expectancy increased by 5.8 years for men and 6.6 years for women between 1990 and 2013. Yes, there are several armed conflicts raging around the world and we may be in the midst of a long period of transition among the world’s most powerful states, but for most people in most places, life has actually been getting better since the end of the Cold War.

Wyne does not discuss a related argument (and one that is more relevant to the focus of this blog): that the world has become a much, much less violent place for the vast majority of humanity over the last thousand years (or more). Steven Pinker makes this argument in his tome, The Better Angels of Our Nature: Why Violence Has Declined (New York: Penguin, 2011). This book has been out for a few years, but I finally got around to reading it last summer and it has stuck with me since then.

In Better Angels, Pinker marshals an impressive body of evidence to document the decline of dozens of types of violence in human society—from homicide, to deaths from war and privation, to slavery, to domestic violence and infanticide. Many of the descriptions in the book are stomach-churning (medieval torture, sadism), but I think this is what makes the book effective: yes, we have isolated acts of violence in contemporary society and there are some places around the world where shocking violence still occurs, but the “average” human of today faces a much lower risk of violence from both what we might call “spectacular” causes (interstate war) and the more “everyday” (child abuse).

I don’t think Pinker succeeds in his attempt to develop a general theory to explain this decline in human violence, but he does present a mountain of empirical evidence to document these trends, most of it very convincing. I think he could have done more to explore what we might call structural or institutionalized violence, e.g., the mass incarceration of African Americans in the United States, nor do I think he gives adequate treatment to structural violence directed against women in contemporary society.

I do think it is important, however, to document and acknowledge long-term trends like these, especially when we (quite naturally) focus on rare, shocking, discrete episodes of violence we see on the evening news. This does not mean that there is not still work to be done or that these trends will never be reversed, but Pinker is correct to highlight one of the most important trends in the decline of human violence in recent history: the absence of war between the most powerful states in the system. There are a few competing theories in international relations about why this is the case (why the United States, China and Russia are not constantly at war one another today, as major powers often were in prehistory or the middle ages in Europe), but I won’t get into them here. It is important to keep in mind that, as frightening as terrorist attacks may be, the human devastation they cause is a mere fraction of the violence that would have been unleashed in a nuclear confrontation between the United States and Soviet Union during the Cold War, or even in a conventional war between China and Russia today. In this election season, we can and should demand that both our current and prospective decision makers keep things in perspective rather than opt for fear as a motivator.

Review: Restraint by Barry Posen (or, the On Security gift guide)

Still looking for a last-minute gift for your Dad, your poli sci roommate, or your aunt who listens to NPR? Or perhaps for something to read on a long flight or train trip home for the holidays? Might I suggest Restraint: A New Foundation for U.S. Grand Strategy, by Barry Posen,[1] the Ford International Professor of Political Science and director of the Security Studies Program at MIT. In this book, Posen explains how the grand strategy of “Liberal Hegemony” has come to dominate the American security establishment since the end of the Cold War, explains why this grand strategy is “more active, and more militarized than seems necessary” (23) given the high level of security that the United States enjoys, and outlines his argument that the United States should adopt a grand strategy of “Restraint.”

Posen defines grand strategy as, “a nation-state’s theory about how to produce security for itself” (1). I think we can be a little more specific. A state’s grand strategy includes both that state’s understanding of its goals and the principal threats it faces, and the means—both military and non-military—by which the state plans to achieve these goals and protect itself from the identified threats. During the Cold War, the United States pursued a broad grand strategy called “Containment,” aimed at forestalling Soviet advances around the world (and in some formulations, rolling them back). When the Soviet Union dissolved at the end of the 1980s, the United States suddenly found itself searching for a new grand strategy to guide its foreign policy, and Liberal Hegemony eventually bubbled up to fill this vacuum.

Restraint is well written and very accessible for those who do not happen to be experts in political science or security studies. Posen tells a convincing and compelling story about how Liberal Hegemony has become the consensus on American grand strategy in the post-Cold War world. Both of the major political parties now agree on the vast majority of the principles embodied in Liberal Hegemony: the belief that the United States should maintain its position atop the global military hierarchy, that it should use its military might to promote liberal values like democracy and free markets, and that the principle threats in the world arise from rogue, failed, and/or illiberal states. Yes, the political parties do disagree on some specifics—notably the role of international institutions—and not everyone thinks we should be keeping all Muslims out of the country as Donald Trump has recently suggested, but with the possible exception of Bernie Sanders, there are no serious candidates currently running for President from either party who are questioning any of these core principles. Democratic frontrunner Hillary Clinton proved herself to be a key champion of Liberal Hegemony during her tenure as Secretary of State.

Posen argues that instead of Liberal Hegemony, we should pursue a much more restrained grand strategy that acknowledges that the United States is actually very safe. The most important threats to the United States will emanate from strong and powerful nation-states or peer competitors, not from floundering failed states. The United States should pull back from the activist foreign policy that has characterized the last twenty years and has made the United States a more attractive target for terrorism. We should scale back our military commitments overseas, stop stationing so many troops abroad, and instead rely on a maritime, i.e., naval presence, around the world to provide security for global commerce and to facilitate counter-terror and counter-proliferation efforts. In sum, we can continue to enjoy a high level of security at much less cost and without angering so many people around the world. Posen also includes a detailed analysis of the implications of Restraint for key regions around the world.

I enthusiastically recommend this book. I suspect, however regretfully, that Posen’s ideas will have a hard time making headway in Washington. There are a couple of reasons for this: 1. The political consensus on Liberal Hegemony is firmly entrenched within both parties, within the national defense industry, and within much of the military leadership. Liberal Hegemony also permeates the so-called mainstream media to an extent that events and arguments that challenge its bedrock assumptions go unreported or simply do not make it into the discussion. 2. Washington is not staffed by people who want the United States to “do nothing” in international affairs. That is, one does not rise through the ranks at the State Department, the Department of Defense, or any of the other braches of the foreign policy bureaucracy (or even the dominant think tanks) by arguing that the United States should refrain from using its power. That is simply not the message people want to hear, and more importantly, I don’t think it’s the message believed by the kind of person who works eighty hours a week for several years (or decades) for a chance at a political appointment. Arguments for restraint are more often believed by the kind of person who finds herself in academia, writing books and blog posts.

[1] (Ithaca: Cornell University Press, 2014).