Who’s next?
Upon learning that Pope Benedict XVI will resign the papacy later this month I took the only logical step: I visited the 21st century Delphi (ahem, the twitterverse) to see what the Oracle (Nate Silver) had to say about the upcoming conclave. Having won his fame successfully predicting baseball and national elections in the U.S., I was sure that the soothsayer would have wisdom and insight to bestow. Instead, I got this:
No pope has resigned since 1415, which is also the last time the Pittsburgh Pirates had a winning record.
— Nate Silver (@fivethirtyeight) February 11, 2013
Thanks for nothing, Oracle. With a few more predictions like that, we would never have had to endure the unending slow-mo scenes of 300.1
Nate may come out with predictions on the papal conclave, but I doubt it. You might not want to either (but of course that probably won’t stop us here at TJP). Silver’s most recent book, The Signal and the Noise: Why So Many Predictions Fail – but Some Don’t, is (you may have guessed) about prediction-making both good and bad. Having combed my way through it these last weeks I predict this: the conclave is a case-study for a poorly-predictable situation. Wait, that’s not a prediction? Whatever. Prediction or not, there are a huge variety of reasons why the conclave will be nigh-on impossible to predict. Here are three of my favorites.
1. Too Much “Noise”; Not Nearly Enough “Signal”
The fundamental problem with predicting the conclave is that in the weeks to come, those following the dynamics of the papal election will be subjected to an information barrage, most of which has the predictive validity of World Cup 2010’s Psychic Octopus. If past patterns are any indicator, the coming weeks will be filled with speculations on what makes someone “papabile” ranging from to global ecclesio-politics (North America/Europe versus the world!), hot-button issues (condoms, and married priests, and women’s ordination, oh my!), and even neat little insider stories about which cardinal is allergic to whose cat and who spilled whose cappuccino at the last consistory coffee klatch.
The Oracle Declaiming
Silver explains the problem this way: “In statistics, the name given to the act of mistaking noise for signal is overfitting.”2 When our understanding of the fundamental relationship of the data to the thing we’re trying to predict is poor, what we do is select a pattern that seems to work, but really has little predictive power.3 In other words, overfitting “makes our model look better on paper, but perform worse in the real world.”4
My intuition (and it’s only that) is that pundits and bloggers will have lots of different models, each of which will be capable of very confidently explaining which candidates are most likely to be elected and why. But in almost every case they’ll be mistaking “noise” for “signal.” The one bit of information that could actually provide predictive insight is simple: what criterion/a will each of the 119 cardinals use to make a decision? And that’s precisely the information we don’t have.
2. We’ll Probably (Mistakenly) Read a Global Institution Through a Local Lens
Fun Game: as you read articles about the pope and conclave in your prefered media outlet, take out a highlighter (or digital equivalent) and mark every time you see the word “liberal” or “conservative.” Funner fact to go with Fun Game: not only do the labels “liberal” and “conservative” mean different things in different political contexts, but they have wildly different applications to Catholic theology. There’s a strong argument to be made that they don’t mean much of anything.
When faced with information overload, as we surely will be, we tend to cheat. The Oracle puts it this way: “The instinctual shortcut that we take when we have ‘too much information’ is to engage with it selectively, picking out the parts we like and ignoring the remainder, making allies with those who have made the same choices and enemies of the rest.”5
The distorting liberal/conservative dichotomy is just one shortcut among thousands that potential predictors could take, shortcuts that will blow past cultural, political, and theological nuances, not to mention – oh yeah! – the fruits of prayer and reflection. When we inevitably cheat, our biases will probably ensure that our cheating is at the expense of accuracy.
3. We’ll Probably Listen to the Wrong People
In a fascinating section of The Signal and the Noise, Silver cites a study of public people who made political predictions. He notes that some had done decidedly better than others. Among the losers, he notes, “were those whose predictions were cited most frequently in the media. The more interviews that an expert had done with the press… the worse his predictions tended to be.”6
The business of media, and especially the business of making prognostications in the media, makes for complicated sets of incentives. In particular, there’s a (distressingly) low incentive for caution, moderation, or even humility. The voices that get heard are, on average, decisive, confident, and loud. This, of course, makes for a very clear byline for a story and frequently can lead to a person appearing in the media repeatedly, but it doesn’t guarantee that it’s a good prediction. As Silver says, sometimes it has the opposite effect. There’s a good chance that various “experts” will make very definite claims about who will or will not be pope that just happen to disagree with one another. This should be a clue to all of us.
Final note: Although technically a new pope doesn’t need to be a cardinal to become pope, it’s been a long time since a pope hasn’t been a cardinal. This means that there are only 119 likely choices for pope, which gets whittled down quite a bit more based on ages of the cardinals. Inevitably, someone will get it right. Just remember, there’s a difference between a correct guess and a good prediction. Getting the right number on a roulette wheel is a correct guess; a solid and successful prediction means that your prediction succeeds for the right reason. Catholics will be praying for the guidance of the Holy Spirit, Whose plans tend to toward inscrutability. Chances are that other prediction styles won’t reveal much more.
— — — — —
