Stephen Pinker and Nassim Nicholas Taleb had a brouhaha a while back about the statistical validity of Pinker’s claims in The Better Angels of Our Nature.
Looks like Taleb really likes having the last word.
And it does seem to be the last and final word.
That has now changed. Just today, Taleb, writing with another mathematician, Pasquale Cirillo, has released a detailed analysis of the statistics of violent warfare going back some 2000 years, with an emphasis on the properties of the tails of the distribution — the likelihood of the most extreme events. I’ve written a short Bloomberg piece on the new paper, and wanted to offer a few more technical details here. The analysis, I think, goes a long way to making clear why we are so ill-prepared to think clearly about processes governed by fat tails, and so prone to falling into interpretive error. Moreover, it strongly suggests that hopes of a future with significantly less war are NOT supported by anything in the recent trend of conflict infrequency. The optimists are fooling themselves.
Violent warfare is on the wane, right? by Mark Buchanan (1395 words).
From the Bloomberg piece:
Unfortunately, it may also be wishful thinking. Getting statistics right is hard, and it’s especially hard when dealing with wars, which – like earthquakes and financial markets – follow a highly erratic pattern. Long periods of relative stability are punctuated by sudden, catastrophic events, such as World War II, which took the lives of millions. Statistically speaking, the data have “fat tails,” meaning that a few observations account for most of the phenomenon being studied. As a result, indicators such as the average number of deaths per year can be misleading – after any big event, things will always look like they’re getting better.
Is the World Getting Safer? Maybe Not by Mark Buchanan (591 words).
The paper itself is a rather involved read given that it doesn’t hold back on the mathematics but it’s well worth ploughing through. An important part of the analysis is a set of statistical simulations to test the effect of the dataset’s unreliability on the overall model.
It’s a clever way of demonstrating that even if, as is almost certain, the data is incomplete and unreliable, the dynamics remain the same.
Contrary to current discussions, all statistical pictures thus obtained show that 1) the risk of violent conflict has not been decreasing, but is rather underestimated by techniques relying on naive year-on-year changes in the mean, or using sample mean as an estimator of the true mean of an extremely fat-tailed phenomenon; 2) armed conflicts have memoryless inter-arrival times, thus incompatible with the idea of a time trend.
On the tail risk of violent conflict and its underestimation by Pasquale Cirillo and Nassim Nicholas Taleb (6679 words).
Pinker’s hypothesis was always a dubious one, especially considering the unreliability, age, and nature of the data.
We can now consider the hypothesis downgraded from ‘dubious’ to ‘bloody unlikely’.