Trust none of what you hear, some of what you read, half of what you see
By Nassim Ncholas Taleb and cross-posted from Medium.com
When Pasquale Cirillo and I examined the historical accounts of wars for our statistical analysis of violence, we discovered huge holes –people take numbers for gospel, yet many accounts were fabrications. Many historians, political “scientists”, and others fall for them, then get to write books. For instance we saw that the scientific entertainer Steven Pinker based his analysis of the severity of the An Lushan rebellion on a shoddy overestimation – the real numbers of casualties could be lower by an order of magnitude. Much of Pinker’s thesis of drop in violence depends on the past being more violent; it thus gets further discredited (the thesis is shaky anyway as Pinker’s general assertions conflict with the statistical data he provides). Peter Frankopan, in his magesterial The Silk Roads, seem to get the point: estimations of casualties from the Mongol invasions were inflated as their accounts exaggerated the devastation they caused in order to intimidate opponents (war is not so much about killing as it is about bringing submission). Our main (technical) paper is here.
But it is not just the bullshitting of Steven Pinker: numbers for many wars seem to have been pulled out of a hat. Some journalist cites some person at a conference; it finds it way to Le Monde or the New York Times, and that number becomes fixed for future generations. For our attempt to build a rigorous method of quantitative historiography, we devised statistical robustness techniques: they consist in bootstraping “histories” from the past, considering the past a realization between the lowest and the highest estimate available, producing tens of thousands of such “historical paths” and evaluate how “robust” an estimator to changes in the aggregate. More depressingly, we found that no historian had bothered to do similar cleaning up work or robustness check – yet the statistical apparatus is there to help.
It hit me that I needed to look into the estimates of Syrian refugees in Lebanon – here again numbers are flying without much rigor, swelling upwards from report to report. But we can assess the bias: they are potentially overstimated. At a certain municipality in Lebanon, I was told that the number of refugees in the town, while large, was considerably lower than what was used by the bureaucrats of the U.N. The real number is about a third of what is published. While this is very optimistic for Lebanon (there should be fewer refugees than claimed, so let us worry less about the stability of the place), it is not good for the economics and funding of U.N. agencies and the lifestyle of their bureaucrats…