Finding the Future in the Past

I do not mean to disparage my graduate student teaching fellow, who has done a fine job developing a curriculum for my government course on technology in international relations. The problem in the development of his curriculum does not come from his beliefs, per se, but rather, from the core assumptions of the academic milieu of which he is a product. 

The problem with my teaching fellow’s curriculum is this: He takes pride in showing us the latest papers in political science that he is sure will be the most relevant for years to come, but papers in political science do not age like living things. They age in reverse, so that the newest papers will be the first to die and the oldest will be the longest yet to live. This principle derives from the Lindy Effect, which shows students that our time in academia is best spent learning not what is new, but what has been around for a long time.

History of the Lindy Effect

The Lindy Effect is an idea that is frequently attributed to Albert Goldman, though further developed by Benoit Mandelbrot and Nassim Taleb. The name comes from Goldman’s experiences at Lindy’s, a New York deli where comedians would gather after shows. The comedians’ observation was that stars would keep coming to Lindy’s — a marker of their success — for a time proportional to how long they had already spent giving shows. The theory was that comedians had a fixed, similar amount of material, so those who had long careers used up their material slowly, while those with short careers used it all at once. Therefore, if a comedian had been on the circuit for a long time, it was evidence that he was using his material at a slower rate and would perform more before exhausting it. Comedians who were new to shows and going to Lindy’s often were more likely to be using up their material at a faster rate.

The more refined understanding from Taleb originates from a statistical property: given that a variable drawn from a Pareto distribution (with certain parameters) has reached a certain number, the expectation of the variable is more than twice that number (meaning it has even more room to grow). Taleb contends that variables in the real world follow similar distributions, such as income inequality. As a complement to his theory of fragility, things that have been around for a long time have shown that they are more robust to random shocks and are thus more likely to survive into the future; ideas that have only been around for a short time have not proven so robust. The longer a theory has not been falsified (or has proved useful), the harder it probably is to falsify it (or lose utility). Taleb believes that the Lindy effect applies to almost everything, including science.

A Visual Understanding

While I think that Taleb is right in general with the Lindy effect, I would like to qualify his heuristic a bit to better understand when it is most applicable. I would like to develop a spatial interpretation that helps visualize potential constraints and builds a more comprehensive view. Consider an “ideas” space where a point would represent a society’s cultural milieu. One axis may be the society’s beliefs on war, where the left would represent more warlike ideas and the right more peaceful theories. There can be as many axes as you’d like: one for insistence on equality of opportunity vs. outcomes, for example. Technological development can be included as well. An idea would then be some subset of this multidimensional space where the points inside the space are where the idea is relevant (or where the technology is useful).

Consider a version where there are only two dimensions. Ideas would be relevant within a circle. If our cultural point represents a random walk around this 2D plane, then the longer we occupy a certain circle, the larger the circle is likely to be. In other words, given a circle of unknown radius around our randomly moving point, the expectation of the radius’s size grows with the amount of time spent within the circle.

The more we know about the circle’s bounds and the more we know about where our cultural point is moving, the less we have to rely on the Lindy Effect. So axes along which we can ignore the Lindy Effect are those where we are most certain about the movement of our cultural point and the relevance of the idea. The less certain we are about our world, the more we should rely on the Lindy Effect. The more you believe that our world remains fundamentally difficult to understand, the farther back you should look to predict the future.

This interpretation allows us to account for what might initially seem like clear counterexamples to the theory. When I talk to students on Harvard’s campus about this idea, they often mention technological advancements where the utility is obvious, like the invention of smokeless gunpowder (as opposed to gun powder which produced more smoke). In their minds, it would seem useless to use the Lindy Effect, which would predict that old types of gunpowder will not be replaced by the new technology. Rather than present counterexamples to the Lindy Effect, however, we can integrate such ideas and inventions into the theory to represent areas of thought where we are more certain about the movement of our cultural point and the relevancy of the idea. We are quite certain that smokeless gunpowder will replace other kinds of gunpowder. Instead of throwing out the Lindy Effect, we must understand that it is most useful for disciplines where we must exhibit the most humility.

Applications to Harvard: Be Humble

In one of Greg Mankiw’s final lectures as course head of Harvard’s introductory economics class, Mankiw emphasized the need for humility. Earlier, students were polled about their beliefs on economic theories along with how certain they were in those beliefs. Mankiw’s punchline was that students exhibited far more certainty than professional economists. 

Economists have good reason to be humble. Former Federal Reserve Chairman Ben Bernanke declared that the world was in the middle of “The Great Moderation” — an era of economic serenity — just four years before the 2008 financial crisis. Had you asked economists what the economic outlook was for 2020, they might have told you stories about rising interest rates or risks to global trade. None would have listed a global pandemic as the cause for a drastic market decline.

Trying to predict the future by looking at the newest theories exhibits undue confidence in our knowledge about the world and our aptitude for prediction. As I have shown, this uncertainty creates the need to study history and use the principles of the Lindy Effect. Do not believe that staying up-to-date on the newest trends in academia will better prepare you for the future than reading old ideas. In economics, you are better off reading Adam Smith instead of Thomas Piketty if you want to understand capital in the 21st century. My graduate student teaching fellow told us proudly that had he done the course just five years earlier, “the curriculum would have looked very different.” If that really is the case, then I would expect his curriculum to be obsolete in about five years’ time as well.

Image Credit: Wikimedia Commons/Bertil Carson

Leave a Comment

Solve : *
19 + 21 =