Unit 2 The Forecast is Always wrong, Video 3 Porcupine Graphs

[SQUEAKING][RUSTLING][CLICKING] RICHARD DE NEUFVILLE: Right. Richard de Neufville here again. Pleasure to talk with you, and for this session, I'm going to talk about porcupine graphs. It's all part of this topic of the forecast being always wrong, not every single time, but in general, we can't count on it being right. And we just expect it, the reality that we face during the-- our project lifetime is not what we anticipated. Now, what's a porcupine graph? Well, let me first describe what a porcupine animal is in case you don't know it or haven't seen it. The porcupine is a North American animal. It maybe exists elsewhere also. I think it does, but I'm not an expert in the area. But the important thing about a porcupine is that it is an animal, four-footed, small, about the size of a small dog, and it has a lot of spikes on it that go up from its back. And as you can see in this example here, they go all back in a variety of ways in many different directions. And what I'm going to show you is some forecasts which contrast with the-- as with a porcupine, that contrast with the actual back of the porcupine which in this case goes down from its head down towards its tail, but the forecasts go up like this, like my fingers on my hand, but are like the spines on a porcupine. Let's take a look at what those might be. Here's the first one. This is an old one. It's has a lot of history around MIT. It's called the NERF fan, the New England Regional Forecast for the use of electricity over a long period from about 1960 to about 1990. And what you see is that the actual growth was exponential at a reasonable rate. And then later on, it fell off and leveled off. Meanwhile, during all this period, the forecast kept going up, as you can see. This is the porcupine, the spines going up, but the actual use was down. Now, the first one to go up is an extension of the original use. So you can just imagine the exponential going on. But what's interesting is that while over the 10 years or so while the graph flattened out, as you can see, the big dark line flattened out, people kept projecting as if it went up again. That is, they knew somehow that it was going to go to where it had been even though the world has changed. This is remarkable, but it is not the unique case. This is an old one, but here is another one, a very recent one for the last few years of the teens of the 2010 to about 2017, The US consumption of AV fuel, aviation fuel. So the actual consumption was this black line which goes-- bounces up and down at the bottom, and all those things going up, they are the forecast. Now, you'd say, well, my goodness. How could this be? We know it's possible to know that the number of people flying in the United States has in fact increased remarkably over this period. How is it possible that consumption of AV fuel didn't grow along with it? Well, the answer is that there have been developments of newer aircraft that are more fuel efficient. People go in larger planes rather than smaller planes, which are less efficient in terms of consumption of fuel per seat mile, and so that there's been a technological change. So you can explain that in terms of, well, yeah, now we can see that would happen this way. But what's remarkable is that the experts in the area making these forecasts got it wrong. They got it wrong the first time, OK, but why did they not correct it later on? It went down, but it will go up again. We'll go up again, but it doesn't happen that way, in fact. And it wasn't as if that the switch from less efficient aircraft to more efficient aircraft wasn't obvious because it takes a long time to manufacture aircraft, and people who are experts in the area could see the aircraft coming down the production line knowing down to the fleets. So they know what will be happening. This does not come as a surprise-- oops, United Airlines has gotten a lot of new aircraft, for example. This is well known in advance. But nonetheless, the experts create what amounts to a porcupine graph. I say they create it. They don't do it on purpose. It's just that when we look backwards at it and say what was the forecast and what actually happened that we observe here, as in so many cases, a porcupine graph. Here is another one. This is also coming back to my experience at airports. This is now a notorious one in the field. This is the forecast for terminal areas. That means this is the number of operations, operations being flights, being landings and takeoff, that's an operation, the number of operations within metropolitan areas-- New York, Washington, San Francisco, Chicago, Texas, Houston, and so forth. So these are the forecasts. So they-- the line consistently going down here like this is what actually happened, and all during this period, the agency responsible for these forecasts, which I will spare them the notice of which they were, but the agency, while this was happening, they projected this. This is now a classic version of the porcupine graph that happens. Now, how is this possible? I mean, if you're observing the drop of what's happening and you know why it's happening because you know how the airlines are scheduling their fleets and everything, why is it that the forecast are, whoops, in the wrong direction? What is it that's happening? Because this is not just this case. This is a case that happens again and again. Why is this? Well, the fact is that very often experts don't learn. They know better or they think they do. Whatever it is that somehow it happens again and again as I've shown through three examples, but there are plenty more, and people send me more examples all the time because it's such a strange phenomenon. But the experts in the field don't acknowledge, don't digest, don't take on board the fact that the world has changed and they don't predict accurately. There's mental biases going on in the profession which lead to this thing called porcupine graphs which is another example of why the forecasts are always wrong.