First, you need to understand that oil is not traded in a free market. Its production and sale is highly politicized since state-owned companies throughout OPEC and elsewhere control much of the world's oil reserves. Resource economist Douglas Reynolds argues (quite convincingly, I think) that state-owned oil companies have much less incentive to invest in expanding their reserves and production. They tend to be more concerned with current profits which can be used for social and military spending. The result is that they hold back on spending for exploration. This means that not as much oil is being brought to the market as a free market might dictate.
Second, since production (until very recently) has been frequently restrained by OPEC in a effort to maintain higher prices, non-OPEC producers have been pumping their oil as fast as they can. They are taking advantage of the higher prices afforded by OPEC's efforts to boost prices. But, there is also a counter-phenomenon taking place. Whenever non-OPEC producers took too much market share from OPEC, OPEC (particularly Saudi Arabia) has boosted production, driving down prices to take that market share back. We saw this in the mid-1980s when oil traded for a brief period as low as $6 a barrel and more recently in 1999 when it traded at $10 a barrel. (The Saudis had so much excess capacity that they could almost literally flick a switch and double their oil production.)
This pattern has two perverse effects. One is that the world will run out of higher-cost non-OPEC oil before it runs out of OPEC oil. It seems like this would mean that oil is going to be cheaper in the long run. But, that's probably not the case. The other effect is to give the wrong price signals to the market. An efficient free market would result in gradually rising oil prices as the cheap stuff is used up first and the more difficult-to-get and therefore more expensive oil is exploited later. Under these conditions a gradually rising price would give signals in the marketplace that would encourage the development of alternative energy sources.
But, that hasn't really happened. True, we have seen the development of wind and solar and other alternative energy sources. But, that has mostly been the result of government subsidies. (Under the circumstances, it's a good thing that the subsidies were there or we'd be much farther behind in making an energy transition.) In general, the cheap price of oil, a price kept down by the constant threat by OPEC to flood the market, has hampered the development of alternatives to oil.
But, the day is bound to come when OPEC will no longer be able to threaten to flood the market, when the so-called "OPEC overhang" will be gone. When that happens, prices long suppressed by competition between OPEC and non-OPEC producers and by the "OPEC overhang" will suddenly skyrocket as market participants come to realize that oil is actually in short supply. (Some people say we may be witnessing just this phenomenon right now.)
This sounds like a short-lived phenomenon that would pass with the discovery and production of new oil fields, fields made profitable to develop by higher prices. There is some truth to this. When prices hit such high levels, Reynolds believes, the reserves once held back by state-owned oil companies will be brought to market. While they won't halt the inevitable decline of world oil production, these reserves and other nonconventional sources such as oil sands will make that decline more gradual than it would have been had the reserves and other sources been exploited to their maximum earlier.
But, the main problem will be that there aren't that many huge fields left to discover, even with much higher prices as I outlined in a previous post on the so-called "Mayflower Effect."
If we're unprepared, the dilemma that we'll all face then will be how much of our limited energy resources to devote to making an energy transition and how much to devote to just keeping warm. There may still be time to make that transition before the crisis comes, but it will require strong leadership, vision and sacrifice, something that seems in short supply today.
(Comments are open to all. See my list of environmental blogs on my sidebar.)