An oft-repeated argument against the notion that speculation drove the recent price-spike in oil is that we never saw an increase in oil stockpiles.

The logic goes like this: If prices are increasing, then buyers of oil would cut back while producers would pump more to sell at the higher price. Who buys the extra supply? Speculators who hope that prices will go even higher. This process would involve the stockpiling of oil by speculators/manipulators. But according to available data, outstanding oil inventories didn't increase during oil price boom.

But this assertion ignores important changes in how oil markets work since crude became an investment vehicle, says MIT's John Parsons. His key claim is that the financial-ization of oil brought about a change in the term structure of oil futures. A term structure is the relationship between future prices at different dates.

Buzzwords like contango and backwardation describe two different shapes that this price curve can take. A contango happens when future prices are less than spot (i.e., current market) prices, while backwardation is the state when prices increase as you go further out in time. Since the cost of storing oil above ground is very high, says Parsons, historically the term structure of oil prices has been in a state of backwardation (which is unusual for a commodity).

In order to investigate the drivers of short-term and long-term price expectations, Parsons breaks down the movements of oil prices into two parts: short-term gyrations and long-term trends. The following chart shows that even during a period when oil prices declined, investors could earn a profit by betting on short-term futures because of higher volatility in those prices:

But as oil prices started surging in 2003, the term structure flipped into contango (i.e., current prices higher than futures). At the same time, the volatility of prices at the long-end of the oil futures curve increased while staying flat on the short-end, meaning there were more profit opportunities on the long-end than the short-end.

And this gets to the core of Parsons' argument: Since the decision to store above-ground stockpiles is mainly driven by short-term volatility, it's not surprising that we didn't see a storage surge even as prices soared. This also means that it's possible to have a situation where short-term prices are rising generally but that the volatility in these prices doesn't increase, which is what happened after 2003.

So, what drove prices higher? Unlike the Fed paper I wrote about in July, which chalked up the general rise in commodity prices to increasing global demand, Parsons believes oil prices were largely a function of financial innovations that allowed speculators to bet on the path of prices further out in the future. Most likely, the positive fundamentals in crude prices helped spur a bubble.

Parsons also makes the important point that even though prices were higher than they should have been, that doesn't mean that they were manipulated:

The idea of an asset bubble is sometimes confused with the notion that financial investors are ‘manipulating’ the price. The two need not coincide. The beliefs driving a bubble can gain traction without there being any identifiable individuals behind it. There is no evidence of manipulation on any scale corresponding to the size of the oil price spike.

The CFTC is currently considering position limits on speculators' futures positions, but Parsons says this is unlikely to prevent commodity bubbles from forming:

The general purpose of speculative limits is to constrain manipulation as well as to limit the sudden rise of order flow that would disrupt an orderly market. Speculative limits will not be very effective against an asset bubble driven by the type of widespread and gradually evolving beliefs that may have been at work in the 2003-2008 oil price. The type of asset bubble that I am speaking about here is really more of a macroeconomic problem, and not readily managed with micro-levers at the individual exchange level.