Commodity Models

My PhD supervisor Helyette Geman is organising a commodity conference in London on 20 January 2012.  Speakers include Alexander Eydeland, author of several commodity-related books and head of commodities at Morgan Stanley and Robert Doubble, head of Quant Analytics at BP.

To register for a place, a programme of events, location infomation etc., see www.bbk.ac.uk/cfc/

There’s a number of TV programmes related to commodities which are regularly shown on UK TV. As usual, they can be downloaded on BBC iPlayer for a few weeks after each TV broadcast (if you are in the UK).  I find they tend to rotate, so if you’ve missed an episode, it will probably be re-broadcast within 6 months.

Windfarm Wars

A series about developers wishing to build a windfarm in a scenic part of Britain, who encounter a lot of local hostility. There’s some information about the science of windfarms, but it’s mainly following the legal planning progress, details about noise data, wind speed data etc.  Interesting because it shows the huge perceived social costs of wind farms, and the long delays in implementing them.  A 4 part series, fairly slow moving.
http://www.bbc.co.uk/programmes/b00zn463#synopsis

The Secret Life of the National Grid

A 3-part series about the construction and social effects of the electricity grid and electricity in general.  Mostly I find it’s about the social revolution caused by having on-demand electricity.

http://www.bbc.co.uk/programmes/b00vkjmy

Richard Hammond’s Engineering Connections

I haven’t watched this yet, but one episode describes the technology in the ships that carry LNG.

http://www.bbc.co.uk/iplayer/episode/b0116cw2/Richard_Hammonds_Engineering_Connections_Series_3_Super_Tanker/

And another episode describes the technology behind an offshore natural gas platform.

http://www.bbc.co.uk/programmes/b00j6r4m

I’ve spent days or weeks looking for good online sources of commodity prices and related economic data.

For large volumes of data, suggest a subscription to Thompson/Datastream or a Bloomberg Terminal.  Be prepared to pay $5000 or so per year per terminal.

For Commodities Tick Data, I have used and recommend TickData.com.  You not only get the data, but also a frequency conversion tool.

This compilation is Copyright ©2010-2012 William Smith, commoditymodels.com

Many Commodities – Monthly

• Since 1980, try the IMF’s .csv file at the bottom of this page.
• Since 1960, World Bank

Agriculturals

Prices

• US Historical Land Values (1997 onwards) USDA
• US Historical Crop Prices USDA

Data

• Solar, Temperature, Rainfall etc (World) NASA Atmospheric Science Data Center (1983-present)
• Production, Consumption, Imports, Exports etc by country : USDA
• Production (no consumption data) : FAOSTAT
• Production by country within the EU : EUROSTAT
• Production by country : UN Commodity Statistics Database

Metals

Prices

• Copper Futures Prices on CME
• Gold Futures Prices on CME

Data

• Silver Supply and Demand from the Silver Institute
• Gold Supply and Demand from the World Gold Council
• Aluminium, Copper, Lead, Nickel, Tin, Zinc : supply & demand – “World Metal Statistics” (payment required)
• Most metals – production by country : UN Commodity Statistics Database (see Metal Ores section)
  

Energy

Prices

• Crude Oil – WTI Futures Prices on CME
• Crude Oil – WTI Historical Prices
• Crude Oil – WTI Historical Spot Prices (1986-present)
• Crude Oil – WTI Historical Futures Prices (1st 4 months only, 1985-present)
• Crude Oil – Brent Historical Spot Prices (1987-present)
• Gasoline – RBOB Futures Prices on CME
• Gasoline – Various Worldwide Locations, Historical Spot Prices (1986-present)
• Gasoline – NY, Historical Futures Prices (1st 4 months only, 1994-present)
• Natural Gas – Henry Hub Futures Prices on CME
• Crude Oil Distillates – Fuel Oil, Heating Oil, Diesel, Jet Fuel, Propane etc Historical Spot  Prices (1986-present)
• Crude Oil Distillates – Heating Oil, Propane Historical Futures Prices (1st 4 months only)
• Electricity – There are dozens of electricity markets, only a few give out their data
  PJM – Hourly prices, monthly files historical average LMP (locational marginal prices)

Data

• Crude Oil – EIA Inventory Report
• Crude Oil – OECD stocks (and see also here)
• Crude Oil Products – US Refinery Utilization data
• Solar – (US) National Solar Radiation Database (1960-2005)
• Solar, Temperature, Rainfall etc (World) NASA Atmospheric Science Data Center (1983-present)
• Oil and Gas – production by country : UN Commodity Statistics Database (see Ores and Minerals; Electricity; Gas section)
• Oil,Gas,Coal,Electricity – production and consumption by country : BP Statistical Review of World Energy

Exotics

• Shipping – Baltic Dry Index (graphs only)
• CO₂ emissions – European Climate Exchange (ECX) Historical Futures Data

Economics

• US Treasury Yield Curve
• US S&P500 Implied Volatility – VIX
• US GDP
• UK Inflation

If the data or price you want is not here, please don’t ask me to find it for you (unless you want to pay me for consulting), find it yourself and then post a comment and I’ll update the table.

This compilation is Copyright ©2010 William Smith, commoditymodels.com

Avid viewers and readers of Bloomberg, like myself, may have noticed a recent article “Commodities’ Biggest Drop Since Lehman Is A Bear Signal”. They use a little-quoted index of commodities, the “Journal of Commerce Industrial Price Commodity Smoothed Price Index”, claim that it “reflects clearer signs of supply on demand because half the items it tracks don’t trade on exchanges used by speculators” and cite a 57% “plunge” in May!

For me, as a commodity researcher, this seems like huge news. Bloomberg seems to be saying that real-world, unmanipulated commodity prices have halved in a month, and speculator’s presence in the bigger markets is hiding this ‘reality’ from investors. Right?

Absolutely wrong. Dangerous misreporting, Bloomberg. I now see people picking up this story around the world on their blogs and suggesting this may be a time to sell investments.

The Bloomberg TV report (I’m trying to get a screen capture, but can’t find the video archive) even compounds this error by showing this -57% figure underneath daily and monthly prices changes for other assets such as WTI crude oil.

First lesson, go to the source and understand your data. Bloomberg, quote the “Journal of Commerce Industrial Price Commodity Smoothed Price Index” as being at 60.56 at the end of April 2010, and 25.97 at the end of May 2010. That’s a drop of 57%, as they state.

So, what is this index measuring? And, how can a commodity price index be negative, as it was in July 2009? The main problem, it’s not a price index! It measures the year-on-year growth, as a percentage, of the tracked commodity prices over the previous year. Confusingly, the Bloomberg page states “the base rate of this index is 2006=100″. To me, this bit is plain wrong. A quick bit of charting shows the index varied between 0 and around 25 during 2006.

So, what the index really tells us, is that the commodities prices tracked by the index grew 60% between April 2009 and April 2010, but only 26% between May 2009 and May 2010. Another way of expressing this, far more fairly, would be:

“Commodity Price Growth reduces to 26% p.a.”

That’s the real story. There’s a huge difference between a growth-rate index and a price index. Reporting the change in a growth-rate index (itself a percentage) by dividing and quoting this as another percentage makes no sense. If inflation goes from 5% one month to 4%, is inflation -20%? No.

Rein in your sensationalist reporters, Bloomberg.

I’ve spent some of the last few months studying oil depletion. It’s less about models and more about strategy, information and international politics. The Geologists are mainly pessimistic, predicting peak oil soon. The economists are more optimistic, maybe because they think you can throw money at any economic problem (i.e. by higher prices) and get more oil. Ultimately though, once non-OPEC oil is gone (which must happen first, because OPEC hold much more reserves), OPEC hold all the cards and nobody can predict what they’ll do. Read my full PDF.

My current project needs a good visualisation of various commodity futures curves and their evolution over time. I’ve seen dozens of ‘Snapshots in time’ pictures in various sources, but I knew there was a better way. Some coding in Matlab has enabled me to observe the futures curves evolving in time, rendered as a movie.

Below are some samples, for the period 2004-2009. The lefthand figure is the futures curve, the righthand figure is the price history of the front-most contract (usually as close as we get to ‘spot’ price). Linear interpolation has been used if some months are not traded.

Please comment if you can’t view the movies (Apple people, can you see them?) or want other commodities.

NOTE : Download the movies for much better quality.

Energy Commodities

WTI Crude Oil, ‘CL’ 2004-2010 (download, 28Mb) / 1985-2005 (download, 100Mb)

Crude Oil, WTI, NYMEX 2004-2010

►

Nymex Natural Gas, ‘NG’ (download movie, 27Mb)

US Natural Gas, Henry Hub

►

Metals

Copper, High Grade, ‘HG’. (download movie, 28Mb)

Copper, High Grade

►

Agriculturals

Wheat, #2 Soft Soft Red Winter, ‘W’. (download movie, 28Mb)

#2 Soft Red Winter Wheat

►

Cocoa, LIFFE (download movie, 29Mb)

Cocoa, LIFFE

►

A paper (PDF) written back in 1991 by Jacques Gabillon is an excellent introduction to futures curves. He describes the general features of futures curves and specifically those in the oil market, such as the typical ways they change over time, the concepts of ‘backwardation’ and ‘contango’, and the term structure of volatility. He compares the futures curve with the term structure of interest rates.

He then goes on to examine simple models for oil prices using a single SDE (stochastic differential equation) to describe the short-term movements, and clearly explains the role of convenience yield and why cannot be a constant in time and across all maturities. He gradually adds necessary features to a simple oil model to captures more and more empirically observed features of the futures curve. This culminates in his proposed 2-factor model, now known as the Gabillon Model, having both the short term and (unobserved) long term price of oil represented as an (arithmetic) Brownian Motion.

As well as an excellent introduction to his own model, Gabillon’s paper (freely available here (PDF), no login required) is a great introduction to futures curves. It is as relevant today as when it was written, and the principles apply to all commodities, not just oil.

Reference:

Matlab 2010a has been recently released, and finally the student version has the Parallel Computing Toolbox available for purchase as an option. I’ve been waiting to play with this for several years.

I document in the PDF below my initial experiments with the toolbox. Specifically, I look into a simple, parallelizable task, that of interpolating commodity futures curves, when we often want to interpolate many days’ curves. Each can be processed independently of the others.

I document a basic way of parallelising this algorithm in Matlab, what kind of speedups to expect using the ‘parfor’ syntax, and the risk of significant slowdown if care is not taken.

I also manage to make the software completely backwards compatible, so it can be shared with those who don’t have the parallel computing toolbox.

Finally, I show a way of making intelligent libraries. For small sized problems, they will execute in serial, because the parallel method imposes big overheads. For larger problem sizes, they will switch to parallel mode.

The PDF includes all the nececessary Matlab software (‘codes’) for others to duplicates these results.

Any comments would be very welcome. I am going to be using the parallel features of Matlab more in coming weeks. It’s good practise, as we seem mainstream 4, 8 and soon 12 core desktops. I have also been experimenting with using CUDA on Matlab, and can recommend the ‘gp-you’ toolbox, which makes CUDA on Matlab easy to learn. It’s yet to be a full-power toolbox though, and is missing things like random number generation on the GPU, essential for monte-carlo simulations.

Download full article (PDF)

I have compiled a list of some of the most important and influential papers and models, as an introduction to commodities finance or for those involved in modelling commodities (oil, gas, electricity, metals and agricultural products). It is not exhaustive; it is intended as a guide to ‘where to get started’.

Version 1.4 (PDF)

I’ve recently been calibrating some oil and gas models, which involved parameter estimation of the well-known Ornstein Uhlenbeck mean reverting process.
I found that it’s very hard to accurately calibrate the mean-reverting-strength parameter of the model, so I looked into this in some more detail, including simulations and Matlab code to simulate and estimate the process, using both Maximum Likelihood and Least Squares estimation techniques, and I’ve documented it here (PDF).

I gave a presentation (PDF) earlier in the week to some past students of the Wilmott CQF course which I took myself back in 2008. The topic was ‘Introduction to Commodities Modelling’.

The audience members were mainly finance professionals (both in London where I gave the talk, as well as various other financial centres where people logged in to watch online). Typical CQF graduates are very good at mathematical finance with strong commercial experience of equities/interest rates/credit products, so I was “talking to the converted”.

The aim was to give a flavour of commodities as an asset class, show how each commodity is different and to cover some of the modelling tools that can be used depending on the problem, such as Geometric Brownian Motion, mean reverting processes, regime shifts, spike modelling, seasonality, the ‘convenience yield’ concept, etc.

The presentation I gave is here (PDF) (with slight modifications – I’ve improved the presentation slightly since I gave the talk).

I’ve been reading a lot of material recently on resource depletion, and specifically on the oil and gas industries.  You’ve no doubt heard some people saying “we’re fast running out of oil”, others saying “we had 30 years oil left in 1970, and we still do today” and such seemingly contradictory statements.

What’s the truth of the matter?  It’s an extremely complex area.  So a paper that gives a summary of the situation is very welcome.  The best I’ve found so far is this paper by R.W. Bentley of the Oil Depletion Analysis Centre, London, UK, entitled “Global oil & gas depletion: an overview” (2002).

Mr Bentley first covers some definitions because there’s lots of difficulties with definitions, units, categories etc in this field.  What is oil?  How do we classify heavy oils, oil shales, natural-gas-to-liquid technology and so on, what is a P50 forecast, etc…

He then covers the available data, why it’s hard to get and why there’s no single ‘definitive’ database of oil wells, discoveries etc. In particular, we know that OPEC’s data, the most important because it covers the most oil, is also probably the least trusted, because of some very suspicious changes to oil reserves in the 1980′s (for some technical and political reasons, most OPEC countries doubled their stated oil reserves ‘overnight’ during the 1980′s).  Now nobody knows what is really true.

Bentley then looks at the various alternatives to ‘conventional oil’.  He covers various  estimates of how much is available, both remaining undiscovered, in existing known reserves, and already consumed (the so called URR or ultimately recoverable resource).  He then looks at various depletion models (what happens to production over time, especially what happens as resources dwindle) and likely shapes of the production curve.  This is one of the key issues.  He covers the people making the major forecasts and their background, including a section on the legendary ‘Hubbert’ predictions.  Finally he draws his own forecast, placing ‘peak oil’ at around the current day (2010).  Bear in mind this prediction was made in 2002, but the paper is barely dated.

In summary, R.W. Bentley’s paper is a great introduction to a very complex topic.

Reference:

I’ve started a new section recommending some of the best papers I’ve read on commodities and energy.

The first entry goes to this paper by Fabian Kesicki of the Energy Institute, UCL, University of London, entitled ‘The third oil price surge – What’s different this time?‘

In a well written 11 pages Fabian covers all the basics of the modern oil market as well as the history of oil in the 1970′s, when the previous two ‘oil shocks’ happened.  He covers in detail demand factors that affect the price of oil (GDP and energy intensity, substitutability of other energy sources) as well as production issues (capacity constraints, investment, various major discoveries, reserves ‘growth’ etc).  He also looks at refining issues, discusses the different types of crude oil as well as the major uses of oil, and what this means for refineries.  He manages to cover geopolitics, looking at OPEC (as you’d expect) and the effect of OPEC’s varying pricing power over time.  He even covers the role (large or otherwise) of speculators in high oil prices, and finishes by examining the effect of the US dollar exchange rate on oil prices.

In conclusion, if you want to read one paper and learn a vast amount about the oil industry and oil prices since the 1970′s, read Fabian Kesicki’s paper.  His paper covers an amazingly wide range of issues, without dwelling excessively anywhere.  I have not read a better summary of the world oil markets.

References:

Kesicki, Fabian. 2010. The third oil price surge – What’s different this time? Energy Policy 38, no. 3 (March): 1596-1606. doi:10.1016/j.enpol.2009.11.044.

I will be presenting a talk on commodities modelling to past alumni of the Wilmott Certificate in Quantitative Finance on 9th February 2010. The content will be at a level suitable for those unfamiliar with the commodity asset class. The topics will include:

o) commodities as an asset class
o) commodity contracts
o) details of the oil, gas and electricity markets
o) spot price modelling of the above markets, including modelling jumps and the use of Markov regime switches
o) commodity correlation (inter-commodity, temporal and locational)

If you are a past CQF delegate, please come along at 7city training, London (map).

My slides for the presentation to the Masters in Financial Engineering students at Birkbeck, University of London are here.

I recently gave a presentation to the students studying ‘MSc Finance’, and ‘MSc Finance and Commodities’ at Birkbeck, University of London on the subject of ‘choosing a dissertation topic’. The slides are here.

Gas: The current amount of gas stored in UK in long, medium and short-term storage is available in a spreadsheet from National Grid (click top link). For comparison, see the 2008-2009 spreadsheet. More information is available at the ‘prevailing view’.

Electricity : monitor UK’s instantaneous electricity demand here, from National Grid

This blog on commodities, financial modelling and related topics is written by Will Smith.  I’m a PhD student at Birkbeck, University of London studying Commodities Modelling (formally, I’m studying Mathematical Finance).