Cross-checking ShadowStats

Last week I wrote about Balaji Srinivasan's idea of creating a decentralized version of the Billion Prices Project. The post got me thinking again about the topic of alternative inflation indexes.

One of the most well-known of the alt-inflation indexes is John Williams' ShadowStats, often cited by gold bugs and bitcoin maximalists. As of August 2011, ShadowStats puts U.S. inflation at 13% versus official inflation of 5%, as illustrated in the chart below.

Source: ShadowStats

That's a huge gap. One of the two data series has to be wrong.

I've always dreamt about writing a blog post on ShadowStats, but never had the gumption or statistical chops for it. So I was happy to see that economist Ed Dolan announced on Twitter that he was  republishing a 2015 blog post in which he carefully critiqued ShadowStats. It's such a good article that I'm not going to bother writing my own ShadowStats post anymore.

ShadowStats attracts a lot of sneers from the econ commentariat. What makes Dolan's post so effective is that he gracefully takes Williams' arguments on their merits and then proceeds to analyze them. Put differently, he doesn't try to damn ShadowStats with straw man arguments. He steel-mans it (or steelwomens it).   

Anyways, do read the post. 

Dolan saves his best criticism for the end. When Dolan was writing his post in 2015, the gap between official inflation and ShadowStats inflation was a whopping 7% (see chart above). What Dolan finds is that the majority of this 7% gap can be attributed to a simple double-counting error committed by Williams. By correcting this double-counting error, the ShadowStats inflation number shrinks. And so the gap between it and the official CPI is actually far less menacing than Williams' anti-government fans like to make out.

Dolan challenges Williams to correct his double-counting mistake. But you can see why Williams might find this difficult to do. He has been selling his data for many years on a subscription basis. Admitting that his product contains errors could anger his customer base.

The other part of Dolan's blog post that I want to draw attention to is a set of simple cross-checks he performs to see whether official inflation or ShadowStats is more accurate. For instance, taking grocery prices from a 1982 advertisement and projecting them forward with both inflation indexes, Dolan finds that the official CPI does a better job of predicting where modern grocery prices actually ended up.

It would be unfair to do just one set of crosschecks. Which is why Dolan does a bunch of them. It's worth reading through each one. ShadowStats does not make out well. (For instance, in order for ShadowStats to be right, you've got to believe that the U.S. economy has been in a recession for the last two decades.)

To finish my blog post off, I'm going to add to Dolan's list of cross-checks by adding one of my own. This cross-check is meant specifically for one of the main consumers of ShadowStats data: gold bugs.

If gold investors think ShadowStats data is right, and many of them do, then they also have to accept that gold has lost 91% of its value since January 1980 (see chart below of the gold price adjusted for ShadowStats inflation). Which means that the yellow metal is an awful hedge against inflation, and anyone who buys it for that reason is making a big mistake.

Source: Bullionstar

The far more reasonable position to take is that the ShadowStats data is wrong, and that gold has actually been a decent hedge against inflation since 1980. Using official inflation numbers rather than ShadowStats, the price of gold today is almost even with its 1980 level.

So gold bugs, you can relax. You haven't lost your sanity -- gold is not an awful inflation hedge. Rather, ShadowStats is an awful measure of inflation.