How Well Calibrated is Predictit?

Published: 03/29/2021

Since June 2020 my raspberry pi has been dutifully logging the Predictit market data every five minutes. I've run up to the 1 TB limit on the first hard drive I was using and ran up to the 64,000 file per directory limit the FAT file system imposes 4 or 5 times. Simply downloading the market data file Predictit provides was perhaps a little naive.

After almost a year I figured it was time to actually try to learn something from the data so I put it in a Postgres database and started asking some basic questions. I have data on 676 markets 567 of which are resolved and 109 still open. I have nearly 50 million rows in my prices table. I'm not quite at big data but I'm starting to have some medium data.

My question for today is: how well calibrated is predictit? In other words historically with what frequency f does a contract trading at a price p resolve to yes. In a perfectly calibrated context we would have p=f. For each resolved market in the data set I looked at its price one day, week and month before resolution and graphed the markets grouped by price against their yes frequency. You can see the results in the following three graphs. Note the green line is the frequency one needs to make a profit at a given price net of fees and bid-ask spread.

Some notes:

Some observations

Let me know if you have any ideas for questions that would be interesting to answer with the data. I might investigate volatility next. If you're interested in receiving access to the data make a bid.

[1] Most might be an interesting thing to quantify next