Did Crowd Wisdom Screw Up? Why Do Prediction Markets Fail?

Two weeks ago I wrote this piece about the then upcoming Greek Referendum. Given the outcome, it certainly seems that my argument needs some revisiting. Here’s what I wrote:

It’s very insightful to look to the betting markets for indicators. Memories are still fresh of March’s Israeli election and last year’s Scottish independence referendum, when the betting sites were right and the opinion polls were way off. In Israel, 2 weeks before the elections, while opinion polls where all favoring the Labor Party, 81% of gamblers were betting on Netanyahu to come through as the victor. In Scotland, while most opinion polls were split down the middle, political betting lines have unanimously experienced much higher “NO” volumes. In this case the current betting jury is out. Predictit is leaning towards a 2015 EuroZone exit while other betting sites are pointing at no EuroZone exit anytime soon.

Guess what? The predictive markets got it wrong and so did I! Paddy Power had to cough out a substantial amount of money, a result of being left Բed-facedԼ/a> after paying out early to gamblers who incorrectly bet that Greek voters would back an austerity referendum.

Barry Ritholtz has written an insightful, must read and very critical piece on the false wisdom associated with the crowd in such situations. He attributes the frequent failure of prediction markets to a number of factors.

  1. Real Markets are diverse and trade in huge volumes. Prediction markets are not. This is key to understanding crowd failures, simply because the crowd is best when it creates a thinking market where such doesn’t exist.
  2. Prediction Markets stakeholders are not close enough to the actual decision makers. In the case of the Greek Referendum, for example, the Paddy Power gamblers were not Greek voters. This is another key, as the crowd must operate under clear incentives in order to create an objective and true judgment.
  3. Polling data was out, which means the predicting market did not predict a future market but reacted to a “past market”. Or in simple words, they acted as a poll interpreting focus group, and not as a good one compared to scientifically designed ones. This is crucial as crowd wisdom requires independent individual thinking and unbiased “vote casting”.

In a way, this reinforces the basic criteria the crowd requires in order to produce real wisdom: diversity, unbiased and independent thinking, incentives for providing the real opinion, and others.

Good luck with creating this type of environment anytime you wish to leverage crowd wisdom

Does this mean we should we give up on crowd predictions? Of course not. It only means we should continue and learn the difference between use and abuse. Crowd wisdom cannot be trusted blindly. The crowd doesn’t “know” what we “don’t know” as people and sometimes it is wise, as Howard Tullman notes, to forget about the crowd and focus on who matters.

My argument is that crowd wisdom can be very effective in areas where efficient markets haven’t manifested themselves. There’s no point in predicting the price of a stock which is already priced by the crowd in its market form. However, Crowd Wisdom can prove super effective when it creates the dynamics of a free market when such does not exist. This can be witnessed in the following cases:

  1. Predicting if an innovative product will capture significant market share. Kickstarter has demonstrated that very well and there are other alternatives which focus on that alone.
  2. Bubbling up supporting information such as threats, opportunities & sensitivities (using the actual individual sense rather then the classic, aggregated crowd wisdom definition). This is a very promising area which is yet to be fully explored.
  3. Identifying important undercurrents before analysts do. When is the tech bubble going to burst? is a great example of an interesting case study for crowd wisdom.
  4. Finding data that actually represents crowd wisdom but that does not involve explicit crowd “polling”. Investigating Google Trends is a great example of a different way of looking at crowd wisdom.

And in short, keep your eyes open, your brain functioning and your opinions flexible. Or, in the words of Charles H. Spurgeon:

“Wisdom is the right use of knowledge. To know is not to be wise. Many men know a great deal, and are all the greater fools for it. There is no fool so great a fool as a knowing fool. But to know how to use knowledge is to have wisdom.”

Aner RavonJuly 15, 2015

Aner Ravon

A technologist with an economists eye for critical thinking. I love building shiny new things and asking tough philosophical questions. I am perfectly comfortable being uncomfortable.