Tim Harford on how to be a superforecaster
27 October 2014
Financial Times columnist Tim Harford believes only when economists use probability will their forecasting produce better results.
The man known as the Undercover Economist revealed research has found economic forecasters are no better than chimps at predicting how economies will perform in the future.
But Harford feels a new breed of superforecaster can banish this reputation and says using probability is key to their success.
“Training in probabilistic thinking can really help,” said Harford, who was talking in front of the Warwick Economic Society at the University of Warwick. “As an example, those looking into Assad’s future in Syria could look at other transitions, the situations in which other leaders were toppled in Middle Eastern countries.
“Such training can really make the difference in a successful superforecaster.”
At the moment, however, many forecasters are not applying this and other such processes to their thinking, thus limiting their success according to Harford. He pointed to research by Philip Tetlock, in his book Expert Political Judgement, to imply chimps are, in essence, just as effective as humans when it comes to forecasting.
Harford, who also hosts BBC Radio 4's More or Less, said: “This may be a somewhat simplified, but not wholly unfair summary of the book, but if you want to forecast an area of major geo-economical concern, macro-economics, the future of Ukraine, etc. get a chimp.
“Chimps are just as good as geopolitical or macro-economic experts at forecasting. Tetlock suggested that when using a binary choice of 50/50 for example, it was just as accurate as an expert forecaster.
“While forecasters might be good at spotting certain things, they are also very good at predicting things that never happened.”
Harford, who has written four books including his latest The Undercover Economist Strikes Back, suggested to the sell-out crowd that forecasting should not be written off however. Tetlock’s dedication to proving forecasting could be done well led to the emergence of a group of what he has referred to as superforecasters, many of whom are involved in his Good Judgment Project.
What makes superforecasters a different proposition to the lamented forecasters according to Harford?
“One, keeping score is important,” said Harford. “Constant feedback and going back to check how a forecast faired are vital to a superforecaster. Normal forecasters are not subject to such stringent analysis.
“Two, training in probabilistic thinking really helps.
“Three, teamwork is really helpful, if anything the superforecasters got even better when teamed with other superforecasters.
“Finally, number four; an actively open-minded thinking style. This means enjoying meeting people with contrasting views, being comfortable with changing your mind and the ability to accept when you are wrong – a regular occurrence in the messy world of forecasts.”
Harford drew comparisons between American economist Irving Fisher’s bullheadedness when it came to forecasting and the Seeker cult of Chicago in the 1950s. Psychologist Leon Festinger, studied the group of believers who were sure there was an upcoming alien apocalypse.
When the impending end never came, however, far from the group casting aside their beliefs, he found they actually became stronger.
Fisher had this same sense of unfaltering belief in spite of evidence contrary to his views on stocks, namely the 1929 Wall Street crash. Even when he said “stocks have reached what looks like a permanently high plateau” very shortly before the crash said otherwise, still he refused to budge in his thinking.
In contrast economist John Maynard Keynes, Harford points out, was more than willing to change his world view, perhaps because he had experienced such falls from grace before.
“Ultimately the different trajectory of Fisher and Keynes emphasises what makes a good forecaster,” said Harford.
“Keynes often changed his mind, and would say something different. He was good at keeping score, but self-critical, he knew which investments under performed for King’s College. Moving in and out of industries, he was not happy as he was 18 per cent down. He knew this, and was worried about an impending crash, although he was not expecting it.
“When the crash came he lost money, but it didn’t challenge his fundamental world view. Criticising himself, he instead picked a small number of companies he understood, known as his ‘pets’. He kept investing very much like Warren Buffett does now.
“Did it work? Yes. While he initially unperformed by 1946, he was beating the market by six per cent a year on average - a great success.
“Fisher, however, ended up broke and in poverty. His optimism, overconfidence and stubbornness betrayed him. As Keynes said: ‘When my information changes, I alter my conclusions’. If only he had taught that lesson to Irving Fisher.”