• First-ever ‘index of national happiness’ created going back to 1820
  • Data from books and newspapers proves a reliable guide to national mood
  • National happiness levels of previous centuries measured for the first time
  • Study analyses eight million books and 65 million newspaper articles

Was there such a thing as ‘the good old days’ when people were happier? Are current Government policies more or less likely to increase their citizens’ feelings of wellbeing?

Using innovative new methods researchers at the University of Warwick, University of Glasgow, Warwick Business School and The Alan Turing Institute have built a new index that uses data from books and newspapers to track levels of national happiness from 1820. Their research could help governments to make better decisions about policy priorities.

Governments the world over are making increasing use of 'national happiness' data derived from surveys to help them consider the impact of policy on national wellbeing. Unfortunately, data for most countries is only available from 2011 onwards, and for a select few from the mid-1970s. This makes it hard to establish long-run trends, or to say anything about the main historical causes of happiness.

In order to tackle this problem, a team of researchers including Thomas Hills, of the University Warwick and The Alan Turing Institute, Eugenio Proto, of the University of Glasgow, Daniel Sgroi, of the University of Warwick, and Chanuki Seresinhe, of Warwick Business School and The Alan Turing Institute, took a key insight from psychology – that more often than not what people say or write reveals much about their underlying happiness level – and developed a method to apply it to online texts from millions of books and newspapers published over the past 200 years.

In the paper Historical Analysis of National Subjective Wellbeing using Millions of Digitized Bookspublished in Nature: Human Behaviour, the main source of language used for the analysis was the Google Books corpus, a collection of word frequency data for more than eight million books – that’s more than six per cent of all books ever published.

The method uses psychological valence norms - values of happiness that can be derived from text - for thousands of words in di?erent languages to compute the relative proportion of positive and negative language for four di?erent nations (the US, UK, Germany and Italy).

Professor Proto said: “Our index is an important first step in understanding people’s satisfaction in the past. Looking at the Italian data, it is interesting to note a slow but constant decline in the years of fascism and a dramatic decline in the years after the last crisis.”

The research team also controlled for the evolution of language, to take into account the fact that some words change their meaning over time.

Dr Seresinhe, of the WBS Data Science Lab, said: “It was really important to ensure that the changing meaning of words over time was taken into account. For example, the word 'gay' had a completely different meaning in the 1800s than it does today.

"We processed terabytes of word co-occurrence data from Google Books to understand how the meaning of words has changed over time, and we validated our findings using only words with the most stable historical meanings."

The new index was validated against existing survey-based measures and proven to be an accurate guide to the national mood. One theory as to why books and newspaper articles are such a good source of data is that editors prefer to publish pieces which match the mood of their readers.

Happy days: National Valence Index plotted from 1820 to 2009

Professor Sgroi said: "Aspirations seem to matter a lot: after the end of rationing in the 1950s national happiness was very high in the UK as were expectations for the future, but unfortunately things did not pan out as people might have hoped and national happiness fell for many years until the low-point of the Winter of Discontent."

Studying the index, the researchers found that:

  • Increases in national income do generate increases in national happiness but it takes a huge rise to have a noticeable effect at the national level.

  • An increase in longevity of one year had the same effect on happiness as a 4.3 per cent increase in GDP.

  • One less year of war had the equivalent effect on happiness of a 30 per cent rise in GDP.

  • In post-war UK the worst period for national happiness occurred around the appropriately named Winter of Discontent.

  • In post-war US the lowest point of the index coincides with the Vietnam War and the evacuation of Saigon.

Professor Hills said: “What’s remarkable is that national subjective wellbeing is incredibly resilient to wars. Even temporary economic booms and busts have little long-term effect.

"We can see the American Civil War in our data, the revolutions of 48’ across Europe, the roaring 20’s and the Great Depression. But people quickly returned to their previous levels of subjective wellbeing after these events were over.

"Our national happiness is like an adjustable spanner that we open and close to calibrate our experiences against our recent past, with little lasting memory for the triumphs and tragedies of our age.”

Chanuki Seresinhe is a Visiting Data Researcher of the Alan Turing Institute and did her PhD at the WBS Data Science Lab.

Follow her on Twitter @thoughtsymmetry

For more articles like this download Core magazine here.