Companies know that employee turnover is expensive and disruptive. And they know that retaining their best and brightest employees helps them not only save money but also preserve competitive advantages and protect intellectual capital.

Most retention efforts, however, rely on two retrospective tools. First, exit interviews are conducted to better understand why people chose to leave, though by this point, it is usually too late to keep them. Second, annual employee surveys are used to assess engagement.

These survey results are later compared to people who left the organisation, in the hope that they will yield any relevant predictors of departures. The problem is that these data don’t give managers a real-time picture of who might be considering leaving.

Our latest research has focused on using big data and machine-learning algorithms to develop a turnover propensity index for individuals – a real-time indicator of who is likely thinking about quitting. We grounded the development of these predictive models in academic research on turnover and then conducted a series of studies. Our results demonstrate that it is possible to develop indices that predict in real time the likelihood of a person to consider an outside offer and to eventually leave the firm.

Past research points to two main reasons why people leave their jobs: turnover shocks and low job embeddedness. Turnover shocks are events that prompt people to reconsider whether they should stay with the organisation. Some shocks are organisational (such as a change in leadership or an M&A announcement) and others are personal (for example, receiving an outside job offer or the birth of a child).

Job embeddedness is when people are deeply connected to an organisation. When people have few good social ties at work or in the community, or when they don’t feel their work fits well with their interests, skills, and values, they have low job embeddedness and are a higher flight risk.

We worked with a talent intelligence firm to gather a large sample of publicly available organisational data on potential turnover shocks, such as changes in Glassdoor or analyst ratings, stock price variation, news articles, and regulatory or legal actions against the firm. We also gathered personal factors tied to embeddedness that were in the public domain, such as the number of past jobs, employment anniversary and tenure, skills, education, gender, and geography. We accumulated these potential turnover indicators for more than 500,000 individuals working in the US across various organisations and industries.

Based on our assessment of these turnover factors, we used machine learning to classify each individual as unlikely, less likely, more likely, or most likely to be receptive to new job opportunities. Each individual in our sample was given a turnover propensity index (TPI) score, and then we ran two studies to see how well this score predicted their openness to outside opportunities and their likelihood of quitting.

First, we wanted to see how well the TPI predicted openness to recruitment messages. We sent email invitations to a smaller sample of 2,000 employed individuals who had been identified by our algorithm as unlikely, less likely, more likely, or highly likely to be receptive to an invitation to view available jobs tailored to their specific skills and interests.

How to identify employees most likely of leaving

Of these, 1,473 received the email; 161 opened the invitation; and 40 clicked through. Those who were rated as 'most likely' to be receptive opened the email invitation at more than twice the rate of those rated as least likely (five per cent versus 2.4 per cent).

Additionally, among those who opened the email, those rated as 'most likely' to be receptive were significantly more likely to click through. This suggests that the TPI score could identify employees at greater risk of leaving. This finding also indicates that companies can strategically target top talent that might be more open to an outside offer – remember this all came from publicly available data.

Second, to look at the ability of the TPI score to predict actual turnover, we used the remainder of the sample of 500,000 individuals. Over a three-month time period, those identified as 'most likely' to being receptive to new opportunities were 63 per cent more likely to change jobs, as compared to those who were 'unlikely'. Those identified as 'more likely' were 40 per cent more likely to quit.

Our work in this area is demonstrating that by using big data, firms can track indicators of turnover propensity and identify employees who may be at an elevated risk of leaving the organisation. This proactive anticipation may allow leaders to intervene to increase the odds of retaining top talent. Moreover, organisations have a huge advantage over outside researchers at developing their own TPI using internal data.

Firms can anticipate organisational shocks such as litigation or regulatory actions. As well as publicly available data, firms have access to other turnover shock data, such as work anniversaries, new educational credentials and birth or wedding announcements, though they have to take care not to violate employee privacy. And firms can track factors that signal job embeddedness, such as participation in career development opportunities, organisational improvement initiatives, or peer recognition programmes.

Firms with a commitment to data-driven decision-making will need to make an investment in carefully collecting and analysing the right indicators for turnover risk. Then their leaders can proactively engage valued employees at risk of leaving through interviews, to better understand how the firm can increase the odds that they stay.

Further reading: 

Holtom, B., Goldberg, C. B., Allen, D. G. and Clark, M. A. (2017) "How today’s shocks predict tomorrow’s leaving", Journal of Business and Psychology, 32, 1, 59-71.

Porter, C. M., Woo, S. E., Allen, D. G. and Keith, M. G. (2019) "How do instrumental and expressive network positions relate to turnover? A meta-analytic investigation ", Journal of Applied Psychology, 104, 4, 511-536.

Read the original article at Harvard Business Review.

David Allen is WBS Distinguished Research Environment Professor at Warwick Business School and Associate Dean and Professor of Management and Leadership at TCU Neeley School of Business.

Brooks Holtom is Professor of Management at Georgetown University.

For more articles like this download Core magazine here.