In Big Data, Small Findings Can Have a Big Impact. Or Not

In HR analytics we are often so eager to discover something significant in the data that we hesitate to think about the impact of the finding. For example, analysis of a big data set might show a statistically significant difference in performance between baristas who like jazz and ones who like rock. That’s interesting for recruiting, but if the difference is that jazz loving baristas are, say, 3% faster on cash then it’s not very important.

This issue highlights a difference between analysis in science and business. In science we are usually interested in findings that have a high degree of certainty, so we set a high bar for statistical significance. In business we are interested in findings that have an impact on the business. If one customer gets sick after visiting our cafe, then we will take it seriously even though it’s only a single case, because, on the off chance the coffee really is poisoning some customers (or social media comes to believe it is), then the impact is great.

The challenging thing for people analytics is that the size of the impact in all the social sciences is usually small. Furthermore, if the impact is big (e.g. short people perform poorly at basketball) then chances are the business has already noticed without any fancy analytics.

If it gives any comfort, biology has the same problem with genes. There isn’t one gene for height, there are dozens, each one only having a small impact. If you are trying to genetically engineer basketball players, it is vastly more difficult than just tweaking one gene.

Article Continues Below

I wish I had a nice solution to the problem of small impacts; however, it’s built into the reality of trying to improve people and organizations. There are a few things we can do:

  • Be sure to think about how big the impact of a finding is, rather than worry only about statistical significance. Better you point out this aspect of your finding rather than have someone else draw attention to it.
  • Note that sometimes a small impact makes a big difference. In the 100 meter dash, a 1% difference in speed can be the difference between winning the gold medal and getting no medal at all.
  • The Japanese have shown with the Kaizen system that if you make a great many small improvements it can add up to a big impact on performance. Don’t turn your back on a small gain if you have a method for getting many of them.
  • If the business decision is important then even a small improvement will matter. (See my piece on why the analytics story lies in the issue not the data.) Make sure to focus your people analytics efforts on business issues that really matter.

Perhaps it is also worth saying that part of the issue is the expectations of management. They read the headlines about supposedly fantastic results from some analytics project. In our organizations we have to do a lot of work in educating management about how people analytics really works to add value.

David Creelman, CEO of Creelman Research, is a globally recognized thinker on people analytics and talent management. Some of his more interesting projects included:

  • Conducted workshops around the world on the practical aspects of people analytics
  • Took business leaders from Japan’s Recruit Co. on a tour of US tech companies (Recruit eventually bought for $1 billion)
  • Studied the relationship between Boards and HR (won Walker Award)
  • Spoke at the World Bank in Paris on HR reporting
  • Co-authored Lead the Work: Navigating a world beyond employment with John Boudreau and Ravin Jesuthasan. The book was endorsed by the CHROs of IBM, LinkedIn and Starbucks.
  • Worked with Dr. Wanda Wallace on “Leading when you are not the expert” which topped the “Most Popular List” on the Harvard Business Review’s blog.
  • Worked with Dr. Henry Mintzberg on peer coaching, David’s learning modules are among the most popular topics.

Currently David is helping organizations to get on-track with people analytics.

This work led to him being made a Fellow for the Centre of Evidence-based Management (Netherlands) for his contributions to the field.