An Uncomfortable Truth About Analytics

As an advocate of analytics its pains me to reveal the truth that analytics never delivers answers. No matter how big your data set or how clever your analysis, you will never be able to prove a proposition beyond doubt.

Your analysis might indicate that MBAs perform better than high school drop-outs in a certain job, but there will always be multiple uncertainties. For example, maybe the sample of workers didn’t allow for an apples-to-apples comparison or maybe your measures of performance were unreliable. MBAs seem to be better, but you can’t be sure.

This issue is familiar to scientists, in fact it lies at the heart of the scientific method: you can disprove a hypothesis, but you can never absolutely prove one.

You may take heart that scientists have gotten quite good at showing that a given hypothesis is really, almost certainly, for sure, true, however in the world of business we are unlikely to achieve that. In the real world of business analytics, we are forever doomed to looking at the data and proclaiming that it seems to show X is true — but maybe not.

You don’t get much kudos for saying “Maybe the data shows that we’re better off investing in onboarding than selection,” or “All things considered, the evidence seems to be saying that our incentive program isn’t working.” Yet, that is the accurate way to frame analytics and if we fail to say it then we may trap ourselves in making claims that are later shot down.

The key to escaping this dilemma is the central lesson I’ve been teaching leaders about analytics: the whole point is to make, on average, better business decisions. If “all things considered” the data seems to show option A is better than option B, then as business people we know what to do. This is far better than the alternative of picking an option based on nothing more than one person’s opinion.

Paralysis by over-analysis

There is another valuable lesson here about when to stop analyzing. If we think we can’t stop until the answer is certain, then we’ll be analyzing forever. If we know that we’re going to have to make a judgement call, all things considered, then we’ll be more comfortable making a decision even when some questions remain unanswered.

Article Continues Below

There can still be a problem if leaders won’t accept a mushy “all things considered the data shows…” type of answer; however, this is an organizational politics problem, not an analysis problem and we’ll leave the answer to that to another time.

∼∼∼∼

Special thanks to our community of practice for these insights. The community is a group of leading organizations that meets monthly to discuss analytics and evidence-based decision making in the real world.  If you’re interested in moving down the path towards a more agile approach to people analytics, then email me at dcreelman@creelmanresearch.com or connect to me on LinkedIn.

David Creelman

David Creelman, CEO of Creelman Research, is a globally recognized thinker on people analytics and talent management. Some of his more interesting projects included:

  • Conducted workshops around the world on the practical aspects of people analytics
  • Took business leaders from Japan’s Recruit Co. on a tour of US tech companies (Recruit eventually bought Indeed.com for $1 billion)
  • Studied the relationship between Boards and HR (won Walker Award)
  • Spoke at the World Bank in Paris on HR reporting
  • Co-authored Lead the Work: Navigating a world beyond employment with John Boudreau and Ravin Jesuthasan. The book was endorsed by the CHROs of IBM, LinkedIn and Starbucks.
  • Worked with Dr. Wanda Wallace on “Leading when you are not the expert” which topped the “Most Popular List” on the Harvard Business Review’s blog.
  • Worked with Dr. Henry Mintzberg on peer coaching, David’s learning modules are among the most popular topics.

Currently David is helping organizations to get on-track with people analytics.

This work led to him being made a Fellow for the Centre of Evidence-based Management (Netherlands) for his contributions to the field.

n