As an advocate of analytics its pains me to reveal the truth that analytics never delivers answers. No matter how big your data set or how clever your analysis, you will never be able to prove a proposition beyond doubt.
Your analysis might indicate that MBAs perform better than high school drop-outs in a certain job, but there will always be multiple uncertainties. For example, maybe the sample of workers didn’t allow for an apples-to-apples comparison or maybe your measures of performance were unreliable. MBAs seem to be better, but you can’t be sure.
This issue is familiar to scientists, in fact it lies at the heart of the scientific method: you can disprove a hypothesis, but you can never absolutely prove one.
You may take heart that scientists have gotten quite good at showing that a given hypothesis is really, almost certainly, for sure, true, however in the world of business we are unlikely to achieve that. In the real world of business analytics, we are forever doomed to looking at the data and proclaiming that it seems to show X is true — but maybe not.
You don’t get much kudos for saying “Maybe the data shows that we’re better off investing in onboarding than selection,” or “All things considered, the evidence seems to be saying that our incentive program isn’t working.” Yet, that is the accurate way to frame analytics and if we fail to say it then we may trap ourselves in making claims that are later shot down.
The key to escaping this dilemma is the central lesson I’ve been teaching leaders about analytics: the whole point is to make, on average, better business decisions. If “all things considered” the data seems to show option A is better than option B, then as business people we know what to do. This is far better than the alternative of picking an option based on nothing more than one person’s opinion.
Paralysis by over-analysis
There is another valuable lesson here about when to stop analyzing. If we think we can’t stop until the answer is certain, then we’ll be analyzing forever. If we know that we’re going to have to make a judgement call, all things considered, then we’ll be more comfortable making a decision even when some questions remain unanswered.
There can still be a problem if leaders won’t accept a mushy “all things considered the data shows…” type of answer; however, this is an organizational politics problem, not an analysis problem and we’ll leave the answer to that to another time.
∼∼∼∼
Special thanks to our community of practice for these insights. The community is a group of leading organizations that meets monthly to discuss analytics and evidence-based decision making in the real world. If you’re interested in moving down the path towards a more agile approach to people analytics, then email me at dcreelman@creelmanresearch.com or connect to me on LinkedIn.