Advertisement

Does Intuition Have a Place In Analytics?

Article main image
Mar 8, 2018
This article is part of a series called Editor's Pick.

When you walk into a meeting and immediately think “this is going to be trouble” that feeling is not based on objective, measurable factors; it’s intuition. As analysts who are trying to shift from a world of opinion to a world of data, what do we do about intuition?

Malcolm Gladwell describes the science behind intuition in his bestseller Blink.

The starting point is to recognize that intuition is a form of cognition. That sense that the meeting is going to be trouble is not an unfounded opinion, it’s based on an unconscious processing of information about the situation. We may have no idea what information led to that intuition; our attempts at explaining our own thinking may be wrong, but that doesn’t mean the intuition has no

value. At a minimum we have to accept intuition can sometimes provide accurate insights.

 

The trouble is, that as soon as we accept intuition as a valid form of evidence, we run into people saying, “Well the data shows one thing, but my opinion — or rather my intuition — shows we should do something different.”

We could potentially try to distinguish between insightful intuition and unfounded opinion. There is some research that is helpful. The research shows that when someone is in a role that provides rapid accurate feedback their intuition can be very good. For example, if someone goes to many meetings, and it’s clear by the end of the meeting if it was or wasn’t trouble, then they are in a position to develop good intuition about meetings. On the other hand, if someone in in a job that provides a small amount of poor quality feedback then their intuition probably won’t be good. For example, if someone’s job is providing therapy meant to contribute to a person changing over the years, then there is just not enough timely, accurate data for them to develop reliable intuitions — though that doesn’t mean they won’t develop strong opinions.

If we need to make a decision right away, and we have some hard data facing off against soft intuitions, then we should take that valuable step of assessing the quality of evidence. Perhaps the data is hard, but sparse, or not a good measure with regard to the issue — in that case even though it’s hard data, the quality may be low. Perhaps the intuition is based on the kind of repeated experience that leads to reliable insights, in that case, even though it’s intuition, the quality of the evidence may be high. We weigh the evidence, we make our choice.

However, the most useful role of intuition is to use it to stage a hypothesis. If someone says their intuition suggests that, for example, a two-week onboarding program is no more effective than a one-week program, then we should take that as a hypothesis to be tested. We don’t tell the person that we don’t value their intuition, we value it as a light to illuminate something we should investigate. This shows respect for the person, it takes advantage of the power of intuition, but at the same time it mitigates the unreliability of intuition by initiating a search for harder evidence.

∼∼∼∼

Special thanks to our community of practice for these insights. The community is a group of leading organizations that meets monthly to discuss analytics and evidence-based decision making in the real world.  If you’re interested in moving down the path towards a more effective approach to people analytics, then email me at dcreelman@creelmanresearch.com

This article is part of a series called Editor's Pick.