Don’t Ignore What People Say Just Because It’s What People Say

Qualitative data can be tough for evidence-based professionals because the whole idea of evidence-based management is to get away from opinion and focus on fact. Yet qualitative data, such as anecdotes and opinions, can be seen as forms of evidence that are too valuable to simply ignore.

The key concepts we need to pull from the theory of evidence-based management are:

  • We want to consider all the available evidence
  • We want to assess the quality of evidence

For example, if our receptionist tells us that a lot of people are unhappy about a certain decision then we should add that observation to our evidence pool — even if data from the engagement survey indicates otherwise. The idea is to cast a wide net in gathering all the available evidence. The receptionist may be picking up signals the formal survey does not.

The second step is to assess the quality of evidence. If the receptionist is seen as reliable and is in a position to interact with many different people, then their observations should be treated as reasonably good evidence. If the receptionist complains about a lot of things and talks with relatively few people, then their observation will probably be considered low quality evidence.

Get all the evidence in front of you, consider the quality, and make an informed judgement. It is possible to do this in a highly disciplined way using Bayes’ Rule (see Chapter 12 in Evidence-Based Management: How to use evidence to make better organizational decisions by Eric Barends and Denise Rousseau); however, more commonly, analysts just make a judgement call.

Article Continues Below

It’s worth remembering that objective quantitative data can be of lower quality than qualitative data. An engagement survey that is two-years old may be less accurate than a receptionist’s observations. Objective data that doesn’t have predictive value is also low-quality evidence. For example, data on someone’s elementary school grades, objective though they are, may have very little value as evidence as to who will be good at sales.

Analytics professionals ought to push managers towards making decisions based on evidence. They will be more successful at that if they recognize there can be value in qualitative evidence; it’s just a matter of assessing the quality.

David Creelman, CEO of Creelman Research, is a globally recognized thinker on people analytics and talent management. Some of his more interesting projects included:

  • Conducted workshops around the world on the practical aspects of people analytics
  • Took business leaders from Japan’s Recruit Co. on a tour of US tech companies (Recruit eventually bought for $1 billion)
  • Studied the relationship between Boards and HR (won Walker Award)
  • Spoke at the World Bank in Paris on HR reporting
  • Co-authored Lead the Work: Navigating a world beyond employment with John Boudreau and Ravin Jesuthasan. The book was endorsed by the CHROs of IBM, LinkedIn and Starbucks.
  • Worked with Dr. Wanda Wallace on “Leading when you are not the expert” which topped the “Most Popular List” on the Harvard Business Review’s blog.
  • Worked with Dr. Henry Mintzberg on peer coaching, David’s learning modules are among the most popular topics.

Currently David is helping organizations to get on-track with people analytics.

This work led to him being made a Fellow for the Centre of Evidence-based Management (Netherlands) for his contributions to the field.