The classic model for HR analytics is that you have some kind of data warehouse and you use it to answer a business question.
In other words you start with the data at hand and then you find the answer.
However, a big part of analytics works in the opposite direction. You don’t have enough pre-existing data, so sometimes you just have to launch an initiative and ask yourself, “How will we know if this is working?”
Sometimes it helps to run a pilot
In essence this is simply the old principle of running a pilot. However analytics adds two new things:
- Being more disciplined about running pilots. One new thing is simply following the discipline of actually running a pilot wherever practical. In a culture of data-based decision making, the idea of just doing something, without gathering evidence on how well it’s working is frowned upon.
- Being more sophisticated about running pilots. If you go into a pilot with the clear idea that you will be gathering some kind of evidence to assess if it’s working, then you will more careful in how you design the pilot so that you can collect good data.
A good example of taking some care in designing pilots comes from Dr. Donald Ledbetter. In running an important training invention he made a point of ensuring he had a control group so that he would have solid data on whether it worked.
Another example of careful design is to avoid changing five things at once, because if you do you’ll never know which of the five changes is having an effect.
Finally, if the organization is in the midst of some big reorganization then running a pilot may be a waste of time because you won’t be able to tell is changes were the result of the initiative or the massive reorganization.
Running pilots isn’t new, but often we are very sloppy about how we design the pilot and how we collect data. We can do better by thinking more carefully about how we will assess “Is it working?” before we launch a pilot.
Takeaway
Recognize that gathering data from pilots is as important a part of HR analytics as analyzing information in an existing data warehouse. Develop the skills and discipline to do so.