Learning Data – How to Derive Meaningful Insights?

As organisations move towards more data-driven decision making, we often find ourselves requiring more sophisticated data collection and analytics tools. For HR in general, we have become quite comfortable with deriving important insights through data analytics. However, for learning and development, these types of operations on the learning data are still in a bit of a grey zone.

As learning moves beyond “just another compliance exercise”, we find that tracking different streams of data can provide real value. These include insights into our employees’ skill level, the effectivity of our learning content and ultimately, the learning ROI. Here are a few illustrations how modern tracking technologies and data analytics can help you to deliver better and more efficient learning.

Assessing employee’s skill levels and learning through data

Assessing our employee’s learning should probably be one of the most important parts in the whole learning architecture. However, many organisations generally fare poorly in this regard (to be fair, the technology has been around for only a few years). Previously, our tracking of learning activities completed has been reliant on very narrow streams of data: the user’s click on “mark as complete” button and/or user input to formal assessment/tests.

First of all, the fact that the user has marked the learning module complete has very little value other than compliance. These types of tick-box exercises tell nothing about the way the learning content was consumed (if at all!). Thus, they provide little to no insight into whether learning actually happens.

Moreover, the formal assessment or tests are not much better. Sure, they again help us to fill the compliance requirements. However, the problem with formal tests is that they are quite dreadful to the learner and can effectively assess a limited part of learning. They do give us a glimpse into how well the learner knows the theory (or how well Google does…). However, they tell us nothing about whether the learning carries through to their jobs resulting in a behavioural change.

What type of data should we collect and utilise instead?

Instead of this kind of tick-the-box data, we should be collecting and leveraging on more qualitative data. How was the content interacted with? In which order were the learning activities completed? How long did it take to complete the learning and how was that time divided between the different sub-activities?

By collecting data to respond to these types of queries, we are producing actionable insights. For example, a learner with lower time-to-complete more likely had higher skills and confidence in their ability than someone who took longer. Similarly, learners who start tackling the difficult content first are more likely to possess advanced skills.

Using data analytics to measure the effectiveness of learning content

Nowadays, learning content plays a major part in the whole learning experience. As we are investing time and money in providing better content, we sure want to keep track of what kind of results we are getting in return.

Previously we would track whether a piece of content was viewed/consumed/marked complete. But again, these kinds of metrics really don’t provide much value to us. If we commit resources into production of e.g. training videos, we surely prefer a much more detailed view to what’s happening. In the case of the videos, we would want to track how long our videos are being watched. If the learners only watch our fancy and expensive video halfway through, that sure seems like money wasted. We would also want to track how learners proceed on the video timeline. For instance, if many of the learners seem to be jumping back and forth multiple times, our video might have failed to communicate the key messages clearly enough.

Using this kind of simple analytics can be a tremendous help in justifying the investments in learning. It’s much easier to get buy-in from the top of the organization once you can show quality insights into the performance of the content rather than just guess work and gut feelings. This also helps us to analyse the Return On Investment of learning. However, a good measure of ROI should also incorporate metrics from the operational side of the business.

Using learning data to determine the Learning ROI

As mentioned, relying only on learning data to determine the learning ROI only gets us so far. As it is, we are rarely learning for just the sake of learning. Rather, we are providing our employees learning experiences in the hopes of them translating into better business results. Therefore, it is equally important that we try to measure the benefits on the business itself, rather than just the fun/participation/liked/etc. index.

By pooling the learning data we collect with other sets of data from the operational side, we can start to assess how well the learning translates to the learners’ daily jobs and how the skills develop. Let’s take sales training as an example, as it is an area where business benefits are easily demonstrable.

Example: combining CRM and learning data for better insights

To assess the effect of learning on sales performance, we could look at results from the sales training and plot them against several sales metrics, such as number of calls made or conversion rates. This kind of data can should be easily extracted from the company’s CRM software. When you see an increase in any of the metrics in correlation to the training you have given and the learning data, you may have managed to produce positive results. It’s also easy to dig deeper to look at cross-performance on both group and individual level.

Going a bit deeper, we can also identify the individuals who have got the most benefit from learning. Could you perhaps gain even more in performance by giving these individuals additional, targeted learning? Similarly, you can identify the individuals lagging in their KPIs and having just ticked the boxes with their learning. It’s time to explain these individuals that it’s simply not enough to mark the e-learnings completed after a brief glance.

Additionally, you’ll also be able to pinpoint the types of content which seem to be driving the increase in performance. Similar to your learners, you can track every piece of content individually. If a certain type of content, e.g. animations or simulations, seem to be the most effective, start using more of them!

Finally, there are so many streams of data that the applications are practically endless. However, you should first pay attention to whether you are collecting meaningful data or not. And then, secondly, think about how to deliver better insights that benefit the overall business.

Are you looking to implement a more data driven approach to learning and development? Would you like to be able to drive business performance through learning and be able to show proof for it? We are happy to help you with your learning data, just drop us a note here.