Quantitative vs. Qualitative Data in Learning
Corporate learning and development is becoming increasingly data-driven. On one hand, learning teams need to be able to track and assess all the different learning experiences. On the other hand, they also need to demonstrate business value. This requires smart use of data collection and analytics. While all efforts towards more data-driven strategies are a step towards the better, we’ve noticed a strong bias towards quantitative data. While quantitative data is important, it’s not quite enough to effectively evaluate learning as a process. To achieve that, we’ll need to also pay attention to qualitative data in learning. To help clear up some of the ambiguity, let’s look at each of the two and how to use them smartly.
What is quantitative data in learning?
Quantitative data, by definition, is something that can be expressed in numerical terms. This type of learning data is used to answers questions such as “how many” or “how often”. In learning, organisations often use quantitative data in the form of tracking:
- Enrolment rates
- Completion rates
- Time spent on learning
- Quiz scores
Unfortunately, this is often where most organisations stop with their learning analytics capabilities. However, the problem is that this type of data tells us absolutely nothing about the learning efficacy or the process itself. While we can always employ better quantitative metrics, such as engagement rates, that will never be enough. In any case, we are always better off with access to qualitative data as well.
What is qualitative data in learning?
There’s a lot of questions that we cannot effectively answer with numbers, hence we need qualitative data. Qualitative data, by definition, is non-numerical and used to answer questions such as “what”, “who” or “why”. Examples of qualitative data in learning could include:
- How the learners progressed through the activities
- The nature and topics of discussion between the learners
- How employees accessed the learning
- How the employees applied the learning on the job
It’s quite evident, that these types of questions go a lot further in understanding behaviours, efficacy and the learning process as a whole. Without this kind of data, you effectively have no idea what works and what doesn’t. Or, you may be able to see the effect (e.g. low completion or engagement rates) but may have no idea of the underlying cause (e.g. irrelevant content, bad user experience). From a learning design perspective, these type of data points are immensely valuable.
How to use quantitative and qualitative learning data meaningfully?
So, in general, there’s a lot of untapped value in qualitative data and understanding the learning process on a holistic level. But of course, that doesn’t mean that you should forget about quantitative data either. Instead, you should always aim to validate ideas and insights derived from qualitative learning data through the use of quantitative data. How else would you know the impact of things at scale?
For instance, we think there’s a lot of value in employee discussions and sharing. These provide a great opportunity for organisations to source knowledge and transfer information between its layers. It often happens that employees in these learning discussions bring up their own best practices and work methods (qualitative), that even the L&D team is not aware of. However, to understand if the practice can be applied across the organisation, we may need to do a survey or a poll to understand the magnitude of the idea (quantitative).
Overall, we believe that a lot of the traditional “LMS metrics” are quite useless for anything other than compliance purposes (and even for that, there are better ways…). To really deliver great learning experiences, organisations need to understand learning as a process and not strip it down to simple numbers of how many people took part. In essence, companies need to focus more on the quality of their data and the ability to quantify the impact of insights derived from qualitative data in learning.
This often requires technical capabilities, such as the xAPI, but once again, buying technology is not enough. Rather, organisations have to understand the meaningful things to measure and cut through the noise. If your organisation needs help in that, or in crafting more data-driven learning strategies in general, we are happy to help. Just drop us a note here and tell us about your problem.