Business Intelligence


Measurement Efforts Don’t Quite Measure Up
A disconnect between learning and tangible business outcomes as well as lagging technology may be contributing to dissatisfaction with measurement and metrics practices.

By Ashley St. John

M

easuring the impact of learning and development efforts is an important practice among L&D departments.

Learning professionals have been using Donald Kirkpatrick’s training evaluation model, or some variation of it, for 60 years, since it was first introduced in 1959. Through a variety of post-training assessments, gathering and analyzing learning outputs has become a big part of the day to day in L&D.

And yet, the majority of learning leaders report dissatisfaction in this area.

According to a survey of the Chief Learning Officer Business Intelligence Board, conducted from June to July 2018, a staggering 58 percent of learning professionals are unsatisfied with the extent of learning measurement that occurs within their organization (Figure 1). This is up from 55 percent who reported dissatisfaction with their organization’s learning measurement in 2017.

The Chief Learning Officer Business Intelligence Board is a group of 1,500 professionals in the learning and development industry who have agreed to be surveyed by the Human Capital Media Research and Advisory Group, the research and advisory arm of Chief Learning Officer magazine.

So, what’s fueling this growing dissatisfaction?

Survey results suggest that one contributing factor could be frustration with process. Only 58 percent of respondents said their measurement and metrics are fully aligned with learning strategy. And only 45 percent report externally benchmarking their measurement and metrics practices.

It also could be a failure to tie learning efforts to tangible business outcomes. While 74 percent of respondents said they currently measure employee response to training, only 37 percent measure learning’s impact on sales (Figure 2). While 58 percent measure impact on employee engagement, a mere 28 percent measure impact on Net Promoter Score. And only 27 percent measure the formal ROI for learning.

Technology — or, rather, the lack of it — may also be a factor contributing to learning leaders’ frustration and dissatisfaction. Only 18 percent of respondents report that their organization has the technology to collect, aggregate, integrate and analyze data from multiple HR systems, otherwise known as big data. Forty-five percent say they are able to do so to some extent, and 38 percent say not at all (Figure 3).

Additionally, when it comes to learning metrics process, the vast majority of organizations are struggling with integration. Thirty-seven percent are manually generating metrics, 20 percent are automating metrics from their learning management system and 14 percent have no formal metrics in place (Figure 4). Without an integrated LMS and enterprise resource planning, learning professionals face greater difficulty in effectively and quickly analyzing data.

On the bright side, 71 percent of learning professionals say their organization plans to increase its learning analytics capability over the next two years. What exactly that entails, and the effects it may have on learning professionals’ satisfaction with measurement practices, remains to be seen.

Figures’ Source: Chief Learning Officer Business Intelligence Board, N=500. All percentages rounded.

Ashley St. John is Chief Learning Officer’s managing editor. She can be reached at editor@CLOmedia.com.