Elearning! Magazine: Building Smarter Companies via Learning & Workplace Technologies.
Figure 2 A Framework for Measuring Learning 2.0
Learning 2.0 Component
Measurement Instrument
lytics such as the amount of time spent reading certain topics is worth gathering as formative data. Other tools can also be added, such as
pop-up surveys and micro-polls, to gather information about quality, usefulness, expected performance improvement, and business impact. With these tools, brevity is essential so the evaluation process is only minimally invasive. Figure 2 represents a simple framework
for measuring the impact of Learning 2.0. To measure impact effectively, the
learning component must be identified, and the appropriate instrument must be deployed at the right time. When multiple sets of content and multiple instruments are used, a system capable of organizing, storing, analyzing and reporting results is also required. Figure 3 shows an example of how the
framework in Figure 2 can be used to measure the impact of a series of Learning
Time of Measurement
It is not too difficult to measure impact. It
begins with
asking the right questions in your poll or survey.
2.0 components. More information about measuring
informal learning is available via KnowledgeAdvisors, including how to use evaluation systems to collect, analyze,
Data Storage
Analysis & Reporting
and report results from disparate Learning 2.0 sources.
MOVING BEYOND SATISFACTION METRICS
Over the years, learning organizations have attempted to measure the impact of traditional learning using surveys and polls, but most settle for achieving reac- tion or satisfaction metrics. It is likely that the same thing will happen with informal learning as well. However, it is not too difficult to meas-
ure impact. It begins with asking the right questions in your poll or survey—ques- tions that relate to application, perform- ance improvement, and business outcomes. While it is important to ask learners if
they like the user interface on a knowl- edge portal, it is more important to ask, "Will you use the information you just learned?" and "Will it improve your per- formance?" Quantifying the responses to
Figure 3 An Example for Measuring the Impact of Learning 2.0.
Learning 2.0 Component
Knowledge portal
Knowledge portal
Mentoring program
Communities of practice
Mobile device podcasts
Measurement Instrument
Web analytics
Pop-up survey
Formal survey
Micropoll
Formal survey
Time of Measurement All users
Every 10th user
Twice annually to sample
Every 5th user
After completion of each
Data Storage Website
Evaluation system
Evaluation system
Evaluation system
Evaluation system
Analysis & Reporting
Avg. time on page; clicks
Value and
usefulness of material
Impact of coaching
Value of info; intent to apply
Value & impact on performance
Elearning! May / June 2012 41