Elearning! Magazine: Building Smarter Companies via Learning & Workplace Technologies.
Page 23 of 52
ing business measures to money, a direct, mathematical comparison to costs can be made, which gives a more accurate accounting of how well the program con- tributed to the business than by reporting benefits as intangible only. Because data are collected at all levels of the chain of impact, the result is: >>Evidence of program success important to all stakeholders
>>Explanations for how you arrived at the ROI
>>Information to help improve the pro- gram
To ensure an acceptable ROI in e-learn-
ing, success at all levels in the chain of impact must occur. By aligning your e- learning programs to the business, you can increase your chances of this occurring.
CLARIFY STAKEHOLDER NEEDS
Initial alignment occurs when stakeholder needs are identified, which begins with potential payoff for opportunities or prob- lems. These payoff opportunities represent an organization's opportunity to make money, save money and/or avoid costs. Some payoff opportunities are obvious, such as a $1 million cost due to unwanted employee turnover. Other payoff opportu- nities are not so obvious, like the desire to become a great place to work. The payoff opportunity represents the potential mon- etary impact that can be achieved by investing resources in various programs and projects, including e-learning. These are the Level 5 needs. Once clear about the potential payoff
opportunity, the next step is to identify the specific business measures that, if improved, will position a program to take advantage of the payoff opportunity. Business needs (Level 4) represent meas- ures of output, quality, cost, and time as well as measures of customer satisfaction, employee satisfaction, work habits, and innovation. Next, identify the performance needs.
Performance needs (Level 3) are those behaviors or actions taken, or not, that if changed will address the business needs. A performance need may be a people issue, a process issue, or even a policy issue. For example, a large pharmaceutical
manufacturer was incurring excessive costs associated with pre-filled syringes
(payoff need). Upon further investiga- tion, managers found that the specific business measure that needed the most attention was false rejects of good syringes (business need). By meeting with inspectors, management found that they were inconsistent in their process. This resulted in some inspectors throw- ing out perfectly good syringes. This inconsistent process represents the per- formance need (Level 3).
A performance need may be a people issue, a
process issue, or even a policy issue.
Levels of Evaluation 0. Inputs and
indicators
1. Reaction and perceived value
2. Learning
Once the performance need is identi-
fied, the next step is to determine what people need to know (Level 2, learning needs) in order to change their perform- ance. In the example, the learning need is that inspectors need to know the differ- ence between acceptable and unacceptable syringes. In addition inspectors needed to recognize the importance of a consistent approach to inspecting syringes. After learning needs are clarified, the
next step is to determine preference needs (Level 1) by asking: How do we deliver the knowledge, skill or information people need to know in order to change perform- ance so that business measures can improve and we can take advantage of the payoff opportunity? The final needs that must be addressed
are the input needs (Level 0). These needs represent the investment required for implementing the solution.
DEVELOP OBJECTIVES
Program objectives correspond to stake- holder needs. Developing powerful pro-
Chain of Impact
R.O.I. DATA COLLECTION Measurement
Focus
The input into the project in terms of scope, volume, efficiencies, costs
Reaction to the project or program, including the per- ceived value
Learning to use the content and materials, including the confidence to use what was learned
3. Application and implementation
Use of content and materials in the work environment, in- cluding progress with actual items and implementation
4. Impact
The consequences of the use of the content and materials expressed as business im- pact measures
5. R.O.I.
Compassion of monetary benefits from program-to- program costs
Typical Measures
Participants, Hours, Costs, Timing
Relevance, Importance, Usefulness, Appropriateness,
Intent to use, Motivation to take action
Skills, Knowledge, Capacity, Competencies, Confidence, Contacts
Extent of use, Task comple- tion, Frequency of use, Ac- tions completed, Success
with use, Barriers to use, En- ablers to use
Productivity, Revenue, Qual- ity, Time, Efficiency, Teacher satisfaction, Student engage- ment
Benefit-Cost Ratio (B.C.R.), R.O.I. %, Payback period
Elearning! March / April 2012 23
