By Hollace S. Masino
Instructional Designer
Before going to graduate school for instructional design, I worked as a training facilitator for teachers in secondary and higher education. Evaluation was an important part of my job: I gave knowledge checks and surveys after trainings, and smiled at how satisfied participants seemed when they scored well. But what value were those participants really getting for the time, energy, and money spent? Did skills learned carry over to job performance? How were their institutions benefiting from sending staff to professional development? These were questions I didn’t ask and they weren’t covered by the surveys. Years later, I realized I should have adopted a more comprehensive process to evaluate my trainings.
Why? Because when done well, evaluation ensures you’re accomplishing what you set out to do—increase the knowledge and skill of participants and positively impact their performance to ultimately attain your organization’s business goals. Moreover, evaluation enables you to provide compelling evidence to leadership that a training program not only delivers on learning objectives but also creates results on the job and contributes to the organization’s mission. In this post, I’ll share key considerations for developing such an effective evaluative approach.
Components of Effective Evaluation
Your evaluation process should systematically look at the merit of your training program so you understand where and how to make improvements. Also, it should enable you to collect the data you need but not be overly complicated, adversarial, or stressful. In my experience, there is one approach that checks all these boxes: Kirkpatrick’s Four Levels of Training Evaluation.
Among the many evaluation frameworks I encountered in graduate school, I’ve mainly used Kirkpatrick’s model because of its straightforward simplicity. Developed by Donald Kirkpatrick while writing his PhD dissertation in the 1950s, it’s now one of the most highly renowned methods for training evaluation. I apply it in my work to assess the value of training programs and make informed decisions concerning what I can do to improve them.
The eponymous four levels are reaction, learning, behavior, and results. The table below presents each level with a description, questions to ask participants, and examples of tools to collect the information you’ll later analyze.
Four-Level Framework for Evaluating Training Programs
| What Is It? | Types of Questions to Ask | Examples of How to Measure | |
| Level 1: | The extent to which training is engaging and relevant. | Ask participants about the training.
|
|
| Level 2: | The extent that participation in training improved knowledge and skills as well as how it affected their attitude, commitment, and confidence levels. | Ask participants about the content.
|
|
| Level 3: | The extent to which participants use what they learned during training in their jobs. | Ask about how training affected participant performance.
| Sometime after training:
|
| Level 4: Business Results | The extent that organizational goals, objectives, and expectations were met as a result of training. This is the most challenging level to evaluate. It requires that clear business goals and standards are communicated ahead of training and evaluation to set targets and connect the dots from training to results. |
| Depending on the goals and mission, use bottom-line business metrics such as:
|
Start With the End in Mind
Notice that as the levels increase, so do the complexity of questions and metrics and the amount of time they take to create and apply. The challenge is to identify how business outcomes are linked to training. To help overcome this challenge, you can begin training and evaluation planning at Level 4 and work backward to Level 1. This helps prioritize goals and objectives as well as provide necessary links in the chain.
Keep in mind that it’s essential to plan for an evaluation of a training program while you plan the program. Determine the goals on all four levels ahead of time and start the process with predetermined benchmarks and targets. That way, you’re aware of what you’re shooting for before, during, and after the training process. You’ll know when you’ve hit your targets—or not—and can make better judgments and decisions to help you and your organization effectively allocate time, energy, and money.
Hollace S. Masino is an instructional designer with a master’s degree in instructional technology and over 15 years of experience in training, education, and leadership. She has filled a broad range of roles, including facilitator, curriculum designer and developer, and director of training programs. Her areas of interest are motivational design, human performance improvement, and the higher stages of human development. At Encompass, Hollace plays an integral role within the Instructional Design team and has been instrumental in the creation and deployment of a variety of professional development and sales training courses.
