Donald Trump’s Trump University has been in the news lately. Allegations are that while the organization purported to be an educational institution providing “real estate secrets” from Donald Trump, in reality it was a sales organization focused on selling products and services. Several lawsuits are pending, and Trump’s political opponents point out that statements and claims by Trump himself about the “university” have proven to be false. Human resource development professionals take note: this seems to be simply a politically charged, high profile example of level 1 vs level 3 evaluation. They liked it, but could and did they use it?
Trump claims, according to a November 2015 article in Time magazine, that 98% of the students who took the $1500 real estate course gave it “rave ratings.” He has copies of the end-of-class surveys students completed to back up claims that students believed they got something of value in the classes. Attorneys for the plaintiffs, that is, students who took the class and are now suing, state that the survey results were not credible because they were not anonymous and were completed at the end of the class — level 1, in Kirkpatrick terminology. Trump’s lawyers have pointed out that many of the students filled out favorable end-of-class questionnaires and later “changed their minds”. They changed their minds about the value of the class when they tried to apply what was taught – level 3.
While most learning evaluations are not as high-profile as this one, it provides an opportunity to pause and take note:
- Do you conduct level 1 end-of-class evaluations of all classes? If you, as 97% of workplace learning professionals do, do you use them to guage the success of the class? How is learning success defined in your organization? Sandra Merwin, noted workplace learning consultant and now artist, did some research years ago that found a negative correlation between level 1 “likes” and level 3 “applies”. In other words, individuals who at the end of class did not give high “like” marks to a class at the end were more likely to apply what they learned. And those who did give high “like” scores after training were less likely to apply it. The explanation might be that when prior knowledge is challenged, it sets up an uncomfortable creative tension. Just one study, but something to think about.
- Consider surveying participants and perhaps their managers three months post-training to see how they’re applying what they learned. Doing this will likely provide more value than end-of-class, level 1 evaluations. My Sticky Note Easy Level 3 Evaluation provides useful tips for doing this.
- For completed classes with level 1 evaluation data available, conduct a level 3 evaluation and compare the results. Some LMSs can provide graphic comparisons. You may be surprised, pleasantly or otherwise.
- Add post-class reinforcement to increase application of what has been learned. Recent transfer technologies make doing this less time-consuming and easier for learners. For an overview of available technologies, see my Training Transfer Technologies white paper. My books Making Learning Stick and Making Elearning Stick provide evidence-based, easy-to-use techniques for both before and after learning events to increase application of learning.
While most learning evaluations don’t capture media attention the way Trump University has, the recent publicity provides a helpful reminder that our end goal in workplace learning is not for participants to like the training but for them to use it, to make the training stick.
Until Next Time…