Are your participants guilty of the illusion of learning?
I came across an interesting concept as I was reviewing recent studies in neuroscience related to Making Training Stick®. It’s called the Illusion of Knowing and refers to people’s errors in perception. For example, we seem to be hearing more in the news about individuals who have been convicted and incarcerated for crimes that later DNA testing proves they didn’t commit. Eye witness accounts have identified the individual as the perpetrator only to be proven wrong years later. These memory distortions arise out of our discomfort for ambiguity and our desire to “have the right answer”.
So in a level 1 evaluation when participants are asked “what did you learn?” their responses may well be shaped more by this illusion of what they would like to have learned and what they know they should have learned, than by what they have actually learned.
Another illusion is called Imagination Inflation, which is the tendency of people who when asked to imagine an event, will sometimes begin to believe, when asked about it later, that the event actually occurred. For example, if a level 3 post-training survey asks a participant how they are applying their learning, they may believe that they have applied it or are applying it when they actually have not or are not. This is particularly troublesome with complex learning of tasks/behaviors where application is less than straightforward such as soft skills: customer service, management, communication.
How to overcome these illusions of learning? Feedback. Studies show that when students have an opportunity to reflect on their demonstration of learning and on their performance, their perceptions of their learning and performance become more accurate. This is called metacognition, the awareness and understanding of one’s own thought processes and learning. If you are one of the many individuals who, based on the notion that adults do not want to be tested, have shyed away from testing (as I have), I urge you to reconsider. Testing and then providing participants with correct answers is one way to provide feedback and reduce their Illusions of Knowing regarding what they have learned. This of course also serves as level 2 evaluation of learning.
Another feedback tool is the Behavior Observation Scale (BOS). This can reduce participants’ possible illusions about what they are applying or have applied. Develop a set of behaviors that demonstrate successful application of the skill(s) during or prior to the design phase. This can be done by the designer and/or other interested stakeholders in the training. Provide a Likert 5-choice scale for each behavior. This BOS should then be used by the participant and/or their manager to assess their on-the-job application. This of course may double as a level 3 evaluation.
- Review your end-of-training level 1 evaluation form and eliminate questions that ask participants what they learned. Consider instead asking a question or two about how they intend to apply their learning. (Research studies have found a strong relationship between reports of intention to apply and actual application.)
- Consider adding testing to all training. If you currently use tests, incorporate opportunities for participants to review their answers vis a vis the correct answers.
- Develop Behavior Observation Scales for complex learning. Distribute these to participants and their managers post-training, when they are most likely to have had an opportunity to apply their learning.
- Consider ways to “motivate” them to respond. (My favorite is to withhold credit for the class until post-training feedback has been received.)
Until next time…