Life of a lesson #8: did we do what we said we would? You sure?

As part of a wider discussion about knowledge management (KM), we’ve recently been looking at the following ten steps in the life of a lesson:

1.       Event takes place – an experience, idea, incident or accident

2.       Analysis and capture – through interview, AAR, workshop, report-writing etc.

3.       Packaging – write-up of lessons

4.       Review for accuracy – editing and improvement by person who identified the lesson

5.       Validation – quality check, ownership assigned and upload into a management system

6.       Review for accountability – periodic checks on progress

7.       Implement recommendations – to avoid/ensure recurrence of bad/good alike

8.       Review for effectiveness – ensure that changes have taken place and/or had desired effect

9.       Closure – lesson status updated but retained in system for reference and to aid analysis

10.    Assurance – as part of risk management, periodic review to ensure closed status remains justified

Last time we looked at the implementation of the recommendations in each lesson; we’ll now look at the ways in which an organisation assures itself that a lesson’s recommendations have been implemented.


A robust lessons management system relies on checks and balances.  It is not enough for us to do the work required to ‘learn’ the lesson; we have to be able to show to others that we have done it.


This means adding reference material to the lesson, such as revised templates, procedures, or policy documents.  We might add certificates, reports, photographs or links to websites wherein further evidence is contained.


Such evidence, along with an audit trail commentary from the lesson owner, showing how the lesson was learned, is then reviewed by the person or body authorised to close lessons, usually at one of the periodic progress meetings we looked at some weeks ago.


A cautionary tale….

Some years ago, I worked as one of a team of analysts in the British Army’s Lessons Exploitation Centre (LXC), helping the Army to learn from the operations in Iraq, Afghanistan and elsewhere.  One of our responsibilities was to help manage lessons to their conclusion – i.e. ensuring that actions took place to demonstrate that the Army had learned from the earlier deployments during which such lessons had been identified.


I recall one lesson in particular, which had identified that two pieces of personal equipment were mutually incompatible and that the older of the two should be re-designed.  Procuring new equipment can take time and it is therefore understandable that, when an updated item is finally manufactured, people respond with enthusiasm, perhaps even excitement.


On this occasion, once it was confirmed that the new items were finished and en route to the troops in Afghanistan, the relevant lesson was recommended for closure.  On the surface, it appeared that the lesson’s recommended action had been taken and, since there was plenty of documentary evidence to show that the new equipment had now been procured and was being distributed, the lesson was closed.


A few months later, an LXC colleague deployed to Afghanistan for a 3 month tour.  Before departure, she was sent for training and kit-issue, receiving all the mandatory ‘Afghan-specific’ equipment that she would need during her tour.  She made a specific point of calling us before she flew, to tell us that she had been issued with the old piece of equipment and not the updated version.


This prompted us to re-open the lesson and we kept it open until we had received confirmation from the deployed headquarters in Afghanistan that all personnel had been issued with and were using the new item.

 
From this small incident we learned:

• It is not enough to be told that a change has been made; you need to confirm that the change has been made and is having the desired effect.
• Changes in themselves are not enough – it is the effect intended by such changes that is sought and which should determine whether or not a lesson can be closed.
• Lesson recommendations need to be written with this in mind.

Furthermore, from this point onwards, we increased the threshold at which we judged lessons to have been ‘learned’ and therefore ready for closure.



I am not arguing that the effect of every change needs to be observed; just that judgement should be used when considering the evidence.  Given that the approach for managing lessons can (and should) be closely aligned with that for risk, I think it makes good sense to err on the side of caution.  This means that we hold out for a higher burden of proof for certain lessons, such as those relating to safety or significant financial impact, either positive or negative.


We’re near the end now; next time, we’ll look at closing lessons and what happens next….


For more information on lessons, lessons management systems, knowledge management (KM) and organisational learning, please visit the Knoco website.