Friday 30 May 2014

Life of a lesson #9: we're done when I say we're done, got it?


As part of a wider discussion about knowledge management (KM), we’ve recently been looking at the following ten steps in the life of a lesson:

1.       Event takes place – an experience, idea, incident or accident
2.       Analysis and capture – through interview, AAR, workshop, report-writing etc.
3.       Packaging – write-up of lessons
4.       Review for accuracy – editing and improvement by person who identified the lesson
5.       Validation – quality check, ownership assigned and upload into a management system
6.       Review for accountability – periodic checks on progress
7.       Implement recommendations – to avoid/ensure recurrence of bad/good alike
8.       Review for effectiveness – ensure that changes have taken place and/or had desired effect
9.       Closure – lesson status updated but retained in system for reference and to aid analysis
10.    Assurance – as part of risk management, periodic review to ensure closed status remains justified
 

Last time we looked at how an organisation assures itself that a lesson’s recommendations have been implemented; we’ll now look at lesson closure.

As we have already seen, robust lessons management systems have checks and balances such as periodic reviews that monitor lesson progress and hold lesson owners and other stakeholders to account.  It is during such meetings that lessons recommended for closure are reviewed and, where sufficient evidence justifies the decision, closed.

Early on in its efforts to identify and manage lessons, the British Army decided upon two closure statuses:
  • Lessons were closed ‘green’ when recommendations had been implemented and no further action was required;
  • Lessons were closed ‘black’ when the lesson was deemed out of date, or that other initiatives had addressed the issue from which the lesson was drawn, or that the recommendations now lacked official endorsement, or that there were neither the resources nor will to implement them.  A recent development in this area has been the continuous management of ‘black’ lessons as risks.
Once lessons have been closed, some organisations ‘remove’ them from their databases or systems but I recommend their retention for 3 reasons:
  • Research - when faced with future problems, it can be useful to be able to consult historic lessons (including the commentary and audit trail) to understand how similar issues were tackled in the past;
  • Analysis – trends, themes and ‘common issues’ can be detected through data-mining, taxonomy analysis and keyword searches;
  • Assurance – any robust system of lessons management should include periodic review of closed lessons to ensure that the decision to close remains valid (we will examine this in greater detail next time).
For more information on lessons, lessons management systems, knowledge management (KM) and organisational learning, please visit the Knoco website.

Tuesday 13 May 2014

Life of a lesson #8: did we do what we said we would? You sure?

As part of a wider discussion about knowledge management (KM), we’ve recently been looking at the following ten steps in the life of a lesson:


1.       Event takes place – an experience, idea, incident or accident

2.       Analysis and capture – through interview, AAR, workshop, report-writing etc.

3.       Packaging – write-up of lessons

4.       Review for accuracy – editing and improvement by person who identified the lesson

5.       Validation – quality check, ownership assigned and upload into a management system

6.       Review for accountability – periodic checks on progress

7.       Implement recommendations – to avoid/ensure recurrence of bad/good alike

8.       Review for effectiveness – ensure that changes have taken place and/or had desired effect

9.       Closure – lesson status updated but retained in system for reference and to aid analysis

10.    Assurance – as part of risk management, periodic review to ensure closed status remains justified


Last time we looked at the implementation of the recommendations in each lesson; we’ll now look at the ways in which an organisation assures itself that a lesson’s recommendations have been implemented.


A robust lessons management system relies on checks and balances.  It is not enough for us to do the work required to ‘learn’ the lesson; we have to be able to show to others that we have done it.


This means adding reference material to the lesson, such as revised templates, procedures, or policy documents.  We might add certificates, reports, photographs or links to websites wherein further evidence is contained.


Such evidence, along with an audit trail commentary from the lesson owner, showing how the lesson was learned, is then reviewed by the person or body authorised to close lessons, usually at one of the periodic progress meetings we looked at some weeks ago.


A cautionary tale….

Some years ago, I worked as one of a team of analysts in the British Army’s Lessons Exploitation Centre (LXC), helping the Army to learn from the operations in Iraq, Afghanistan and elsewhere.  One of our responsibilities was to help manage lessons to their conclusion – i.e. ensuring that actions took place to demonstrate that the Army had learned from the earlier deployments during which such lessons had been identified.


I recall one lesson in particular, which had identified that two pieces of personal equipment were mutually incompatible and that the older of the two should be re-designed.  Procuring new equipment can take time and it is therefore understandable that, when an updated item is finally manufactured, people respond with enthusiasm, perhaps even excitement.


On this occasion, once it was confirmed that the new items were finished and en route to the troops in Afghanistan, the relevant lesson was recommended for closure.  On the surface, it appeared that the lesson’s recommended action had been taken and, since there was plenty of documentary evidence to show that the new equipment had now been procured and was being distributed, the lesson was closed.


A few months later, an LXC colleague deployed to Afghanistan for a 3 month tour.  Before departure, she was sent for training and kit-issue, receiving all the mandatory ‘Afghan-specific’ equipment that she would need during her tour.  She made a specific point of calling us before she flew, to tell us that she had been issued with the old piece of equipment and not the updated version.


This prompted us to re-open the lesson and we kept it open until we had received confirmation from the deployed headquarters in Afghanistan that all personnel had been issued with and were using the new item.

 
From this small incident we learned:
  • It is not enough to be told that a change has been made; you need to confirm that the change has been made and is having the desired effect.
  • Changes in themselves are not enough – it is the effect intended by such changes that is sought and which should determine whether or not a lesson can be closed.
  • Lesson recommendations need to be written with this in mind.
Furthermore, from this point onwards, we increased the threshold at which we judged lessons to have been ‘learned’ and therefore ready for closure.



I am not arguing that the effect of every change needs to be observed; just that judgement should be used when considering the evidence.  Given that the approach for managing lessons can (and should) be closely aligned with that for risk, I think it makes good sense to err on the side of caution.  This means that we hold out for a higher burden of proof for certain lessons, such as those relating to safety or significant financial impact, either positive or negative.


We’re near the end now; next time, we’ll look at closing lessons and what happens next….


For more information on lessons, lessons management systems, knowledge management (KM) and organisational learning, please visit the Knoco website.