Audits fail. Let's start by agreeing on that basic truth. No matter how much we might want to think we are perfect/that our work is impeccable/that our findings are accurate/that our opinions are flawless, we make mistakes and, in some instances, the results of the audit – the findings, the report, the opinion, the message that is delivered – do not reflect what actually exists. As with so many things in life, the first step is admitting you have a problem. The next step in this case is to figure out why we failed.
By the way, if you are not doing a postmortem on your audits – failures, successes, or something in between – you are losing out on a fantastic learning opportunity. And bring the whole team into those meetings. If the audit succeeded, the team will learn how they can all achieve that success; if it failed, they will learn what not to do; and if the degree of success lies somewhere in between, they will get to see the good, the bad, and the ugly, and learn from that.
So, as we bring this series of posts to a close, what does a post mortem completed many years later teach us? (What are we doing a post mortem on? Why is it many years later? What have we discussed so far? If you need to catch up, start with this post and work your way through.)
I've said it before; I'll say it again – we have to be willing to ask the questions of ourselves that we ask of our auditees. Monday I suggested that one reason so many projects and initiative fail is not from a lack of control over the details; rather, it is because those involved lose sight of the overall vision/objective/purpose of the project. An analogy was drawn between this failure and the discussion by conductor Benjamin Zander in which he showed the differences between the playing of a new student and those with more experience. That difference was a transition from being someone who plays notes, measures, and phrases (the details) into someone who plays (and, maybe more importantly, interprets) the entire piece. Many projects and initiatives fail because they are focusing on the notes, measures, and phrases rather than the entire piece.
If auditees' projects fail because they do not remember to keep an eye on the entire picture, do our audits fail for a similar reason?
In the "fails" I previously cited, the auditors had looked at the pieces and said all was good. In particular, we tended to look at one section of what was being audited and claimed the entire thing was a success. That is like saying "I didn't listen to the whole concerto, but the notes I did hear were correct, so the whole thing must have been great."
I have seen examples of this time and time again in my own career and examples others have shared. We focus on the notes without thinking about the entire project. Examples:
- After completing flowcharts of a particularly involved process, I asked the auditor what she had learned. She stared back and said that she had completed the flowchart. I followed up by asking if she had seen any issues or breakdowns. She stared and basically repeated that she had completed the flowchart. Flowcharting for flowcharting's sake just to complete the task
- A client once described a discussion with the external auditor in which they requested the client to complete a test. The client explained why the test was superfluous. The external auditor agreed and then explained that the test would have to be completed anyway because it was in the audit program. Testing for testing's sake without regard for the overall needs.
- In an audit related to specific compliance issues, the auditee advised the auditor that there was no back up for critical computer systems. When the issue was discussed with audit management it was decided this would not be included in audit work as it was outside the scope. Performing an audit for the sake of performing an audit rather than identifying and acting on significant risks.
Just three tales I have heard/experienced; I'm sure you have your own.
And yet, if a postmortem had been done on the audits in the above examples, would it have recognized that the problem was not in the individual internal audit processes, but in the fact that the audit/auditor/audit management lost sight of the overall purpose of what audit should be trying to achieve? Did it forget that the purpose of audit (see Monday's post) is to "make things better"?
Don't get me wrong; I'm not saying that every audit has to look at every part of everything. That is one of the things we mean by risk-based auditing. However, by keeping the broader perspective in mind – by not focusing on the parts of the audit, by not just looking at the notes and the measures and the phrases – then there is a better chance the audit will actually speak to what is important to our customers.
My apologies for how long this has all gone on. But as I said in the beginning, I think this is a big deal. I believe that when we do postmortems on our internal audits – when we are looking for where they went right and where they went wrong – we are not looking big enough. I think we are looking for solutions within the notes. And I believe the solution lies in looking beyond the notes, beyond the measures, beyond the phrases. I believe the solution lies in ensuring that we have looked at the big picture – the big picture of what internal audit is trying to achieve and what the organization is trying to achieve.
I'll end this with an interesting line from Lynda Barry. "Keep in mind as you read these words that you are paying no attention at all to the letters of the alphabet."
Quit focusing on the letters of the alphabet; quit working too intently on the individual notes; quit testing because you have to test, interviewing because you have to interview, writing because you have to write. Perform (and note the choice of that word) the full audit.