Thank You!

You are attempting to access subscriber-restricted content.

Are You Ready to Experience Everything Internal Auditor (Ia) Has to Offer?

​Peak Performance​

The Denver Auditor's Office uses performance audits to further accountability and provide recommendations for improvement.

Comments Views

​​The Great Recession will be remembered in the United States for all manner of dramatic events: the collapse of financial behemoths, government bailouts in the banking and auto industries, unprecedented home foreclosure rates, and sweeping legislation to stimulate economic recovery. Lack of accountability is often cited as an underlying factor contributing to the financial crisis. Not surprisingly​​, demand for greater accountability has been growing—public and private institutions alike are being held to higher standards of integrity, ethics, and transparency, and the internal audit function plays an integral role in meeting those expectations.​​

In the public sector, government audit functions deliver on the demand for greater accountability directly and proactively. One of the methods they use to safeguard taxpayer dollars is performance auditing, which assesses whether government organizations are using available resources efficiently and effectively and includes recommendations for improvement (see "Performance Auditing as an Accountability Tool" at right). The examination is objective and systematic, using the Generally Accepted Government Auditing Standards, also known as the Yellow Book, as a guide for methodology.​

At the City and County of Denver Auditor's Office, headed by an independently elected official who provides a check and balance to Denver's strong-mayor form of government, we use performance audits to increase accountability and measure the success of government operations. Our office has put several elements in place to help ensure the effectiveness of performance audits and applied the approach with considerable success. The effort provides lessons for other government auditors and a potential road map for establishing their own performance audit program.​​

Performance Auditing as an Accountability Tool 

Lennis Knighton is widely considered the inventor of modern performance auditing. In his 1967 book, The Performance Post Audit in State Government, Knighton defined performance auditing as "an independent examination … for the purpose of providing … an evaluation and report of the manner in which [government] administrators … have discharged their responsibilities to faithfully, efficiently, and effectively administer the programs of the state." For U.S. government audit shops that undertake performance auditing today, many look to the federal Government Accountability Office (GAO) for guidance. The GAO has established professional standards, commonly referred to as the Yellow Book, to provide a framework for government auditors to conduct their work with competence, independence, integrity, and objectivity. The Yellow Book defines performance audits as:

​Audits that provide findings or conclusions based on an evaluation of sufficient, appropriate evidence against criteria. Performance audits provide objective analysis to assist management and those charged with governance and oversight in using the information to improve program performance and operations, reduce costs, facilitate decision-making by parties with responsibility to oversee or initiate corrective action, and contribute to public accountability.

Both Knighton's early work and the more recent GAO guidance capture the spirit of performance auditing as an accountability tool that uses the traditional audit framework enhanced by principles from the social sciences. Rather than merely identifying where performance deviates from established criteria — such as policies and procedures, contract terms, and legal requirements — performance audits compare stated goals to actual outcomes and make recommendations for improvement. The Yellow Book guides auditors to recommend actions that will correct problems identified by the audit with the goal of ultimately improving programs, operations, and performance. Such prescriptions take accountability one step further, from simply determining compliance with established criteria to assessing the efficacy of the criteria themselves.​

The Right Approach 

Performance auditing can provide significant benefits to government organizations — but it is certainly not a panacea for accountability. Some audit functions may struggle for success, depending on the extent to which they are faced with common challenges. These challenges include achieving a significant degree of independence, ensuring that audits are timely and relevant, and navigating the political landscape, all of which the Denver Auditor's Office has managed to handle successfully.​


Audit independence is a cornerstone of the profession, but the degree of independence granted to government performance audit functions can vary widely. Many performance audit shops are not empowered to determine independently what they will audit. Few have the authority to access every conceivable record and type of audit evidence necessary to carry out their mission, and many cannot command formal responses to their findings and recommendations. Without sufficient independence, the function can be impaired and opportunities for impact diminish.

At the City and County of Denver, a change to the city's charter in 2008 through a vote by the citizenry established our office as one of the most independent government audit functions in the country. Denver has an elected auditor, broad audit authority, and an independent audit committee. Adherence to the Yellow Book ensures a baseline level of independence, but our auditors also have unlimited access to city records and the legal authority to require audited entities to respond formally to audit recommendations. Establishing our own audit plan further strengthens our independence. These elements increase the likelihood that Denver's audits will have a tangible and lasting impact on the city's operations.​

Timeliness and Relevance 

Potential for impact can be reduced when audits are not timely or relevant. Many government audit shops, when determining what to audit, generate an audit universe — a tool used widely in traditional auditing. However, in government performance auditing, cataloging and ranking all auditable areas can prove complicated and time consuming. Moreover, the audit universe as a planning tool does not account for the ever-changing nature of public policy and public opinion.

Rather than adhering to an approach focused on identifying and ranking an audit universe, Denver uses a realistic "audit horizon" strategy to identify, prioritize, and manage audits deemed critical to city operations (see "Denver's Audit Horizon Approach" below). The horizon approach uses a risk assessment methodology that explicitly acknowledges the inherently changing nature of risk over time as well as the limited resources available to execute the audit plan. To stay on top of emerging public policy issues, the annual audit plan is structured to be flexible and responsive, ensuring that the office's audits are timely and relevant.​

Navigating the Political Landscape 

Even when in sync, performance auditing can be viewed as political, questioning policies and activities that often evoke strong opinions and threaten vested interests. Programs that are popular with elected officials and the public do not always withstand audit scrutiny. Pointing out program shortcomings and unmet expectations can put auditors in the middle of the political cross fire — an uncomfortable place for a necessarily independent function.

Many of Denver's audits have the potential to become intensely political. Audit findings that question the basis on which a program is founded can invite more than mere disagreement. Accordingly, we must take care to preserve our reputation as an oversight body, rather than as a political player. To maintain this important distinction, Denver has implemented a communications and customer relations strategy. It entails frequent communications and meetings with city elected officials and executive management to educate them about the audit process and to address emerging issues and concerns. We also include audited entity responses in final audit reports and provide a suite of advisory services. By directly and proactively communicating our role and objectivity, the audit function can remain solidly in its appropriate analytical corner and stand firmly behind its evidence-based recommendations.

Our communications strategy extends beyond elected officials and city managers to our ultimate client: the citizenry. We strive to write all of our audit reports in plain language, providing thorough explanation of any technical issues. The background section of our reports serves as an educational component, providing valuable context to the reader who may not be familiar with the subject matter.​

Principles in Action 

What do these peak performance audit principles look like in the Denver audit arena? In December 2011, the Denver Auditor's Office released an audit of the city's photo radar and photo red light program. Although it is just one of the many audits we released in 2011, it demonstrates how we are using performance auditing to raise tough accountability questions that are directly relevant to the citizenry.

Colorado state law allows municipalities to use cameras to enforce compliance with speed limits and red lights. Through the audit, we sought to determine whether the photo radar and photo red light program was being conducted in accordance with the law, safety elements had been evaluated effectively, and revenues could be increased or processes improved.

Denver's Audit Horizon Approach  

In theory, an audit universe represents all auditable areas within an organization. An audit universe risk assessment helps prioritize audit work by using various criteria to develop a quantified rating for each of those areas.

While appropriate for financially focused audits, the approach is less effective for prioritizing audits of government performance. For example, it is difficult to quantify the risks related to vulnerable populations served by governmental social service functions, yet the exposure to the city of problems occurring in these areas is significant. Moreover, generating a comprehensive universe can prove complicated and time consuming, revealing literally thousands of possible audits that would require tens of thousands of audit hours to complete.

Recognizing the limitations of the audit universe approach to audit planning, in 2009 the Denver Auditor's Office began using a realistic "audit horizon" strategy to identify, prioritize, and execute audits deemed critical to city operations. Rather than identifying everything auditable, we use a risk-based methodology to identify and prioritize a select number of audits for inclusion in the annual audit plan. The horizon itself extends for three years, identifying planned audits for the current year and potential audits for the ensuing two years. The audits included in the horizon are selected based on available audit hours and staff resources to ensure that we are establishing realistic expectations. The horizon also includes contingency hours to account for audit requests that will inevitably be made by the mayor, city council, and department heads. This type of flexibility allows the Denver auditor to address emerging issues timely, providing high-quality and responsive customer service to elected officials and operational management.​

The safety component was of particular importance to the audit because the program has been billed as a public safety effort. The public policy was premised on the idea that strategically placed photo radar vans would reduce speeds and thus reduce accidents and pedestrian injuries associated with improper vehicle speeds. Similarly, city public safety officials proffered that red light enforcement would reduce accident rates at intersections.

Our audit yielded 14 distinct recommendations. The findings that led to those recommendations were numerous and complex, though at a high level the audit revealed much about the safety and revenues associated with the program. First, we found that the city could not prove the program has had a positive impact on safety. In fact, the program is not capturing the right data to determine whether there has been an independent, positive effect on accident rates. Second, we found the program was generating a significant amount of revenue — well over what was necessary to cover its costs. During a budget shortfall, the revenues have certainly been a welcome addition to the city's general fund. However, for a program billed as a safeguard for drivers and pedestrians on Denver's rights-of-way, the program was not living up to its promise. It was instead widely viewed as a cash grab, undermining the public trust.

Tackling an audit that raises objective questions about the effectiveness of highly visible public policies demonstrates why performance auditing can be so risky — our work was shedding light on a program that has been popular with elected officials. An audit function without our degree of independence may never have been granted the opportunity to evaluate such a program. Even with the authority to establish our own audit plan, an audit like this one may not have come out of a traditional risk assessment were it not for our use of the audit horizon methodology, which includes risk assessment criteria focused on emerging or sensitive public policy areas, regardless of the size or cost of a program or service.

In the mandatory response from the audited agency, we only received partial agreement with our recommendations. Some disagreement with an audit like this is expected, considering its contentious and political nature. Even though some of our recommendations will not be implemented, the process served one of the fundamental purposes of the performance audit function: to raise difficult questions about the extent to which government upholds the public trust. By releasing a clearly communicated, compelling, and educational report, we generated a lot of media attention as well as outreach from citizens. In a society hungry for accountability, our audit function served as the catalyst for thoughtful discussion about public policy and the role of government.​

Performing with Persistence 

An audit function that conducts performance audits to reach its potential as a force for accountability should ideally achieve maximum independence, maintain unfettered access to records and personnel, and adhere to professional standards. It should also establish formal reporting requirements, use effective risk assessment practices, audit relevant areas of public policy, and remain apolitical.​

Whether an audit function that is conducting performance audits is close to or far from this ideal, internal auditors require one additional element to answer the call for accountability, day in and day out: persistence. Fundamentally, performance auditing involves changing government — but change does not happen overnight. Oftentimes, change is effected incrementally, beginning with dialogue about the current state of an entity or process and ideas about how it could be improved. With persistence, performance audits can be an early catalyst in a longer process, bringing about the kind of accountability that may only be possible with time.

Kip Memmott, CGAP, CRMA, is director of audit services, and Emily Jacobson is a communications specialist for the City and County of Denver Auditor's Office.​

Internal Auditor is pleased to provide you an opportunity to share your thoughts about the articles posted on this site. Some comments may be reprinted elsewhere, online or offline. We encourage lively, open discussion and only ask that you refrain from personal comments and remarks that are off topic. Internal Auditor reserves the right to remove comments.



Comment on this article

comments powered by Disqus
  • Gleim_Oct2018_Premium 1
  • IIA CERT CIA_Oct2018_PRemium 2
  • IIA CIALS_Oct2018_Premium 3