Properly established change management controls are the foundation of an effective IT control environment. Although considerable guidance exists on the proper ways to audit change management controls, many auditors often have trouble determining whether an organization's IT change management process is effective. This is the result of several challenges facing the internal audit profession. First, much of the audit guidance is in the form of checklists that don't properly identify which control activities are most important. Second, many audit guides predate new research that can help auditors focus on the right areas. Finally, many auditors are not equipped to challenge IT management effectively and do not fully understand their organization's change management issues.
To conduct more effective reviews of change management policies and procedures, auditors could use the "3-C" approach to change management — focusing on the organization's culture, controls, and credibility. The 3-C approach will not only help auditors fine tune their skills, it will also enhance their knowledge of change management fundamentals.
Why Audit Change Management?
Change management audits help protect the business from risk. However, in today's compliance-centric world, it is easy to lose sight of this simple, primary goal. For instance, many organizations implement policies and procedures simply to "pass the audit," without paying much attention to how policies are enforced. Worse still, some auditors base their reviews on static checklists that only allow them to mark the box providing the best answer, rather than conducting audits that accurately review the current IT landscape. The result is the approval of highly ineffective change-management processes and controls.
With this dynamic in mind, the information below will help internal auditors recommend effective and efficient change management best practices based on the 3-C approach. The goal of these practices is to encourage IT departments to operate in a way that protects the business from internal and external threats, while satisfying the organization's compliance requirements.
Beyond Checklists: The Three Cs of Change Management
Auditors can determine the effectiveness of a change management system by testing three primary areas referred to as the three Cs of change management. The framework provided by the three Cs allows auditors to ask open-ended questions and enables them to apply their detective skills when conducting audits of change management processes and activities, even if they feel at a disadvantage due to a lack of technical knowledge. Below is a description of how to audit each primary area.
Understanding an organization's culture toward change management will help auditors obtain background information about the different change management processes and the company's overall attitude toward change. Key aspects of an organization's change management culture that auditors need to look out for include the company's:
Tone at the top. An organization's tone at the top sets the standard for employee behavior and compliance with internal policies and procedures. When reviewing an organization's tone at the top, auditors should pay close attention to clear and consistent communications from senior management that indicate whether or not change management protocols must be followed.
Accountability process. Policies are meaningless unless they are enforced. Therefore, auditors should identify whether the organization has a system in place that enforces policy violations and the consequences for these actions.
Culture of causality. A culture of causality means organizations have an established way of analyzing the impact of IT changes before and after they occur. Such analysis should focus on predicting the impact of changes on risk mitigation. Companies also should conduct a post-incident analysis to learn from failed or improperly managed changes. This allows organizations to assign risk ratings to changes based on historical data and helps to scrutinize high-risk changes more effectively.
Collaboration and communication activities. High-performing organizations have processes that enable them to make changes in plain sight so that stakeholders and constituents have the opportunity to see, anticipate, and provide input on changes before they occur. This allows companies to mitigate risk and prevent "change drive-bys," where one group is surprised or impacted by changes initiated by other groups.
Emphasis on people, processes, and technology. High performing organizations understand that change management is a complex interplay of people, processes, and technology, Consequently, they establish explicit policies to set expectations, provide well-documented processes, and clearly define expected roles and duties. Once policies and processes are in place and roles are clearly defined, technology may provide preventive controls to guide appropriate behavior; detective controls to identify when inappropriate activities occur; and corrective controls to return things to a trustworthy state.
Besides reviewing the elements above, auditors need to identify any potential red flags to determine if the organization's culture and attitudes toward change management might be in jeopardy. For example, some companies pay lip service to change management. When this happens, a change management process is in place, but employees are not held accountable for any violations, often because management doesn't know when violations occur. Another red flag is when the same types of outages occur repeatedly. This may signify that the organization is not learning from its past mistakes or that the organization is relying on individual knowledge to run the business, rather than building an organizational "play book" that allows them to establish repeatable processes and knowledge.
A final red flag is the ownership of multiple software products that perform similar functions. Purchasing different products to do the same work is often a sign that the organization is looking for a "silver bullet" solution to its change management problems. This often occurs because it is easier to justify buying new software than it is to drive a change initiative that alters how people and processes work.
Researchers at the IT Process Institute and Carnegie Mellon University's Software Engineering Institute have found that organizations achieve the highest leverage when implementing controls that center on change, access, and accountability. This allows organizations to focus on how work should be done within the company, who is allowed to do that work, and ensure that changes only happen within the organization's approved processes. By preventing personnel from circumventing change processes, high performers maintain "good IT hygiene" and encourage effective communication about change. This prevents the unexpected negative consequences associated with poorly managed changes, such as unexplained outages and high amounts of unplanned work.
Auditors can benefit from these studies by acquiring the necessary knowledge to help organizations establish controls that help to prevent, detect, and correct vulnerabilities and potential risk areas. For instance, auditors may review the effectiveness of IT controls aimed at preventing uncontrolled or unauthorized changes to the production environment. Below are examples of preventive, detective, and corrective controls:
Pre-production staff cannot access production systems and must submit proposed changes via the change management process (preventive control).
All changes must be reviewed and authorized by the change advisory board prior to implementation (preventive control).
Independently detected changes must be reconciled with work authorizations to ensure they are appropriate, documented, and executed properly (preventive control).
Automated monitoring must be used to record changes to production configurations (detective control).
Exceptions must be removed from the environment or be escalated as a security incident (corrective control).
After organizations implement the necessary preventive, detective, and corrective controls, auditors should ask questions that seek credible and demonstrable answers, such as:
Can IT departments detect all changes to the production environment? If so, how?
Can the organization produce a history of those changes? If so, how?
What are the criteria used to differentiate between authorized and unauthorized changes?
How long does it take to discover an unauthorized change?
What happens when an unauthorized change occurs?
Finally, auditors need to identify potential red flags that may affect established controls. A potential red flag might be the organization's inability to answer questions or provide credible evidence in a timely manner during the audit process. Unsatisfactory answers often hinge on "gut feelings," conjectures, time-consuming manual reviews of system event logs, e-mail-based approvals, and other inefficient methods.
Red flags also might include an inability to identify when changes occur outside the change management process. For instance, an organization might have preventive and corrective controls in place, but lack effective detective controls. Although these companies can report changes that go through the change management process, they are unaware of changes taking place outside this process until a problem occurs. Another red flag is spending a high percentage of resources performing unplanned work, such as fixing failed changes and responding to system outages.
Credibility is where it all comes together. In this context, credibility deals with the quality of the answers auditors receive when they test for effective control processes. This is the area where high-performing organizations shine: High performers understand their main goals are to protect the business from existing and unforeseen risks and operate in a manner that expedites the compliance process. In addition, organizations with high credibility have an established change management culture that sets clear organizational expectations on how corporate work will be conducted. They also have effective controls in place that identify how data will be collected and used. Finally, high performing companies have a system of checks and balances that helps people stay true to the change process and demonstrates there are clear consequences for noncompliance with internal regulations.
When testing for credibility, auditors need to ask questions that would be difficult for organizations to answer without established effective controls and a change management culture. During quality reviews, auditors may ask organizations questions concerning the following:
Copies of change management compliance reports showing total changes for a given period of time with breakouts of authorized versus unauthorized changes.
Statistics about failed changes and their causes.
A list showing all unplanned outages and their causes.
Exception reports showing changes made outside the official change management process.
A common indicator of a credibility concern is the presence of a high volume of unauthorized changes, often relabeled as "emergency changes," which indicate that employees are not following established change authorization and review procedures and are finding loopholes to circumvent the approval process. High numbers of system availability issues (e.g., unexplained outages or security incidents) and increasing amounts of unplanned work also could alert auditors of the possibility of a faulty change management process. These symptoms are often the result of poorly coordinated change management practices and inadequate pre-production testing.
Another credibility concern indicator is the presence of unplanned work rates that exceed 20 percent to 25 percent. This typically indicates the organization is facing a culture or control problem. Late projects, cost overruns incurred from hiring external personnel to fix IT issues, and employee turnover also are indicators of systemic change management problems. Two final red flags include an IT department's inability to produce verifiable evidence that substantiates the presence of controls, as well as overly long response times when validating a control's presence or effectiveness. These indicate a lack of systematic tracking, documentation, and reconciliation of change authorizations, as well as a lack of post-incident scrutiny when service-interrupting incidents occur.
Moving beyond checklists by asking open-ended questions during an audit and probing for credible details is an effective way for internal auditors to avoid being "hoodwinked" during the review process. Applying deductive powers enables auditors to get to the audit's core purpose quickly. This includes verifying that managers are taking responsibility for their control environment and are behaving in a manner that protects the business from threats. To this end, auditors need to anchor their change management audits based on the company's culture, controls, and credibility, and coach their IT teams on how to implement control strategies that satisfy all three Cs.
During the past five years, there has been significant progress establishing a causal relationship between key IT controls and IT effectiveness. This research was started by the IT Process Institute (ITPI), a nonprofit organization that promotes the use of best-known IT practices. Other organizations have joined ITPI, including Carnegie Mellon's Software Engineering Institute, ISACA, and The Institute of Internal Auditors (IIA). Detailed information about ITPI's activities, including its IT Controls Benchmarking Survey and practices of high performing IT organizations, can be found on its website, www.itpi.org.
In addition, auditors and IT professionals should consult The IIA's Global Technology Audit Guide on change and patch management, Change and Patch Management Controls: Critical for Organizational Success. The guide can be downloaded free of charge from The IIA website, www.theiia.org/guidance/technology/gtag2/.