More and more organizations have been turning to the Agile methodology for their software development efforts. According to PricewaterhouseCoopers’ Global Portfolio and Programme Management Survey 2014, use of Agile has increased by 11 percent since 2012. At the same time, many internal audit functions are struggling with how to interact with Agile projects, especially those whose experience lies with more traditional, system development life-cycle (SDLC) controls.
Agile processes help project teams manage unpredictability through a focus on adaptive planning and rapid, flexible response to change. The Agile philosophy encompasses several iterative software delivery methodologies — including scrum, extreme programming, and feature-driven development — that emphasize a lean, interactive approach to product development. In fact, Agile is not confined to a single method of delivery — most organizations take a hybrid approach, drawing from multiple iterative development methodologies. The products to which Agile is applied typically emphasize making usable code available quickly to meet business needs.
Agile project management focuses on perceived value-add processes. The values that underpin this approach, as defined by the Agile Manifesto, specify that: a) individuals and interactions are more valuable than processes and tools, b) working software is a higher priority than comprehensive documentation, c) customer collaboration is more important than contract negotiation, and d) responding to change is preferable to following a rigid plan.
Auditors familiar with traditional SDLC controls will likely recognize that some of the Agile values conflict with more established methodology. The traditional controls are typically implemented “after the fact,” and they rely heavily on documentation — neither of which works well with Agile methodologies. To help close the gap between their knowledge of traditional models and the Agile method, internal auditors should consider five steps aimed at enhancing work with Agile teams. Following this approach, practitioners can help the team, and the organization, execute its compliance responsibilities effectively while making sure not to erode the value of Agile methodologies.
1. Get Involved Early, Understand the Processes
The earlier internal audit gets involved, the better. Working with Agile teams in the early stages of project development increases understanding of the project’s life cycle and its key benefits, drivers, and objectives. That understanding, in turn, enables internal audit to better contribute to the project as the team defines its risk management approach and strategy.
Before internal audit can begin scoping an Agile project, it has to understand the processes. Auditors should spend time with the process owners and ask them to explain their version of Agile. Although scrum is the most commonly used approach, auditors should never assume that scrum, or any other method, has been selected.
Numerous Agile variants exist, and some organizations even develop their own in-house methodology based on Agile’s core values. Several variants, in particular, are commonly encountered:
Scrum is often used interchangeably with Agile and focuses on the project management of the product or SDLC. The methodology emphasizes collaboration, functioning software, team self-management, and the flexibility to adapt to emerging business realities. Scrum is highly collaborative, often benefiting from cohabitation of resources.
Extreme programming (XP) is an Agile variant that focuses on the software engineering component of SDLC. The approach is best suited to small, focused teams and promotes simplicity of code. It features frequent releases in short development cycles, coding in pairs, and unit testing of all code.
Lean development is a variant common to scrum that focuses on SDLC project management. Lean development’s roots are grounded in Lean manufacturing theories — the methodology consists of start-up, steady-state, and transition or renewal project phases.
Crystal methods are a collection of various Agile-like methodologies focused on streamlined, optimized, integrated teams, with a specific method applied to each project depending on communication requirements, system criticality, and project priority.
Other variants of Agile include hybrids such as feature-driven development, test-driven development, Waterfall-Agile, the dynamic system development model, and the Agile unified process. Internal auditors should make sure they understand the project methodology’s objectives, process controls, and documentation and process requirements before a risk management approach and strategy are defined.
2. Assess Risk and Control
Once the chosen methodology is understood, internal auditors should map out process control points — even if the project team doesn’t necessarily view them as controls. Two control points from the scrum methodology provide illustrative examples:
Product backlogs comprise the store of all user requirements in the form of stories that communicate what the end user should be able to do, and the benefits accruing from those features. A backlog, and variations of it, exists for every Agile project and should be available to everyone involved. Documentation such as test cases and results, as well as specifications, vary from team to team. If a product backlog can’t be produced, the auditor should inquire about it with members of the Agile team.
Burn-up/burn-down charts are the primary tool many teams use for tracking their progress. They measure the total in-scope work, the amount of work that should have been completed by a particular time, and the work actually completed. In effect, the charts take the place of several traditional project controls and could be viewed as a type of earned value analysis. Such charts reveal where project efforts are focused, and where they should be focused, as well as help identify significant changes in scope.
Once internal auditors develop an understanding of the inbuilt controls, they should examine the project’s inherent risk profile. While Agile development can provide significant benefits to a project — such as more frequent releases of code and better alignment between users’ needs and the finished product — it also introduces risks that need to be considered and managed correctly.
The traditional roles of business users, developers, testers, and IT experts have become more cross-functional and integrated to support leaner project teams and continuous delivery. Consequently, some of the traditional control gates may not exist as expected on Agile projects, particularly with regard to segregation of duties. That’s especially true in organizations that have adopted a development operations (DevOps) strategy. DevOps sees operations and development engineers working together throughout the life cycle, from design to production support.
Auditors need to understand the project team’s approach to segregation of duties and code production, and examine controls within that approach. Agile processes should result in an increase in automation, including testing and approval, as opposed to traditional manual sign-offs. Internal audit must become familiar with those tools and processes as well as know how to interpret the outputs of automated systems and logs.
Auditors should also be mindful of the risk that Agile project iterations could become delayed by traditional functions such as change and release management. They should assess the project team’s ability to integrate with those functions, and raise issues related to interactions with them — including the functions’ ability to support rapid-delivery models.
Documentation issues may also present a risk. While Agile-delivery methodologies by their nature seek to generate less documentation than traditionally required, that doesn’t mean documentation should not exist. Auditors should work with the team to find the minimum documentation standard acceptable and determine whether the product backlog, or an extension of it, achieves the required level of comfort while still promoting Agile principles.
Lastly, one of Agile’s biggest benefits — its short turnaround cycles — also represents one of its inherent risks. The discipline’s iterative nature can make it difficult to realize the promised business value, if the effort’s scope is continually evolving. Agile teams need to put a mechanism in place that isolates the effort while still capturing future functionality in the product backlog. That functionality should then be turned into a separate effort that can be controlled independently.
3. Know How Agile Teams Define Done
One of scrum’s primary tenets states that teams following the methodology are self-organizing and self-directed, meaning that individual teams largely identify and implement their own standard practices and quality control metrics. And because quality measures can vary from one team to the next, differing notions of what constitutes project completion may exist. Examples of the Agile team’s methods for defining when a project is “done” include:
A code/configuration review process. The team may require many levels of solution-level reviews to confirm adherence to design or development standards, to promote optimized and sophisticated error logging and error handling, or to meet other required solution needs.
Traceability. Many project teams apply the contents of the Agile Manifesto to promote a document-free process versus a documentation-driven process. However, a well-practiced Agile team can, for example, provide traceability that links working product features to requirements (user stories taken from an approved product backlog), design documentation, test evidence, and release strategy and documentation.
Understanding the team’s definition of done leads to an entry point for a risk-based conversation about the effective use of Agile to deliver business value. The definition serves as a quality control mechanism, though it also acts to promote adherence to practices aimed at reducing risks associated with Agile development.
4. Assemble the Right Skills
Agile-based projects feature unique risks and control structures, and understanding them is crucial to the review process. Audit teams need to align the right expertise with planning and review activities, enabling practitioners to:
- Ensure a sound understanding of the problems and risks.
- Establish credibility and confidence in the program team.
- Build empathy with the delivery team.
- Deliver practical, meaningful insight to the project team.
Subject matter specialists with experience in both delivering and reviewing similar projects are also key to successful reviews. Specifically, auditors reviewing Agile projects should have more than a basic understanding of Agile processes, familiarity with the toolset being used, an understanding of how to extract and interpret the required information, and a grasp of the path to production that is being used by the project teams.
Once the review team is in place, auditors should make sure their approach focuses on delivering value. In particular, they need to understand what the project team is trying to achieve and link audit activities to those aims. To achieve alignment, practitioners should consider an objectives-based audit program. Rather than reviewing compliance against a particular risk and issues template, for example, the team should determine whether the overall objective of “managing risks and issues effectively” has been met. Auditors may want to consider using an assessment framework that goes beyond control outcome.
Practitioners need to provide relevant, actionable, and timely feedback that will enhance the likelihood of project success. Moreover, reviews should not be limited to solution and delivery risk — practitioners may want to consider external and commercial risk and examine any corresponding mitigation strategies. These factors contribute to the likelihood of project success and may be critical to a meaningful review. Auditors should familiarize themselves with not only the expected controls outcomes of the project, but also the required technical and business outcomes, allowing for a more rounded view to be developed.
5. Establish Reporting Parameters and Provide Real-time Feedback
To deliver maximum value to the project team, auditors should explain the nature of the engagement and obtain agreement up front regarding how and when they will release reports. Is the review a formal internal audit, or is it a health check or other activity aimed at performance improvement? Will the reporting be delivered through standard channels or directly to the project’s governance structure? The answers to these questions guide the reporting for eventual review.
Internal audit and the business should also agree on the most efficient and practical reporting format. Agile projects run at high speed and in high-pressure environments — quite often, value can best be realized by near-real-time feedback. Timely, practical, and actionable reporting is key to Agile’s success.
Relevance and Value
As noted in PwC’s 2015 State of the Internal Audit Profession Study, internal audit functions that focus on adding value are outperforming other teams in terms of business alignment and talent models. Understanding a project’s objectives, as well as the risks associated with project methodology, helps enhance the value internal audit can deliver. The key is simple: Engage with teams early, understand what they’re doing, modify the approach as needed, and provide relevant feedback — all while helping the Agile teams and the organization better understand and control risk.