Technology plays a vital role in any organization's strategic initiatives, yet every year countless initiatives fail to deliver value. Take Cover Oregon, a $305 million health insurance exchange website intended to help people find, and sign up for, health coverage. When it failed in 2014, the state resorted to paper forms and hired hundreds of workers to enroll people manually.
Such failure is not limited to business applications. Today, a new car has more lines of code than Microsoft Office, and project failure can lead to death or, in the case of Volkswagen, fraud. The company's diesel emissions scandal has cost it $30 billion.
Over the past two decades, about 70 percent of IT projects have failed, according to the Standish Group, a Boston-based firm that researches software development project performance. Some of these projects are canceled and never used, while others fall short of achieving the original business intent. Despite this high failure rate, some organizations have found ways to deliver more projects on time, on budget, and with better outcomes. The Project Management Institute's (PMI's) 2018 Pulse of the Profession report calls these organizations champions because of their 92 percent average success rate. Internal auditors can learn from both the failures and successes of these organizations.
Governance is about making good decisions. Many organizations have an IT governance function, which provides a formal structure for aligning IT strategy with business strategy. The International Standards for the Professional Practice of Internal Auditing requires internal auditors to make sure IT governance sustains and supports the organization's strategies and objectives (Standard 2110: Governance). IT governance should address the progress and decision-making of projects. At Volkswagen, governance failed at the highest levels, while there was no single point of authority overseeing its development at Cover Oregon. These findings resonate with PMI research reports that show that an actively engaged executive sponsor is a leading factor in project success.
Measuring Progress Projects do not fail overnight, but employees often do not accurately report project status information or speak up when they see problems, a Spring 2014 MIT Sloan Management Review article asserts. According to "The Pitfalls of Project Status Reporting," when employees see negative outcomes for others who have delivered bad news, they may fear that executives will "shoot the messenger." Such was the case at Volkswagen. Rather than telling management that the engineers could not meet the emission standards, they modified the software to manipulate the results, according to a whistleblower's account.
Successful organizations do not hide problems. They have a culture that encourages people to bring problems into the open where they are solved quickly. Internal auditors should assess the culture around project reporting to ensure it is transparent and honest.
Decisions A $10 million IT project will have approximately 15,000 decisions, the Standish Group estimates. With each bad decision, the odds of success diminish. Yet, the most critical decision is whether to start the project at all. For Cover Oregon, this first decision could have changed the outcome of the project. The organization opted to develop a web application from scratch when an existing solution was available.
Internal auditors should review the criteria organizations use for evaluating, selecting, prioritizing, and funding IT investments. Decision-makers need an accurate picture of the resources needed for each proposed project, but estimating these resources is difficult. People tend to be overly optimistic. This is known as the planning fallacy, which can lead to time overruns, cost overruns, and benefit shortfalls.
Internal auditors should counteract the planning fallacy with a stress test. Research from Bent Flyvbjerg and Alexander Budzier, published in the September 2011 Harvard Business Review, found that one in six of the nearly 1,500 IT projects they studied had a 200 percent cost overrun and almost 70 percent had a schedule overrun. Based on this data, they devised a stress test. An organization should proceed with a large IT project only if it can absorb a budget overrun of 400 percent and is comfortable only achieving 25 percent to 50 percent of the projected benefits.
Organizations also should consider ways to reduce the project's complexity. Technology is rarely the cause of project failure. It is the complexity of other factors that lead to failure. When planning any change initiative, the organization needs to consider the impact the project may have on the existing organizational culture, the training resources needed, the effect of new regulations, changes to the business environment, the effort to change business processes, and how the organization will manage vendor relationships.
Often, these factors fall prey to the planning fallacy, which can quickly increase the complexity of a large IT project and reduce the chances of meeting the original business intent. An example is the 2013 U.K. National Health Service System, which overran costs by £11 billion ($15.3 billion) and was delivered nine years late. The complexity resulting from using four vendors and numerous specification changes led to failure.
The most effective way to reduce complexity is to limit the size of the project, the Standish Group advises. Based on evaluating more than 50,000 IT projects, the firm's researchers found that a small project, consisting of six team members and completed in six months or less, works best. The firm recommends turning large projects into a series of small ones, which can dramatically increase the chances of success.
Research from the Boston Consulting Group aligns with these findings. The firm has developed an online tool called DICE that internal auditors and organizations can use to assess the readiness of a project based on four elements:
- Duration, or the interval between the project's major "learning milestones" if it lasts six months or longer.
- Performance integrity of the project team. This element encompasses both the overall skills and traits of the team, and how the team has been configured.
- Commitment to change shown by the senior management and the people actually undergoing the change.
- Additional local effort above normal working requirements that is needed during implementation of those undergoing the change, as opposed to the project team.
Although lessons learned are an important part of the project management life cycle, it often is the most ignored part of a project. Organizations with poor success rates do not have a good process for identifying and applying lessons to new projects. Many organizations have not established a repository for sharing knowledge across the business. As a result, valuable knowledge can be lost or forgotten and projects continue to fail for the same reasons. Internal auditors can review whether the organization has a culture of learning from mistakes and how it shares and applies that knowledge to future projects.
Improving Success Chances
Despite the high risk of IT project failure, internal auditors can help their organization beat the odds by reviewing the governance, complexity, and lessons learned from projects. Specifically, they should evaluate the risks related to large technology projects and perform health checks during key project milestones defined in the project plan. Moreover, they should benchmark the organization's current project success rate against the PMI Pulse of the Profession. A future of more successful technology initiatives starts with improved controls today.