Editor's Note: The Human Factor's Note: The Human Factor<p></p> <p>I’m a big fan of the TV series <em>Westworld</em>. For those who haven’t seen it, HBO’s science fiction thriller takes place in a Western-themed, no-holds-barred amusement park where guests interact with lifelike robotic hosts. The show’s many plot twists keep viewers guessing, though eventually we learn there’s much more going on than just gun fights and pleasure seeking. The park’s creators have been quietly taking advantage of guests to carry out a hidden agenda. And while the plan relies in part on Westworld’s futuristic technology, one of its main tools is simple human deception.</p><p>Beyond the realm of fiction, of course, people’s susceptibility to deception and manipulation is a real-world concern for organizations — particularly when it comes to cybersecurity. With a phone call, email, social media exchange, or in-person conversation, skilled social engineers can gain the trust of their victims to commit fraud or other organizational crimes. And as Kimberly Hagara, vice president, Audit Services, at University of Texas Medical Branch, notes in “<a href="/2018/Pages/Pulling-Strings.aspx">Pulling Strings</a>,” the attackers are becoming increasingly sophisticated. “Now the tactics are much more trust-based,” she says. “Getting into an organization or a system relies more on human interaction.”</p><p>In some cases, the attackers leverage systems access to hold the organization’s data hostage. Their success depends not only on malicious software, known as ransomware, but often on the perpetrators’ ability to deceive. According to a recent survey by security firm SentinelOne, nearly 70 percent of successful ransomware attacks in 2017 resulted from hackers gaining access to enterprise networks by phishing via email or social media. </p><p>In our cover story, “<a href="/2018/Pages/Held-Hostage.aspx">Held Hostage,</a>” author Arthur Piper examines the risk of ransomware, how to respond to an attack, and considerations for prevention and detection. The article also stresses that employees often represent the greatest vulnerability to these types of attacks. With that in mind, risk management advice includes ensuring training is provided to all personnel and that policies on responding to ransomware incidents have been well-communicated.</p><p>Cyberattacks don’t have to be high-tech to present a real threat. Despite all the sophisticated tools available for carrying out an attack, crafty perpetrators can weasel their way through even the best defenses with simple techniques that exploit human psychology. Ironically, in the age of artificial intelligence and advanced digital security, preventing cybercrime often comes down to a deeper understanding of nontechnological, human factors. The weakest link in the security chain is often the employee who opens the door, physical or virtual, to an intruder. And when that happens, to borrow from <em></em><em></em><em></em><em></em><em></em><em></em><em>Westworld</em>’s season two tag line, “chaos takes control.”​</p>David Salierno0
Held Hostage Hostage<p></p> <p>The City of Atlanta is still trying to recover from the March 2018 SamSam ransomware attack that demanded $51,000 in bitcoin. More than one-third of the city government’s online systems were frozen, and staff were initially told not to turn on their computers in case the malware spread. Atlanta’s public safety services, such as 911, police and fire rescue, as well as Hartsfield-Jackson Atlanta International Airport, were mostly unaffected.</p><p>When the attack occurred, the city was in the process of improving its cyber defenses following an internal audit report. Chief Audit Executive (CAE) Amanda Noble says it is too early to tell what lessons can be learned from the incident, but she says the fact that most emergency services stayed up and running suggests that the city had done a good job of segmenting its network before the attack — one of the audit recommendations.</p><p>Noble says about 600 of the city’s 8,000 computers were affected. What struck her most immediately following the attack was the difficulty communicating throughout the city without email. Because local device hard drives had been potentially compromised, it was important to identify which ones were impacted before giving people access to their equipment.<br></p><p>“The day after we learned of the attack, building security was passing out notices asking staff not to use their computers,” she says. While the City Auditor’s Office had done a business continuity audit for the city, they had not done one for her own department. Auditors were locked out of their laptops for several days. </p><p>She says that organizations prioritize their most sensitive assets first — which is only natural — but they should be looking at how the entire enterprise can be affected during an attack, whether they have the resources in the short term to deal with those other areas or not. “It is worth remembering that Atlanta was not a uniquely vulnerable organization and that this was not a particularly sophisticated attack,” she says. “Organizations should start approaching this by thinking in terms of not if this will happen, but when. Think about how to recover and about your communication plan.”</p><h2>To Pay or Not?</h2><p>Initial clean-up costs in the weeks following the Atlanta attack have been widely reported to have topped $2.6 million, with more remediation efforts needed longer term. In June, Daphne Rackely, the city’s interim chief information officer (CIO), requested an additional $9.5 million for recovery efforts from city council as the city continues to find more problems with its systems, including the loss of more than a decade of legal documents and years of police dash-camera footage. </p><p>Ramsomware is a specific type of malware that infects computers and mobile devices and, in doing so, restricts users’ access to files. Attackers often threaten to permanently destroy data quickly unless a ransom is paid — or they increase the size of the demand incrementally each time a deadline for payment has been reached. The initial ransom demand can be small. So, with recovery effort amounts in Atlanta now topping $14 million vs. the total reported ransomware demand of $51,000, why not just pay? </p><p>Official government advice in the U.S. and U.K. is not to pay. “From the U.S. government perspective, we definitely discourage the payment of ransom,” Neil Jenkins, former director of the U.S. Department of Homeland Security’s Enterprise Performance Management Office, told the online magazine ZDNet last year. “From a national perspective ... paying ransom encourages the business model,” he said. “The reason this has become such a popular thing to do is they’re actually making money off of this.”</p><p>Cyber defense experts tend to agree, even though the financial calculations may initially make payment attractive. “If you are a CEO losing $100,000 a day and the ransom is $300,000 in bitcoins, you could potentially get your money back in three days,” Raj Rajamani, vice president of products at endpoint protection company SentinelOne in Mountain View, Calif., says. “But in the longer term, you are paying the attackers to become more sophisticated by helping them reinvest in building better attack technology.”</p><p>Not only that, but paying ransom does not work in most cases. According to the SentinelOne Global Ransomware Report 2018, of the 45 percent of U.S. companies imp​acted by ransomware in 2017 that paid at least once, only 26 percent got their systems back from the attackers. Seventy-three percent of those that paid were attacked again. For most, paying was a lose-lose scenario. </p><p>Most worrying, 44 percent of respondents claimed that ransoms have been paid without the involvement or sanction of IT and security teams. “Depending on how high up in the organization the employee is and what kind of data has been stolen, maybe he or she doesn’t know how to react, sees it as their fault, and wants to hide it under the radar until the data can be retrieved,” Rajamani explains. “The intention is understandable, but the reality is you are putting the rest of the organization at risk.”</p><p>Organizations need to accept that people make mistakes and that if they become a victim of ransomware, they should feel free to raise their hand and tell someone immediately, Rajamani says. “These attacks are inevitable, so organizations should avoid creating a culture of fear where people feel they’ll lose their jobs for coming forward with a problem,” he adds.</p><h2>Make Routines Routine</h2><p></p><table cellspacing="0" width="100%" class="ms-rteTable-default"><tbody><tr><td class="ms-rteTable-default" style="width:100%;">​ <style> p.p1 { line-height:12.0px; font:14.0px 'Interstate Light'; } p.p2 { line-height:12.0px; font:42.5px 'Interstate Light'; } p.p3 { line-height:12.0px; font:9.0px 'Interstate Light'; } p.p4 { text-indent:12.0px; line-height:12.0px; font:9.0px 'Interstate Light'; min-height:11.0px; } p.p5 { text-indent:-12.0px; line-height:12.0px; font:9.0px 'Interstate Light'; } span.s1 { vertical-align:1.5px; } span.s2 { letter-spacing:-0.1px; } </style> <p><strong>Six Steps to Better Security </strong></p><p>As ransomware is on the rise, Michael Lisenby, managing partner at Rausch Advisory Services LLC in Atlanta, gives advice for minimizing the odds of an organization falling victim to an attack. ​</p><ol><li>Establish security awareness campaigns that stress the avoidance of clicking on links and attachments in email from unknown senders. That could include, for example, the technology department running phishing campaigns, which internal audit evaluates in terms of the effectiveness of the organization’s training and education processes and to identify frequent offenders.<br></li><li>Ensure antivirus software is installed and is up-to-date across all endpoints within the business. Antivirus software on its own is unlikely to be enough, so the organization may also evaluate next generation antivirus programs that include endpoint protection. This can look for ransomware attempts and provide IT with the ability to monitor attacks to stop them from spreading. Internal audit should be looking at the cyber defense IT road map and strategy and evaluate configurations.<br></li><li>Use content scanning and filtering on mail servers. Inbound emails should be scanned for known threats and should block any attachments that could pose a threat. While spam protection should identify and block a lot of these attacks, advanced threat protection tools should be inserted into the mail flow, which will look for and quarantine unsafe messages that may contain malware, for instance. It can also scan URLs to ensure phishing attachments are identified and protected.<br></li><li>Restrict users’ ability (permissions) to install and run unwanted software applications and apply the principle of “least privilege” to all systems and services. Restricting these privileges may prevent malware from running or limit its capability to spread through the network.</li><li>If the data is backed up to an external storage device, remove the device after backup so that if ransomware does infect the computer, it won’t be able to spread to the device. Where organizations depend on cloud backup, ensure there is off-site replication of essential data.<br></li><li>Apply a patch management system, making sure all desktop clients are fully patched. Ensure the system is patching commonly exploited third-party software — such as Java and Adobe Flash — which will prevent many of these types of attacks from being successful.​<br></li></ol></td></tr></tbody></table><p>Organizations need to ensure they are paying close attention to basic IT routines. “The reason attackers are able to get in and get this kind of control over companies’ systems is because the company has failed to do something it should have done,” says Neil Frieser, senior vice president of internal audit at telecommunications company Frontier Communications in Norwalk, Conn. And internal audit’s role is to understand whether basic security policies and routines are in place and have been followed.</p><p>“Failure to patch vulnerabilities in a timely way is No. 1 on the list of cybersecurity issues,” Frieser says. Manufacturers regularly update their hardware and software with patches that help to protect those devices and programs from attack via vulnerabilities. Unlike consumers, who can generally download the latest updates with the click of a button, companies have to ensure that when they apply a patch to a particular system, it will still work as intended on the network. Frieser says it is critical for someone on the network infrastructure team to ensure that patching happens timely across the organization.</p><p>“I’m a big believer in the concept that routine things need to be done routinely and patch management falls into that,” he says. “It has to be a priority because it only takes one vulnerability to create potentially serious problems.”</p><p>During Frontier’s annual cybersecurity audit, Frieser’s team looks to see whether the business has any exposures on patching that are known about, but not yet dealt with. They also look at the process. “Just because there are no outstanding issues does not mean that the patching process is good,” he says. “Someone may have just done the patch updates because they knew the auditors were coming.”</p><p>The other major issue for Frieser is access reviews. Auditors should be periodically looking at all of the users in key systems. Generic IDs and passwords should be weeded out. Key questions to consider, he says, are whether there are IDs that have not been used for long periods or IDs that are associated with people who are no longer with the company or with people who have changed roles and no longer need the same access levels.</p><p>“If you have a generic ID for administrator, with “admin” set as the password — and where it’s shared — it is crazy to have that in your company’s infrastructure,” Frieser says. Privileged access is a critical area for auditors to focus on, because hackers who get into the system can begin to shut things down associated with that access point — and potentially hold the business for ransom. </p><p>While organizations and auditors are generally aware of both of these key areas, they need to be constantly monitored. “Issues often arise due to laziness,” he says. “For example, someone might set up a generic admin ID and password in the throes of implementation, which they intend to change, but then forget about it and it becomes a vulnerability.” </p><h2>The People Factor</h2><p>Even with good controls over patch management and access rights, organizations can still be at risk of a ransomware attack. </p><p>“A lot of technical security has been commoditized to the extent that it is hard to switch off the safety measures in the software where it has been properly patched,” says Edward Wolton, deputy CEO at the London-based security consultancy Templar Executives. “People are often the greatest vulnerability, especially if they do not know what to do in the case of an attack.” Organizations need to put in place training for all personnel and have a well-circulated policy on what to do in case of a security breach.</p><h2>Boards Are Paying Attention</h2><p>One of the more fortunate side effects of recent attacks, such as that on the City of Atlanta and last year’s WannaCry that affected the U.K.’s National Health Service (NHS) among many others, is that it has brought the issue into the boardroom. In May 2017, WannaCry caused the NHS to cancel 20,000 hospital appointments and affected 80 of its 236 Trusts, which are responsible for running the organization’s health services — everything from hospitals to ambulance services — as well as hitting 200,000 computers in at least 100 countries. An April 2018 report by the U.K. government’s House of Commons Committee of Public Accounts said the attack most likely exploited unpatched vulnerabilities in Windows XP — even though the NHS had been warned about the dangers repeatedly since 2014. </p><p>Wolton says media coverage of the NHS attack in the U.K. suddenly made organizations and their boards pay attention and has, in some ways, made ransomware less of a threat due to raised awareness. It also has provided CAEs with an opportunity to advocate for cybersecurity to be moved further up on the agenda. “Traditionally, responsibility for IT security has been pushed downstairs by the board to the CIO,” Wolton says. While many CAEs are not on the board, he advises them to ensure there is board-level sponsorship for the issue — and that the sponsor really understands the nature of the threat to the organization.</p><p>“While it is changing rapidly, too many businesses fail to have a senior-level sponsor who understands the risks and the level of network and governance controls needed to minimize the threat,” he says. In devising a policy on ransomware that spells out the organization’s response, boards will need to decide on the level of risk they are prepared to accept and review their backup policies and procedures. If they decide that in certain circumstances they will pay the ransom, they will also need a cryptocurrency policy and capability.</p><p>Internal audit has an opportunity to educate the board and expand its influence. From a board perspective, internal audit should be working with boards to develop reporting metrics and monitor protocols to evaluate the organization’s cyber defenses and, in turn, help mitigate the risk of future attacks.</p><h2>Recovery From an Attack</h2><p>Wolton says organizations have become a victim of progress when it comes to backing up critical information. Twenty years ago, for example, most businesses had separate monthly, weekly, and daily backups, with the first two types being stored off-site. Today, many rely on continuous cloud-style backups. With this newer technology, it can be difficult to wind the clock back after a ransomware attack and identify when the system first became infected. That is why a robust backup policy and detection capabilities are crucial.</p><p>In fact, while awareness of ransomware threats is rising, many organizations are not looking at the problem from a recovery perspective. “A lot of CAEs and CIOs are now doing risk assessments on ransomware, but fewer are considering it from a disaster recovery perspective,” says Michael Lisenby, managing partner at Rausch Advisory Services LLC in Atlanta. Lisenby says CAEs should be approaching the problem from the perspectives of prevention, detection, removal, and recovery.</p><p>“That entails conducting table-top scenarios with all those who are likely to be involved in dealing with a ransomware crisis,” he says. The more the team members have practiced the routine, the less likely they will be surprised by their vulnerabilities. In the Atlanta and NHS attacks, for example, the reality of having to communicate without emails had not been fully tested. Lisenby says it is worth the team considering the threats to their operations both from a business and an IT perspective to get a full view of the enterprisewide nature of the risks. Because the entire organization is affected, he says the heads of legal, finance, human resources, IT, risk, internal audit, and others should be involved — as should regulators, where appropriate.</p><p>“This is not a once-in-a-lifetime exercise, it has to be done annually,” Lisenby says. That is because the nature of ransomware attacks and their impact on an organization are constantly changing. For example, Internet of Things (IoT) devices are opening up new and unlikely vulnerabilities. “I know of a casino where player data was stolen from its systems,” he says. The culprit? A smart thermostat in an aquarium on the shop floor.</p><p>“There are products out there that enable you to scan to see if IoT devices have been added, and you can make sure they are segmented from the network and access of least privilege is associated with them,” Lisenby says. “But only if you keep on top of the issue and make sure you have the right routines in place.” </p><h2>Best to Be Prepared</h2><p>Ransomware attacks are simple and effective. Organizations need only one point of weakness to be vulnerable, so, as Noble says, it is more a case of when it happens, rather than if it will. Having a proactive approach to the problem with regular and effective training for staff across the entity is a good place to start. But organizations also need to have well-tested plans for when an attack strikes successfully, with effective data protection systems in place and business continuity routines that work. </p>Arthur Piper1
The Audit Bots Audit Bots<p>​Macro trends such as global cost competitiveness and cyber risk are driving organizations to innovate. One of the innovations that many organizations are implementing is robotic process automation (RPA), the automation of repetitive human activity in existing systems. </p><p>RPA uses coded scripts, or "bots," that work across multiple applications to perform repetitive, nonvalue-added, manual tasks. Automating these processes yields a higher processing rate in a fraction of the time. All facets of business can benefit from bots processing transactions, data, or requests.</p><p>In addition to their benefits to the business, emerging technologies such as RPA also impact internal audit and its value proposition. While the use of bots promotes an environment for internal controls and compliance auditability, additional review and considerations may arise. Management will call on internal audit to review these automated processes. </p><h2>What's a Bot? </h2><p>RPA bots perform tasks consisting of a wide variety of steps. They can validate system data, determine conclusions through logical checks, generate documents and information requests, and input received data into systems. Examples across the business include following up on invoice delivery dates with suppliers, performing pricing analysis, and processing leave of absence requests. </p><p>For invoice delivery, the bot begins by executing validation reports for invoices with unknown delivery dates and the associated supplier contact information. Then, the bot populates an email template requesting additional information from the supplier, performing follow-up as needed. Suppliers respond via a standardized form that allows the bot to extract the information and upload it to the purchasing system. If information is received in an incorrect format the bot can't read, escalation protocols flag these instances for human review. Furthermore, the bot works within the purchasing and email systems under a provided username and password, following the regular established invoice delivery date follow-up process. Bots can complete thousands of these emails in a day, which would otherwise take a group of employees more than a week.</p><p>By performing these types of repetitive time-consuming tasks, RPA can increase productivity, speed, and consistency of execution and employee satisfaction. The increase in employee bandwidth can enable employees to focus on higher-value tasks, further increasing efficiencies and driving down cost. Moreover, RPA can minimize common errors caused by human operators. </p><p>The process improvements from using RPA can quickly surpass gains from other business strategies such as outsourcing. Unlike some outsourcing arrangements, though, RPA allows an organization to retain control over business processes, quality consistency, and security of intellectual property.</p><p>Moreover, organizations can get started with RPA quickly. Bots can be trained for their new job (coded, tested, and deployed) and then turned over to the business for monitoring and management within 12 weeks. Once deployed, bots can be scheduled to process large amounts of mundane tasks continuously. Other bots may only need to run periodically to handle the workload available. No changes to current systems or interfaces are required, lowering the cost of entry, as bots work within existing processes. This activity includes logging into systems as a user to complete tasks, providing an audit trail for their activity.</p><h2>A New Mission</h2><p>To accomplish its mission to help add value and improve the organization's operations, internal audit needs to review the three core elements of its value proposition: assurance, insight, and objectivity. As innovations change the business landscape, internal audit must adapt and realign its value proposition to help the business meet its strategic, financial, and compliance objectives in new ways.</p><p>In organizations that are adopting RPA, internal audit can provide assurance that controls programmed into automated processes will drive down risk. The goal of leveraging technology, especially in repetitive tasks, is to drive down overhead cost. As a result, reducing overhead applied to products and services will have a positive impact on overall expenditure. </p><p>However, leveraging emerging technology comes with risk. The security of new technology is not as robust as established technology and is more prone to breaches, hacks, and malware. <a href="" target="_blank">Economic Impact of Cybercrime</a> (PDF), a February study by the Center for Strategic and International Studies, estimates that the global cost of cybercrime could be as much as $600 billion. Recent headline-making breaches of personal information and cyberattacks have negatively impacted companies' brands and may severely limit their ability to maintain customers or win new business.</p><p>From mom-and-pop shops to Fortune 100 companies, cybersecurity is critical, yet the demand for cybersecurity skills has outpaced the supply. Internal auditors with a strong competency in IT controls will be crucial in providing assurance that the technology control environment is effective in safeguarding proprietary information from intruders. </p><h2>Reviewing RPA</h2><p>Identification of automation opportunities occurs continuously. Internal audit should get involved at this first step, which they can accomplish as part of the regular audit process. During reviews, auditors should partner with stakeholders to examine and map current processes and discuss potential inefficiencies. </p><p>Outside of the audit process, internal audit and other internal groups such as data scientists, process-improvement experts, IT, and business process owners can collaborate to explore solutions to streamline processes. These teams should continuously assess existing processes and procedures to break out of the "if it is not broken, do not fix it" mindset.</p><p>A key difference between RPA and other IT applications/tools is the shortened development cycle. While many aspects remain the same — such as separate testing/production environments, quality assurance testing, and change management — additional concerns and control differences may arise from the agile development cycle and the use of the bots themselves. Reviews should ensure general IT controls and processes such as enterprise password requirements, backups, and regression testing are followed during bot development. International companies also should consider foreign regulations and export/import concerns.</p><p>Teams involved in RPA implementation may leverage existing control testing and monitoring activities, but they may need to consider new aspects. For example, U.S. Sarbanes-Oxley Act of 2002 IT testing reviews items such as appropriate access to financial systems and generic usernames. When there are bots involved, this testing also should include bot user profile review and testing for ownership/access to the bots themselves. </p><p>When performing RPA audits, internal auditors should ask questions concerning new considerations such as: </p><ul><li>Who manages password updates for the bots to ensure company password requirements are being followed? </li><li>Is there a plan in place to address situations where a bot fails to appropriately escalate exception incidents that would impact the financial statements, systems, or processes? </li><li>Are bots accounted for on software license reviews and are they functioning on the most current versions of software?</li><li>What is the disaster recovery and business continuity plan for RPA?</li></ul><h2>Automating Audits</h2><p>Internal audit's role in robotic process automation is not limited to assisting stakeholders and other functions in their solutions to streamline processes. Becoming part of the company's RPA team also means reviewing processes within the audit department, embracing change, and encouraging innovative discussions to leverage emerging technologies. </p><p>The internal audit department also can benefit from bots. Tasks that can be automated through RPA include manipulation of digital/electronic data, standardized inputs and formats, and rule-based processes that yield a low number of exceptions. Many of these types of tasks exist within the internal audit process.</p><p>After the annual audit plan is finalized, a bot could set up and populate each audit with appropriate checklists and templates to eliminate the need for manual set-up. During audits, bots could handle a variety of tasks such as automatically creating workpaper attachments and filling out standardized templates and headings in workpapers, streamlining cross-references and issues, and creating reports. </p><p>Data analysis could be performed with trends noted and ready for management review. After an audit is completed, a bot could send the audit report to stakeholders, consolidate management responses, and follow up as due dates approach.</p><h2>Challenging the Organization</h2><p>While the core elements of value delivered to the organization remain the same, internal audit also must act as a proactive thought leader about new innovations. The audit function must continuously challenge the organization and communicate the importance of automation, driving efficiencies, and streamlining processes. </p><p>By partnering with technology and implementation teams, internal audit can be an integral part of implementing RPA and provide assurance on system controls. In addition, auditors must continuously educate themselves to be ready to tackle obstacles the organization may face over the next horizon. </p>Jaimie Yang0
Auditing Analytic Models Analytic Models<p>T​he analytics gold rush is on. Organizations around the world are spending considerable money to build or buy analytic models and analytics capability to take advantage of big data, machine learning, and artificial intelligence (AI) technologies. These models have made their way into every aspect of business and are being relied on as decision support — and, in the case of machine learning and AI, actually making the decisions — for issues such as:</p><p></p><ul><li>Determining the probability of default for potential borrowers (corporate and individual).</li><li>Evaluating new employees' probability of success and tenure with the organization (from professional athletes to salespeople).</li><li>Forecasting success and return on investment for new marketing initiatives.</li><li>Making product mix and store location decisions.</li><li>And coming soon, making life-and-death decisions in self-driving vehicles.</li></ul><p><br></p><p>Today's organizations have billions of dollars riding on the accuracy and performance integrity of analytic models. With model performance becoming a strategic enabler, organizations need to manage the risks associated with analytics. </p><p>To effectively manage these risks and move beyond simple financial model or spreadsheet auditing, organizations need a system of controls around analytic model development, application, and maintenance. These analytics controls provide checks and balances around model selection, validation, implementation, and maintenance. Periodic internal audits can help determine whether analytics controls are designed appropriately and operating effectively. </p><h2>Models and Controls</h2><p>An analytic model is a mathematical equation that takes in data and produces a calculation such as a score, ranking, classification, or prediction. It is a very specific set of instructions for analyzing data to deliver a particular kind of result — behavior, decision, action, or cause — to support a business process.</p><p>The objective of analytics controls is to ensure that:</p><p></p><ul><li>Analytics personnel have the appropriate skills and training.</li><li>Input data is appropriate, complete, authorized, and correct.</li><li>Model selection procedures are documented and justified.</li><li>Model validation and testing have been conducted in accordance with scientific principles. </li><li>Outputs are accurate, complete, and being used by the business as intended.</li><li>The model is refreshed and reevaluated periodically. </li><li>The organization maintains a record to track the processing of data from input, to processing, to the eventual output.</li></ul><p><br></p><table class="ms-rteTable-default" cellspacing="0" style="width:100%;"><tbody><tr><td class="ms-rteTable-default" style="width:100%;">​Read <a href="/2018/Pages/Analytic-Model-Controls-and-Tests.aspx"><span class="ms-rteThemeForeColor-1-0">"Analytic Model Controls and Tests"</span></a> to see examples of tests auditors can perform to provide assurance on controls used in analytic models.<br></td></tr></tbody></table><p>There are several types of analytics controls. <em>Skills controls</em> provide assurance that data analytics personnel are competent and sufficiently trained in relevant analytics methods. <em>Business-use controls</em> provide assurance that the model addresses the intended business objective. <em>Data controls</em> are used mainly to check the integrity of data entered into an analytic model. <em>Model selection controls </em>ensure model selection is appropriate and reasonable to provide decision support. <em>Model validation controls</em> address what is done to ensure the model output is reasonable and accurately reflects the underlying nature of the input data. <em>Output controls</em> provide assurance that the model output is presented and used in an appropriate and justified manner to ensure it remains consistent and correct. <em>Maintenance controls</em> address the need to reevaluate and refresh analytic models periodically to ensure they are still relevant in the current environment.</p><h2>Analytics vs. IT General Controls</h2><p>Internal auditors need to understand the relationship and difference between analytics controls and IT general controls. Otherwise, an analytics controls review may not be scoped appropriately, negatively impacting the audit's quality and coverage.</p><p>IT general controls apply to all systems components, processes, and data present in an organization or systems environment. The objectives of these controls are to ensure the appropriate development and implementation of applications, as well as the integrity of program and data files and of computer operations. </p><p>Analytics controls differ from IT general controls because they relate to the methodology and data pertaining to each analytic model. They are specific to each individual application. </p><p>Internal auditors must note the degree to which management can rely on analytics controls for risk management. This reliance depends in part on the design and operating effectiveness of the IT general controls. If these controls are not implemented or operating effectively, the effectiveness of analytics controls is greatly diminished. For example, if the IT general controls that monitor program changes are not effective, then unauthorized, unapproved, and untested changes to an analytic model can be introduced to the production environment, thereby compromising the overall integrity of the model.</p><h2>Analytic Model Categories</h2><p>There are three main categories of analytic models: descriptive, predictive, and prescriptive. Each category can provide an organization value and strategic insight.</p><p><strong>Descriptive</strong> These models allow organizations to condense big data into smaller, more digestible pieces of information. Typically, organizations that use analytics meaningfully have mountains of raw data at their disposal. Descriptive analytics enables an organization to summarize that data and determine what really happened. Most analytics in use are descriptive: sales breakdowns, social media likes and followers, ratings, and reviews.</p><p><strong>Predictive</strong> The next level up in data analysis, predictive analytics uses a variety of statistical, modeling, data mining, and machine learning techniques to study recent and historical data, enabling analysts to identify patterns and correlations in the data. Based on these identified patterns and correlations, analysts can create a model of the future results given selected inputs. For example, based on certain borrower characteristics, a bank may use a predictive model to forecast its amount of loan defaults.</p><p><strong>Prescriptive</strong> The highest level of analytics, prescriptive analytics recommends one or more courses of action and shows the likely outcome of each decision. Unlike a predictive model, a prescriptive model shows multiple future scenarios based on a decision the organization makes today. Prescriptive analytics requires a predictive model with two additional components: actionable data and a feedback system that tracks the outcome produced by the action taken. An example of prescriptive analytics would be a casino floor product mix optimization model that predicts revenue gains given various game configurations. </p><h2>Risk Assessment</h2><p>Auditors should use risk assessment techniques to identify critical vulnerabilities pertaining to the organization's reporting and operational and compliance requirements when developing the risk assessment review plan. These techniques include the review's nature, timing, and extent; critical business functions supported by analytic models; and the extent of time and resources to be expended on the review.</p><p>To add value to organizationwide analytics control risk assessment activities, internal auditors should define the universe of analytic models and supporting technology (modeling software, data services, etc.). They also should summarize the risk and controls using the risk and control matrices documented during the risk assessment process. </p><p>Next, internal auditors should define the risk factors associated with each analytic model by answering questions such as:</p><p></p><ul><li>Does the model support a regulatory requirement?</li><li>How complex is the model type?</li><li>How effective is the design of analytics controls? </li><li>Is the model prepackaged (off the shelf) and customized or developed in house? </li><li>Does the model support </li><li>more than one critical business process?</li><li>How is the data processed by the model classified (e.g., financial, private, or confidential)?</li><li>How frequently are changes made to the model?</li><li>How complex are those changes? </li><li>What is the model's financial impact?</li><li>How effective are the IT general controls residing within the application (e.g., change management, logical security, and operational controls)</li></ul><p><br></p><p>Once they have answered these questions, internal auditors should weigh all risk factors to determine which risks need to be weighed more heavily than others (see "Assessing Model Risk" below). From there, they should determine the right scale for ranking each application control risk by considering qualitative and quantitative scales, such as:</p><p></p><ul><li>Low, medium, or high control risk.</li><li>Numeric scales based on qualitative information (e.g., 1=low-impact risk, 5=high-impact risk; 1=strong control, 5=inadequate control).</li><li>Numeric scales based on quantitative information (e.g., 1=less than $50,000 and 5=more than $1 million).</li></ul><p><br></p><p>With this information in hand, internal auditors should conduct the risk assessment, rank all risk areas, and evaluate the risk assessment results. Finally, they should create a risk review plan that is based on the risk assessment and ranked risk areas.</p><p><img src="/2018/PublishingImages/Sammy_pp.52-53.jpg" alt="" style="margin:5px;" /><br></p><h2>Audit Methodology</h2><p>Internal auditors should keep in mind that the review's scope, depth, approach, and frequency depend on the results of the risk assessment and the availability of internal audit resources. If the analytics team uses a recognized methodology for model development such as the Cross-Industry Standard Process for Data Mining (CRISP-DM) or some other widely accepted system, then internal auditors should consider auditing to that standard. In addition, some organizations have established a model risk management function. Internal audit can audit that area using similar methodology to that applied to other compliance functions. </p><p>For organizations whose analytics teams do not use a prescribed model development methodology, there are two approaches auditors can use to audit analytics controls: the Integrated Model Review Methodology (IMRM) and the Stand Alone Model Methodology (SAMM). These methods apply CRISP-DM principles in an internal audit context.</p><p><strong>IMRM</strong> This approach can be used to evaluate model risk by examining all the business processes that feed or are dependent on the model being reviewed. When using the IMRM, internal auditors should include within the review's scope all the organization's systems that are involved in the model under review and whether the implementation of the model is consistent with the organization's analytics strategy. In other words, the auditor needs to include within the review's scope the separate processes that make up the different components of the model cycle. The auditor then can identify the inbound and outbound interfaces within the model and complete the scoping activity. For example, when auditors review a marketing campaign response model, they would scope in survey methodology and data collection processes, customer segmentation processes (inputs), and marketing decisions made based on model output.</p><p>Using the IMRM approach automatically devotes more audit resources to those analytic models that affect a larger portion of the organization's operations. To use the IMRM effectively, auditors need to understand the business processes surrounding the use of the model being reviewed and how data flows into and out of the model.</p><p><strong>SAMM </strong>The alternative approach, the SAMM, is used when the auditor wants to review the controls within a single model. The SAMM is useful for new models or when audit resources are limited. Essentially, the auditor is verifying that the model, itself, has appropriate controls and performs the intended function. It does not provide assurance as to whether the organization is using the model output effectively or whether the model inputs are valid. Although SAMM is effectively a subset of the IMRM, internal auditors should clearly specify which methodology they are applying so that management and the audit committee know the extent to which they can rely on the results.</p><h2>It's Still Internal Auditing</h2><p>Although many auditors may be unfamiliar with analytic models, machine learning, and AI, the fundamentals of internal auditing remain the same. As with all new technologies and processes that organizations have embraced, internal auditors have a responsibility to learn how analytic models can be useful in their work and adapt their methods to serve their stakeholders.<br></p>Allan Sammy1
The Human Side of Transformation Human Side of Transformation<p>​More organizations are making strides with digital transformation initiatives, but the human side is proving difficult, according to <a href="" target="_blank">new research</a> from <em>MIT Sloan Management Review</em> (<em>MIT SMR</em>) and Deloitte Digital. Just one-fourth of survey respondents in The Coming of Age Digitally report say their organization is in the early stages of digital transformation, down 9 percentage points from the 2017 study. </p><p>Forty-four percent say they are at the developing stage, while 30 percent describe their organizations as maturing. The researchers surveyed 4,300 managers, executives, and analysts in 123 countries for the study.</p><p>Leadership is a key characteristic of digitally mature organizations. These organizations are "four times more likely to be developing needed digital leaders than the least digitally mature" organizations, the report states.</p><p>Digital transformation requires empowered and collaborative leaders, says David Kiron, executive editor of <em>MIT SMR</em>. Effective digital leaders provide vision, enable experimentation, encourage people to think differently, and facilitate collaboration throughout the organization, the report notes. "That typically requires a new mindset around leadership and learning for all employees," Kiron explains.</p><p>Yet, more advanced organizations aren't complacent. More than half say they need new digital leaders. </p><h2>Productivity Gains Slow</h2><p>Despite their progress, organizations may need a reality check about the immediate benefits of transformation projects. A new <a href="" target="_blank"> <em>McKinsey Quarterly</em> article</a> points out that digitization isn't yet creating productivity growth. The article notes that many companies that are investing in innovation and adapting their businesses "are still trying to understand how to make the most of digital technologies."</p><p>The article cites a McKinsey survey of global companies in which respondents described only a few business activities, products, and services as digitized. The study attributed this to adoption barriers, lag effects, and transition costs. Moreover, organizations undergoing digital transformation report their own digital products and services have cannibalized 17 percent of the market share from existing core products and services.</p><p>Historically, though, a delay in productivity growth is to be expected. For example, U.S. productivity growth lagged during the beginning of the computing revolution in the 1970s and 1980s, until it picked up in the 1990s.</p><p>The article suggests there are unrealized digital opportunities in industries such as automotive, retail, and utilities. It estimates productivity will rise 2 percent annually over the next decade, with most of the new growth resulting from digitization.</p><h2>Digital Characteristics</h2><p>There's still work to be done before organizations can realize those digital opportunities. And much of that work is about people, not technology.</p><p>The <em>MIT SMR</em>/Deloitte Digital report identifies some characteristics of more digitally mature organizations. Most push decision-making down the organization, although not as much as CEOs believe. Although most CEO respondents say this is happening, just one-third of vice presidents and directors agree. However, the researchers say their evidence suggests "employees may be reluctant to step up and assume their roles as digital leaders."</p><p>Another characteristic of digitally mature organizations is a fast, flexible, and distributed workplace, with "a different culture and mindset than traditional businesses." The rapid pace of business is the biggest difference respondents cite about digitalization. That pace forces organizations "to act and respond faster than they ever have before."</p><p>To compete, organizations will need to change how they operate, the report suggests. Respondents say they will need to learn to experiment, take risks, and deal with ambiguity.</p><p>They will need new skills, too — which is turning into a roadblock. Ninety percent of respondents say they need to update skills each year, while half say they must do so continuously. But about one-third say their organizations don't support skill development. Part of the problem may be the organization's reliance on formal training, when employees may learn better on the job.</p><p>This skills issue applies to the organization, itself. Established organizations often assume the factors that led to past success will apply to digitalization. Yet, those success factors may not apply to the changes wrought by today's digital technologies, the report notes. Organizations that don't overcome these "competency traps" may be vulnerable to new competitors.</p><h2>Learning to Try</h2><p>Perhaps the greatest problem digitalization efforts face is the organization's inability or unwillingness to experiment. The report finds that digitally mature organizations are more likely to experiment and make iterative changes. </p><p>Think about technology innovators. The most successful innovators are working on improvements constantly, even before the next version comes out. They are open to disrupting their own products if they invent something better. The mentality is "Try something and see if it works."</p><p>That's not the traditional business mentality, nor one with which leaders may be comfortable. "Traditional companies often struggle with experimentation because fear of failure is part of their organization's culture," says report co-author Gerald Kane, a professor of information systems at Boston College. But they can change by embracing experimentation. </p><p>Specifically, the report says organizations that learn and innovate through experimentation "encourage new ideas to be shared and tested at all levels of the organization." They learn to work in new ways and encourage people to share feedback about failed experiments. </p><h2>Compete Now and in the Future</h2><p>But experiments by themselves aren't the objective. Organizations need to build on experiments to drive change throughout the organization, the report advises. Moreover, they must balance innovation with existing competencies and practices. "Established companies must find ways to experiment to compete in the future, while also maintaining the core business to remain competitive in the present," Kane says. </p>Tim McCollum0
Partners in Protection in Protection<p></p><p>Despite organizations increasing cybersecurity spending by 23 percent last year, successful security breaches rose 27 percent compared to 2016, according to the 2017 Cost of Cyber Crime Study. The joint study by Accenture and the Ponemon Institute is based on interviews with more than 2,100 cybersecurity and IT professionals worldwide. To find out what went wrong, researchers looked at the value organizations gained from nine areas of cybersecurity investments. What they discovered is that organizations are investing in the wrong areas when it comes to cybersecurity and risk. </p><p>Take perimeter security, for example. Advanced perimeter controls are the highest spending category, while being fifth in cost savings. Yet, focusing primarily on perimeter security makes less sense when most companies can’t even define their perimeter in the age of the Internet of Things. Research firm Gartner predicts there will be 20 billion internet-connected devices by 2020, up from 6 billion devices in 2014. </p><p>As the areas where attackers can target continue to expand, organizations need their cybersecurity and internal audit functions to partner to more effectively deploy resources against cyber threats. Cybersecurity teams and executive management can leverage internal audit’s insight into organizational risks to invest in areas that can provide the greatest protective and efficiency value to the business. To build this relationship, both internal audit and cybersecurity professionals will need to change how they do business and collaborate to build cybersecurity and risk management strategies and inform executive management.</p><h2>Hiding Flaws</h2><p>Neither cybersecurity professionals nor internal auditors are wholly innocent when it comes to how they work together. Too often, cybersecurity teams are defensive when it comes to internal audit. They don’t want to look bad in front of their peers and management, so they try to conceal their flaws from auditors. At best, this produces a strained relationship between internal audit and cybersecurity, and at worst, it exposes the business to vulnerabilities and threats. </p><p>Executive management needs clear information about the risks so it can make the best decisions on where to spend resources to enable the business to operate securely. Internal auditors can help cybersecurity professionals provide this information by giving them a second pair of eyes to find security flaws before a malicious user might exploit them. In addition, a strong relationship with auditors can provide the cybersecurity team a broad view of the organization and its risks. Otherwise, the cybersecurity team can lose sight of the organization’s overall risks as it concentrates to protect the business’ systems and assets. Finally, with its access to executive management and the board of directors, internal audit can communicate the severity of risks and their impact to the business when the cybersecurity team cannot get the appropriate visibility. </p><h2>Ignoring Cybersecurity Plans</h2><p>Internal auditors share blame, too. Often, auditors are quick to make independent assessments outside of the cybersecurity team’s plans, which can lead to inappropriate prioritization of risks. Consider this example:</p><p>Bill performs an IT security audit of his business. While planning his audit, he researches the generally accepted frameworks, best practices, and the company’s IT security policies. Bill does not consider the cybersecurity team’s roadmap or plans, which show that the team’s No. 1 priority is to shore up the business’ asset management program.</p><p>During the fieldwork, Bill finds that not all systems have the appropriate security agents installed on them. He reports his finding and a management action plan and date are set. Because the company takes internal audit seriously, that action plan takes priority over the cybersecurity team’s roadmap. </p><p>The problem with this scenario is that if the cybersecurity team is forced to concentrate on agent deployment, it can’t shore up its asset management. That can lead to future issues with agent deployment because the business lacks a clear understanding of its hardware and software assets. Without a clear partnership between internal audit and cybersecurity, the business may overspend and under protect its assets.</p><p>Internal audit, itself, stands to benefit from partnering with the cybersecurity team. Cybersecurity professionals can become deep experts in their field and have access to the latest research from security-focused professional associations. They can give auditors a better understanding of current and upcoming threats to the business and how they interplay with other business risks. </p><p>Auditors also can benefit from learning how the tools and strategies the cybersecurity team has deployed work with each other to build defense in depth. Often, auditors may have a single understanding of how a certain set of controls should be implemented to protect an area of the business. For example, developer access to production historically has been considered a security issue that must be addressed, with clearly defined lines of segregation of duties needed. However, DevOps and continuous release change management are blurring the lines of traditional segregation of duties risks. Today, small, agile teams rapidly create, test, and auto-deploy application code. This would be impossible in traditional segregation-of-duties-based development life cycles. Partnering with the cybersecurity team will help auditors understand the risks this new way of working brings to the business.</p><h2>Team Building</h2><p>A successful collaboration between cybersecurity and internal audit requires two essential ingredients: communication and empathy. Communication should happen at least monthly, and the two functions should conduct a full agenda focused on risk management and cybersecurity threats and plans at least quarterly. The other meetings can be less formal with some emphasis on getting to know people to cultivate empathy.</p><p>Empathy is about walking in someone else’s shoes. There is no better way to do that than to actually do that person’s job. Cross-training employees can help an organization be successful. Because internal audit and cybersecurity have a common concern with risk management, they are a natural fit for job rotations between them. </p><p>Another way to build empathy is to have internal audit and cybersecurity team members pair up to present training sessions at events such as in-house lunch and learns and local conferences. Finally, the two teams can partner to perform the organization’s cyber risk assessments.</p><h2>A Symbiotic Relationship</h2><p>Ultimately, the key byproduct of internal audit’s partnership with the cybersecurity team will be to give management and the board a clear understanding of the cyber risks and opportunities the business faces. That information can enable them to make the best decisions about which security tools to invest in and how and where to deploy those resources. This can’t happen without a symbiotic relationship between auditors and cybersecurity professionals. By gaining a deeper view into the organization’s security risks, internal audit can produce a global assessment of cyber risks and leverage its relationships with executive management and the audit committee to drive effective change to protect the organization.  </p>Cliff Donathan1
A Boost for Cyber Resilience Boost for Cyber Resilience<p>​Large organizations faced twice as many cyberattacks on average last year, an Accenture study notes. Despite their best efforts to ward off ransomware, distributed denial-of-service, and other attacks, organizations experienced an average of 30 breaches, according to the <a href="" target="_blank">2018 State of Cyber Resilience</a> study. </p><p>Clearly, organizations need help. They need a framework.</p><p>Last month, the U.S. National Institute of Standards and Technology (NIST) updated its Framework for Improving Critical Infrastructure Cybersecurity. <a href="" target="_blank">Version 1.1</a> (PDF) clarifies and enhances the framework, which has been adopted by governments and businesses worldwide. Matt Barrett, program manager for the framework, says the revision "applies to a wide range of technology environments, such as information technology, industrial control systems, and the Internet of Things."</p><h2>What's New</h2><p>One addition to the framework is a section on "Self-assessing Cybersecurity Risk With the Framework" aimed at helping organizations understand and measure cybersecurity risk. The section advises that assessing the effectiveness of cybersecurity investments starts with understanding organizational objectives, how they relate to cybersecurity outcomes, and how those outcomes are implemented and managed. </p><p>This section recommends organizations take care in how they apply metrics, and be able to explain how the measures contribute to the organization's cyber risk management. Moreover, it warns against relying "on artificial indicators of current state and progress in improving cybersecurity risk management."</p><p>The revision also expands how the framework can be used to manage cyber risk in the supply chain. Some recent cyberattacks have targeted large organizations by going through their business partners. </p><p>Additionally, cyber supply chain risk is now included in the framework's implementation tiers and the Framework Core now includes a supply chain risk management category. Moreover, a new section on buying decisions discusses how to use the framework to address risk associated with purchasing off-the-shelf products and services.</p><p>Other revisions to the framework include updates on user authentication and identity, and vulnerability disclosures. The framework's terms also have been clarified.</p><p>NIST plans to release a companion to the framework, the Roadmap for Improving Critical Infrastructure Cybersecurity, later this year. That document will cover areas such as development, alignment, and collaboration, which Barrett calls "essential to the framework's success."</p><h2>Faster Responses</h2><p>As the Accenture report findings indicate, the need to strengthen cyber risk management is greater than ever. Still, there are some positive signs. </p><p>The organizations in the study prevented 87 percent of all focused attacks, up from 70 percent in the 2017 report. Accenture defines a focused attack as one with the potential to penetrate network defenses to cause damage or extract high-value assets. </p><p>"Only one in eight focused cyberattacks are getting through, versus one in three last year," says Kelly Bissell, managing director at Accenture Security. Accenture surveyed 4,600 enterprise security professionals from large companies in 15 countries.</p><p>Organizations are also finding security breaches faster. Nearly 90 percent say they detected breaches within one month, compared to 32 percent last year. Most (55 percent) found them within one week.</p><p>There's some bad news, as well: Organizations' information security teams are only finding about two-thirds of security breaches. The remainder they are finding with help from white-hat hackers, peers, and other business and government sources. </p><p>Many respondents say the emergence of new technology tools, including cyber threat analytics, security monitoring, and artificial intelligence, may help them battle threats. "For business leaders who continue to invest in and embrace new technologies," Bissell says, "reaching a sustainable level of cyber resilience could become a reality for many organizations in the next two to three years." </p>Tim McCollum0
Embrace Change or Become Obsolete Change or Become Obsolete<p>​Innovative, disruptive technology represents a key focus for today's organizations. With increasing regularity, we hear about a new technological advancement that will completely change the way businesses, and even internal audit functions, operate. And while some auditors welcome these developments, others shy away from them, often worrying how the technology could affect their work. But we have become accustomed to adapting to the business environment and using it to showcase our value. In fact, adaptation is not just an important part of our work — it's a professional imperative. Internal auditors must embrace and leverage technological innovations, or risk becoming obsolete. </p><p>Neglecting to familiarize ourselves with new technologies impacting organizations will cause us to fall behind and become less relevant to stakeholders. Internal auditors cannot possibly provide meaningful assurance or add value if we don't keep up with the latest developments and factor them into our work. There is no shortage of information available on topics such as artificial intelligence (AI) and blockchain, and there is no excuse for neglecting to research them. Not only do we shortchange our clients by ignoring these areas, but we also cannot make the technologies work for us without first understanding their capabilities and potential applications. </p><p>Ignorance of technological change prevents internal auditors from leveraging innovative tools as multipliers of capacity. While AI will almost certainly eliminate some jobs, the Gartner Research report Predicts 2018: AI and the Future of Work forecasts a net jobs increase due to AI by 2020. Imagine a situation where manual and tedious internal audit tasks are automated, allowing practitioners to focus on driving real value to the organization. While this scenario only scratches the surface of what may be possible with AI, it illustrates the powerful, multiplying effect of using the technology. </p><p>Ultimately, neglecting to grasp and absorb technological change is a disservice to ourselves, the organization, and the profession. The IIA has taken a clear stance on professional development through Standard 1230: "Internal auditors must enhance their knowledge, skills, and other competencies through continuing professional development." There is no better skill to develop than one that will ensure the future relevancy of our profession. </p><p>For internal auditors to genuinely embrace technology and leverage its potential multiplying effect, we must act without fear to understand the possibilities, keep an open mind, and continually evolve. But at the same time, technological advances should never be used to replace our skills — they should augment them. As always, the skills that will set auditors apart in the digital age will be the ability to think critically and communicate clearly. The most successful future audit leaders will be those who can understand and leverage technological change, as well as clearly articulate its potential impact to stakeholders.</p>Seth Peterson1
Out of Step With Analytics of Step With Analytics<p>​Internal audit departments still are not widely using data analytics and other technology tools that could massively impact the work auditors do and their value to organizations, according to recent reports on the profession.</p><p>These reports warn that failing to adopt such "foundational" tools may make internal audit obsolete and unprepared to address the opportunities and risks associated with technologies such as artificial intelligence (AI) and robotics. Internal audit's failure to use data analytics more extensively also may impact corporate strategy and competitiveness as company information is not being mined effectively to inform management decision-making.</p><p>Understanding where data resides and uncovering patterns and insights to enhance decision-making is increasingly critical to business success. Additionally, experts say the appropriate use of data and data analytics is equally important for internal audit's effectiveness and value to an organization.</p><p>Yet, reports such as The IIA Audit Executive Center's <a href="" target="_blank">2018 North American Pulse of Internal Audit</a> describe many internal audit departments' use of data analytics as developing in maturity, at best. While nearly one in three of the survey's 636 respondents say they use simple analytics techniques extensively, few are automating routine tasks or adapting more advanced techniques.  </p><h2>Arrested Development</h2><p>Many internal audit departments are still struggling to develop a formal methodology for integrating data analytics, according to a survey of more than 1,500 chief audit executives by global consulting firm Protiviti. Moreover, audit functions are only using analytics tools as "point solutions" on a case-by-case basis, rather than as part of a broader initiative to leverage analytics throughout the audit process. </p><p>Protiviti's <a href="" target="_blank">2018 Internal Audit Capabilities and Needs Survey</a> notes that while two-thirds (66 percent) of internal audit functions that do not currently use data analytics plan to do so as part of the audit process within the next two years, one-third (34 percent) still have no plans to do so. For those departments that are implementing the technology, data analytics "allows internal audit to provide better and more detailed information to inform corporate strategy and for management to leverage business opportunities," says Brian Christensen, executive vice president, global internal audit at Protiviti. </p><p>One barrier to realizing these benefits is a lack of analytics knowledge and skills within the audit function. "CAEs need to focus on increasing the levels of education in their internal audit functions, and more specifically, to move from general plans and discussions about using analytics to actually advancing and integrating analytics, robotic process automation, and other digital initiatives into the audit plan," Christensen says. "Those who fail to integrate these initiatives risk becoming obsolete as their organizations continue to undergo digital transformation at an increasingly rapid pace."</p><p>Protiviti's research also finds that U.S. internal audit functions have been slower to adopt the technology than their counterparts in other parts of the world. Three-fourths (76 percent) of organizations in Europe and the Asia-Pacific region are using data analytics in the audit process more frequently, compared to only 63 percent from North America. </p><h2>Evolving, Following, or Observing?</h2><p>Results from PwC's latest <a href="" target="_blank">State of the Internal Audit Profession</a> report deliver more pessimistic results. Just 18 percent of respondents say their internal audit function currently uses analytics for advanced testing procedures — 38 percent plan to do so within two years. Only 13 percent say internal audit uses analytics to identify risk and determine audit scope and planning, but 30 percent plan to do so within the next two years. A mere 10 percent say internal audit has adopted tools to help with analytic visualization, and 27 percent plan to do this by 2020.</p><p>CAEs are aware of the problem. Most internal audit leaders surveyed (56 percent) say they are concerned that lack of technology adoption will result in diminishing value for their organization. </p><p>In fact, PwC deemed only 14 percent of internal audit functions surveyed as "advanced" in their technology adoption. PwC refers to these functions as "evolvers" (as opposed to "followers," which adopt new technologies at a slower pace, and "observers," which are constrained by lack of budget and technical knowledge). More than 80 percent of evolvers are self-sufficient in their data extraction, and use tools and skills for enhanced productivity. </p><p>Furthermore, evolvers are more likely to invest in technology risk management and IT training than their peers. As a result, they are rated more valuable to their organization. For example, twice as many evolvers than their peers report that their organizations' risk management programs respond to innovation very effectively. </p><p>Evolvers are realizing direct value from their adoption of analytics. For instance, they rate high on focusing on their organizations' critical risks and on auditing emerging risks. And tech-savvy audit functions benefit in other ways, too. Nearly three-fourths of evolvers excel at recruiting and training the talent they need because they are seen to invest more resources in people and training, compared to 46 percent of followers and 29 percent of observers.  </p><p>Lauren Massey, principal in PwC's internal audit, compliance, and risk management practice, says data analytics has been a topic of discussion in the profession for several decades, yet adoption continues to be slow. As a result, those internal audit departments that fail to take up analytics will be at a disadvantage as new technologies emerge. "If internal audit functions are unable to embrace the benefits that analytics has to offer, or cannot find the resources to train themselves in how to use it," she says, "there will be the constant challenge for internal audit to get up to speed with cutting-edge technologies like robotics and AI quickly."</p><h2>Making Up Ground</h2><p>Despite such warnings, it is not too late for internal audit functions to turn the situation around. Protiviti's report outlines several actions CAEs can take to improve their department's analytics capabilities. </p><p>For departments that are just beginning to use analytics, the easiest way to become familiar with the technology is to start in more familiar areas such as account reconciliations, journal entries, payables, fixed assets, payroll, human resources, and threshold/limit controls. "The internal audit function may find it easier to test data based on information it already knows," Christensen says.</p><p>CAEs also should find champions to lead and support the analytics effort. Protiviti notes that 59 percent of respondents agree that when internal audit shares detailed information about analytics with the audit committee, committee members also are highly interested in the use of audit analytics. </p><p>Other ways to increase the use of data analytics tools and techniques include embedding analytics as part of the audit process and expanding internal audit's access to quality data. Moreover, internal audit should find ways to measure and report to management and other stakeholders the successes directly associated with the technology's use. </p><p>"Internal audit groups that can successfully demonstrate tangible value will build a stronger business case for increased budgets and resources dedicated to a data analytics function, as well as underscore throughout the organization the importance of analytics and, in the process, boost internal audit's reputation internally," the Protiviti report says. </p>Neil Hodge0
Behind the Data the Data<p>​Businesses are having a love affair with data analytics. The potential to unlock secrets hidden in the vast quantities of data generated daily makes the technology almost irresistible. And why not? Tools enabling the organization to uncover data patterns that reveal how to implement efficiencies, make better decisions, increase agility, identify untapped market niches, and appeal more viscerally to customers can be extremely valuable. </p><p>Internal audit is no stranger to using data analytics to fulfill its responsibilities to the organization. But not only does internal audit use data analytics itself, it also is called on to review the data analytics use of the business units. Such audits are performed because of the growing realization that insights are not alone, hiding in the data; risk lies there as well. And where there is risk, there is a need for internal audit.</p><p>"The same types of questions we would consider for other processes in terms of where things could go wrong apply to data as well," says Judi Gonsalves, senior vice president and manager, Corporate Internal Audit, with Liberty Mutual Insurance Group in Boston. And with ever-growing volumes of data on hand, and further organizational dependency on that data, those questions become more and more important to ask. </p><h2>Assessing the Risks</h2><p>The possibility of things going wrong explains why internal audit should start, if it has not already, reviewing the use of data analytics in the organization. More than 70 percent of chief audit executives (CAEs) surveyed in The IIA Audit Executive Center's 2018 North American Pulse of Internal Audit research indicate that their organization's net residual data analytics risks are "moderate" to "extensive." But what, exactly, are those risks?</p><p>A risk cited by several experts can be summed up in the familiar phrase, "garbage in, garbage out." If the data being analyzed is inaccurate, incomplete, unorganized, dated, or siloed, the conclusions drawn from it can hardly serve as the basis for a winning business plan. "We worry most about the completeness and accuracy of the data pulled together and upon which management may rely," notes Katie Shellabarger, CAE with automotive dealer software and digital marketing firm CDK Global in suburban Chicago. "Management may take the information prima facie and not know that the data is wrong."</p><p>Tom Rudenko, CAE with online business directory provider Yelp Inc. in San Francisco, echoes this concern about data quality. "Our audits evaluate the risks around the completeness, accuracy, integrity, and security of data," Rudenko says. "For example, if a data warehouse is part of the data analytics process, we look at risks and controls around the entire path of the data: the sources of the raw data, the methods and technology around transferring the data to the warehouse, the controls over the warehouse, and the transfer to the end user." Rudenko explains that, in this example, if there are errors or problems with the data at any point along this path, then the end result may be flawed and any decisions or conclusions relying on this data may also be flawed. "If there are any weak links along the journey to the end user, then the entire chain may break," he adds. </p><p>Alternatively, the data may be sound, but the algorithms used to analyze it flawed. They may contain an ancillary function, such as an edit check, that is doing something other than its intended purpose, without the business unit being aware. This anomaly may not influence the result. But then again, it might. </p><p>In addition, questions should be asked about the data collection process itself. Was it ethical? Is the data being used for the purpose for which it was collected? Was it collected in a way to provide objective results or to prove a point?</p><p>"We have to be careful of bias in how we, as auditors, test," says Charles Windeknecht, vice president of Internal Audit with Atlas Air Worldwide in Purchase, N.Y. "We cannot let our initial impressions drive our subsequent actions. If we are unduly influenced by an early fact, we may go down an incorrect path, getting a result that appears accurate while not realizing we are unintentionally overlooking other data."</p><table class="ms-rteTable-default" width="100%" cellspacing="0"><tbody><tr><td class="ms-rteTable-default" style="width:100%;">​<p><strong>Getting Started </strong></p><p>CAEs and internal auditors just beginning to audit the organization's use of data analytics may welcome some words of wisdom to ensure favorable results. The experts offer several suggestions:</p><ul><li>Consider the advantages and drawbacks to building analytics capability in the existing team versus acquiring talent.<br></li><li>Engage with management, especially in the planning process. "If they are not involved, the process may get started, but it is less likely to be sustainable," Rudenko says. <br></li><li>Start small. Understand the process and break it into manageable, auditable parts. <br></li><li>Have realistic expectations. While the internal audit function may hope to spring from level 1 to level 4 with regard to its ability to use data analytics effectively in the audit process, the reality is that it takes a lot of effort just to go to level 2. The level of internal audit's understanding and capacity to use data analytics does influence how to effectively audit a control process with heavy reliance on similar routines.<br></li><li>Take the time to work through the false positives that are likely to arise during the initial execution of the audit testing routines. <br></li><li>Look for a win. "Start by auditing candidates, or processes, where you are likely to gain success," Windeknecht advises, "then build on that success."<br></li><li>Look to local IIA chapters for shared experience/expertise and libraries of data analytics routines and audits of data-analytics-driven control processes. Some have formed discussion groups specific to data analytics.<br></li><li>Have the end game in mind. "Know who is relying on the data and what they are using it for," counsels Robert Berry, executive director of Internal Audit at the University of South Alabama.<br></li></ul></td></tr></tbody></table><p>Other risks related to data analytics are many and varied. The more data the organization has, the more incentive it may provide malicious actors to hack into it, thus compromising security and privacy. In addition, change management techniques and monitoring/maintenance of who has access to the data are causes for internal audit attention. </p><h2>Proven Methodologies</h2><p>When faced with a diverse and complex range of risks, tried and tested audit approaches often yield the best results. Take, for example, the timing of data analytics-related audits. Windeknecht indicates that his team's audits are generally driven by the annual plan, which is updated quarterly. "However, if there's a process that's identified as risk-driven, such as analytics, we will audit that process and test those controls as an addition or replacement to the formal plan." </p><p>Often, the timing of data analytics reviews depends on the nature of the data. "If the data is critical to the production of our financial statements, then it gets reviewed as part of the ongoing Sarbanes-Oxley process," Rudenko says. "If the data relates to operational, technical, or regulatory risks, the frequency of our reviews is factored into our audit planning process."</p><p>But scheduling is not the only area where established practices can prove beneficial to review of analytics use. The techniques used to conduct the audit can be relatively standard as well. For example, Robert Berry, executive director of Internal Audit at the University of South Alabama in Mobile, asks the department he is auditing what reports it generates. "Depending on the source of the data and how it is used, we may need to look at it, because management may be making critical decisions based on it," he says. Berry's team relies on a structured approach to audit the data analytics process and reuses approaches that have worked well in one department for other departments.</p><p>A traditional approach applies also to the controls recommended to address any findings: input controls (the data's completeness, accuracy, and reliability), processing controls (reconciliation of changes made to normalize/filter the data), and output controls (accuracy, based on inputs and processes). Consider, for example, the data warehouse, which supports data analytics. It has teams of personnel dedicated to operating and maintaining it, and features pipelines from the sources of data to the warehouse and from the warehouse to the end users. In this scenario, Rudenko suggests assessing whether or not:</p><ul><li>Personnel have the necessary expertise to ensure the completeness, accuracy, integrity, and security of the data.<br></li><li>Processes and controls surrounding the use and security of data are clearly documented and communicated.<br></li><li>Appropriate and relevant access and change management controls are in place and tested for operating and design effectiveness.<br></li><li>Changes to the control environment and supporting databases are tracked and monitored.<br></li><li>The analyses are supported by built-in quality and effectiveness checks to ensure they (and the data) mirror the changes and evolution of the business. </li></ul><p><br></p><p>Personnel-related controls are critical in relation to data analytics, particularly management oversight and user education. Shellabarger points out that if users have flexibility to create their own reports/analysis, they need to know how to use the tools correctly and how to evaluate the inputs and outputs. "Essentially, they need to be able to address the completeness and accuracy issues related to using data and tools," she says.  </p><h2>The Finer Points</h2><p>While proven methodologies may come into play throughout the process of auditing the business units' data analytics use, that does not mean such audits do not present their own unique challenges. As with every audit, there are subtleties that must be recognized, understood, and resolved. </p><p>For example, Windeknecht points out that even the apparently basic exercise of identifying data analytics is far from straightforward. "What do we define as data analytics?" he asks rhetorically. "Business units are doing analyses in different shapes and forms, using different algorithms and basing their analyses on different assumptions." Risks can arise when the internal auditor or the business unit itself incompletely or incorrectly understands or agrees on such foundational issues. "Are the assumptions still valid?" he continues. "How do you perform integrity checks? When was the most recent review of the algorithm? How does one data event influence subsequent activity?"</p><p>Internal auditors make a big mistake if they do not validate key assumptions with facts (i.e., confirmation of key data points and the underlying assumptions) before continuing with testing. "I've seen audit teams reach completely inaccurate conclusions because they went down the wrong path early in testing," Windeknecht says. "The root cause for the error was not sufficiently validating assumptions and initial results. The issue is a huge hit to the integrity of the testing and audit process.<br> The issue is not one you want to confront during the reporting phase of the audit." </p><p>Berry points to challenges even in knowing exactly what to audit. He explains, "On a micro level, when you look at a specific department, you have to understand the objectives of the deliverables/reports, the sources of the data, and the distribution of the data." It is important to review the process undertaken to produce reports: how the data changes through the cycle and how the changes are accounted for. He advises framing the audit around "reconciling base data to final output." </p><p>On a macro level, it is important to prioritize. "Every department has data it is analyzing and using to produce a result, every department has goals and objectives, and every department has to report on how it performs against those goals," Berry says. "You have to work with the departments to identify reports used in management's decision-making process. That will help you know which activities to review and why."</p><p>And, finally, even the most thorough, meticulous audit will fail if its findings cannot be explained in a way that resonates with the business unit that has been audited. Internal auditors must consider the learning modalities of their audit clients when discussing the findings; people hear, see, and experience things differently. While the natural inclination may be to simply hand over a written, text-heavy report, it may be more effective to use visually appealing, concise images in support of the text. A verbal presentation — in support of the written report — that includes concrete examples of the findings or the risks that may accompany the findings is also likely to make a more lasting impression. This gives clients multiple ways to absorb and understand the recommendations, based on the way they process information.</p><h2>Mind the Details </h2><p>The old saying that "the devil is in the details" is particularly apt for reviewing data analytics. And, as with many aspects of internal auditing, a dose of healthy skepticism is helpful. Says Gonsalves: "We cannot assume that just because information comes out of a system, it is automatically correct." </p>Jane Seago1

  • SCCE2018_August2018_Premium 1
  • IIA FSACACGABookstore_August2018_Premium 2
  • IIA EHS2018_August 2018_Premium 3