Thank You!

You are attempting to access subscriber-restricted content.

Are You Ready to Experience Everything Internal Auditor (Ia) Has to Offer?

​Auditing Culture: Conducting an Entitywide Survey

Internal auditors can gain valuable insight on culture by developing an employee survey, administering it, and analyzing the results. 

Comments Views

​Executives and board members want to understand their organization's culture. But where does culture exist? Codes of ethics and value statements express the desired culture, but they may or may not reflect the real culture. And while executives and boards have some insight into the real culture, it is usually limited by an upward filtering mechanism — employees don't want to give bad news to their supervisors so they present information in the best possible light. Their supervisors take what they heard and present it in an even better light to their managers, and so forth up the chain of command.

The real culture exists within the perception of employees. Whatever they believe the culture to be is in fact what it is, because their beliefs shape their behavior and decisions. A well-designed and administered entitywide employee survey can be an accurate measure of these perceptions, and it is therefore the most efficient way to gain an impression, though not necessarily a full view, of the actual culture.

Many organizations use an entitywide survey to gain cultural insight. In organizations that do not use this tool, an opportunity exists for internal audit to add substantial value by conducting the survey themselves. Navigating the process requires an understanding of benefits and challenges, as well as how to develop the survey, administer it, and analyze the results.

Management Support

Internal auditors will likely confront numerous challenges when looking to create and administer an entitywide survey (see "Auditing Culture: Audit Project Surveys" for a detailed discussion of challenges and recommended solutions). Perhaps chief among them, however, is getting support from management. When experiencing lack of support, internal auditors can cite several advantages of conducting the survey to help advance their efforts. 

Preventing Scandals Examples of highly publicized scandals caused by cultural issues might be persuasive. Especially compelling are examples from one's own industry. For instance, when an audit department in the utility industry performed a review of executive behavior, it included this statement in its report: "Recently, peer utility companies have tarnished their reputations as a result of allegations of unethical and inappropriate behavior." The report then identified the companies and behaviors. Audit committee members wanted assurance that their organization was not at risk for something similar.

Providing Executives Information They Can't Get Any Other Way Because of the upward filtering mechanism discussed earlier, executives often don't receive an accurate picture of the organization's culture. For example, a colleague of mine once worked for a company whose CEO's misjudgment of culture led to a costly mistake. When it looked like the company's quarterly financial results might not match Wall Street analysts' expectations, he asked several subsidiary leaders if they could do anything to help — but he didn't specify "ethically." He assumed they would understand that's what he meant. Most did, but some didn't. The results were big headlines, a fine from the U.S. Securities and Exchange Commission, and a significant drop in the company's stock price. 

This is one example of a common problem. Good executives try to send the right message, but for one reason or another it is sometimes misunderstood. A survey can be the feedback tool (i.e., monitoring control) that enables them to identify when and why the tone at the top they are setting is not what they intend.

Improving Culture The survey sends a message that the issues it addresses are important, as long as improvements are made as a result. When employees see that management takes these issues seriously, they are more likely to do so as well. 

Reassuring Stakeholders An effective survey demonstrates to stakeholders that the culture is basically sound and continually getting better as action plans are implemented. As long as employees feel safe being honest and believe their concerns will be addressed, it is highly likely that a significant problem in the culture will be identified. 

Leveraging Internal Audit's Expertise Internal auditors can and have developed effective surveys. One example used by numerous organizations can be found here. Internal auditors' broad organizational purview and analytical skills make them well qualified to lead the survey effort.

Consider the comments of Sharon Watkins, the whistleblower at former energy giant Enron, from a conference presentation. She explained that, about six months before the company's massive accounting scandal became public, Enron board chair Kenneth Lay personally sponsored an entitywide survey. Not only Watkins but about half a dozen others she knew of used the survey to express their concerns over the company's financial reporting. She cited the value of this type of survey, but also mentioned something else to consider — the survey was administered by human resources, and the only criterion it used to report results was how many people said the same thing. Because only a small number wrote comments implying Enron's financial statements were fraudulent, nothing was reported. Watkins' point was that professionals such as internal auditors, who can do a qualitative review and follow up on a small number of potentially meaningful comments, should be involved in analyzing the results. She also said that the survey identified what would eventually bring the company down. 

Survey Development

Developing an effective survey is a skill unto itself. These guidelines should at least provide a useful starting point.

  • Use good models. In addition to the Entitywide Survey Example, many other good models can be found on the internet. Also, "Examples of Survey Statements on Cultural Topics" (PDF) has a selection of statements drawn from various surveys and "Auditing Culture: Employee Surveys" features a list of suggested culture topics that might be addressed in a survey.
  • Work with stakeholders and second-line functions. Audit committee members, executives, and even some mid-level managers might have issues of concern that the survey could help assess. Second-line functions (risk management, compliance, etc.) are very likely to have issues of concern and may be able to share information that internal audit can build on. Interviews and focus groups are effective techniques for gathering input.
  • Phrase survey statements clearly and simply. Complex words and sentences inhibit readability, will reduce the response rate, and might generate misleading responses if misunderstood.
  • Provide clear instructions. For example, a survey statement that asks something specific about "management," without specifying the level of management, can be confusing.
  • Ask for a range of responses. Cultural issues can rarely be captured by yes/no survey responses. The most common and usually best answer scale is the Likert scale, in which respondents indicate the extent to which they agree or disagree with statements (e.g., Strongly Agree, Agree, Disagree, Strongly Disagree, and Don't Know). Some Likert scales have a neutral option in the middle of the scale, but many users find it better to force respondents to agree or disagree, unless they just don't know. The responses can then be tabulated on a four-point scale, with Don't Know (or N/A) omitted from the tabulation. 
  • Ask for comments at appropriate intervals. To identify the specific issues that cause employees to agree or disagree with statements, the survey needs to include a mechanism for following up on responses. Explanatory comments from respondents can be invaluable in pointing internal auditors in the right direction. Asking for comments just once at the end of a lengthy survey, however, is unlikely to illicit much information — employees won't remember why they responded as they did to statements early in the survey. Auditors could instead divide the survey into sections, each focused on a particular topic, and ask for comments at the end of each section. 
  • Field test the statements to identify ambiguity. It is hard to write statements with no ambiguity. Auditors can identify ambiguity by giving the survey to several people — first within the audit department, then to others who are willing to participate — and asking them to note any statements that are confusing or for which they didn't understand the purpose. 
  • Balance the number of issues addressed against survey length. A survey with too few statements won't yield much information, and including too many statements will drive down the response rate. A good starting point might be to aim for 30 to 50 statements, but the optimal number will differ by organization. Some trial and error might be needed to determine what works best. 
  • Balance confidentiality against the ability to follow up. Knowing where negative responses and comments came from helps identify their root cause. But asking for detailed demographics can inhibit candid responses from employees, for fear of discovery. The key is to find the right balance with demographics that enable investigation of potential issues without compromising confidentiality. One interesting example from a small organization is a survey with check boxes for "No one reports to me," "Fewer than 20 people directly or indirectly report to me," or "More than 20 people directly or indirectly report to me." A larger organization was able to ask each respondent to identify his or her location, years with the organization, and grade level without being able to identify individual respondents. The level of confidentiality needed will vary by organization. In addition, most survey platforms have the ability to identify each respondent's IP address — internal auditors should disable this feature and communicate to employees that their IP address will not be tracked. 
  • Consider legal review. Legal departments don't want survey results to create liability. It is better to get their input early and address any concerns than to have them find out about the survey when they get the email inviting them to take it.
  • Get approval. Once the survey form and the specifics for administering, analyzing, and reporting are finalized, auditors should get approval from executives and the audit committee.

Survey Administration

In addition to developing the survey instrument, there are several decisions to make with regard to administration. The following guidelines should be kept in mind. 

  • Consider asking a trusted individual to sign the invitation. Internal auditors should ensure the invitation letter is signed by the person most likely to elicit a good response rate. In one organization, the auditors reviewed their survey with the Global Risk Committee (GRC) to get committee members' perceptions of the issues addressed. They then asked the head of the GRC to sign the invitation letter. Because the GRC's members were the most senior executives in the organization, the committee head's signature helped boost survey response rate. 
  • Track response rate. If the response rate is low or getting lower year to year, auditors should find out why and change the survey process as needed.
  • Tailor to your organization. Most of the process variables (e.g., length of the survey, level of confidentiality) will vary by organization and therefore should be adapted to meet specific needs. One public sector audit department that audits agencies with known management problems — where employees might have a high fear of retribution — conducts its surveys offline. The auditors gather employees in an auditorium with no managers present, explain the process to them, and ask them to complete the survey in hard copy form. When done, they ask the employees to pass the forms to the end of their row so the auditors can't see who completed which form. Management is usually resistant when presented with the results, but in many cases they address the issues and the improvements they see help them view the process in a much more positive light. 

Survey Analysis

Once the survey responses are tabulated, they need to be analyzed. Those performing the analysis should consider several recommended practices.

  • Analyze responses by level, geographic region, or other demographics. Stratifying the results by level in the organization can be eye-opening for those at higher levels if they have been given an unrealistically positive picture of the culture. Using demographics to identify the source of possible issues as precisely as possible vastly improves the ability to investigate them. As noted earlier, the precision will be limited by the need to preserve confidentiality. 
  • Substantiate issues. As the Enron example above illustrates, even a small number of comments may be meaningful. On the other hand, a small segment of respondents may just be misinformed or disgruntled. Auditors should look for physical, documentary to analytical evidence, or conduct interviews or focus groups to determine the validity of the responses. There may even be cases in which a large percentage of employees have a negative perception that is not supported by the facts. It is actually a misperception. In these cases, auditors should share the information with management, which can then correct the misperception and improve morale. 
  • Look for best practices. Although internal auditors' main goal will be to identify weaknesses in the culture, the process may identify strengths as well. These can be strengths of the organization as a whole, or if the demographics allow it, areas that are particularly strong in some way. Finding the root cause of the positive results might yield best practices management could share with others.

Using the Survey Results

An excellent cultural survey will be a waste of resources if it does not lead to positive actions. This primarily involves action plans to improve the culture, which can also improve internal audit's work. 

  • Report to executives and the audit committee; track trends over time. One audit department has a survey with five sections, corresponding to The Committee of Sponsoring Organizations of the Treadway Commission's five components of internal control. It aggregates the ratings in each section and produces a bar chart showing the level and the four-year trend of each. The audit committee considers this chart one of the most valuable pieces of information it gets from internal audit each year. It serves as a key risk/early warning indicator for the committee and a barometer for when something is happening that the committee needs to be concerned with. 
  • Publicize action plans. If the survey identifies significant opportunities for improvement but employees don't see any improvements made, they may not bother to respond to future surveys. Worse, their opinion of management may be lowered. When action plans are implemented, employees should hear about it. When they can't be implemented, employees should be told why.   
  • Use the results for global risk assessment and audit planning. If an organizational area is significantly low on some measures of culture, that is certainly a key risk factor that should increase its overall risk rating. If the organization in general rates low on certain aspects of culture, there might be an opportunity for a valuable entitywide thematic audit.
  • Use the results to plan and scope audit and consulting projects. If the demographics allow it, the survey results for an auditable entity could be meaningful in planning a project.
  • Use the results to support audit issues. The root cause of exceptions, for example, might be a cultural issue identified by the survey.

A Variety of Tools

An effective entitywide cultural survey could be one of internal audit's most valuable contributions to the health of the organization. But of course, no single tool or technique alone is sufficient for assessing culture. Auditors need to complement surveys with their own observations, available data that reflects the culture, interviews, and whatever other tools might be useful. The goal is to continually increase stakeholders' understanding of the culture. An well-planned survey can contribute significantly to that understanding.

James Roth
Internal Auditor is pleased to provide you an opportunity to share your thoughts about the articles posted on this site. Some comments may be reprinted elsewhere, online or offline. We encourage lively, open discussion and only ask that you refrain from personal comments and remarks that are off topic. Internal Auditor reserves the right to remove comments.

About the Author

 

 

James RothJames Roth<p>​James Roth, PhD, CIA, CCSA, CRMA, is president of AuditTrends LLC in Hastings, Minn., and author of <em>Best Practices: Evaluating the Corporate Culture</em>. He received <em>Internal Auditor</em>’s John B. Thurston award for the article, “How to Audit Culture” and conducts the seminar, Auditing Culture: Challenges and Proven Techniques for The IIA. </p>https://iaonline.theiia.org/authors/Pages/James-Roth.aspx

 

Comment on this article

comments powered by Disqus
  • PwC-October-2021-Premium-1
  • FastPath-October-2021-Premium-2
  • AuditBoard-October-2021-Premium-3

 

 

Stopwatch Auditinghttps://iaonline.theiia.org/blogs/jacka/2021/Pages/Stopwatch-Auditing.aspxStopwatch Auditing
Thanks, We Already Know Thathttps://iaonline.theiia.org/blogs/jacka/2020/Pages/Thanks-We-Already-Know-That.aspxThanks, We Already Know That
Hidden Goalshttps://iaonline.theiia.org/blogs/jacka/2021/Pages/Hidden-Goals.aspxHidden Goals
Building a Better Auditor: Which Way Should I Go?https://iaonline.theiia.org/blogs/Your-Voices/2021/Pages/Building-a-Better-Auditor-Which-Way-Should-I-Go.aspxBuilding a Better Auditor: Which Way Should I Go?