Thank You!

You are attempting to access subscriber-restricted content.

Are You Ready to Experience Everything Internal Auditor (Ia) Has to Offer?

​Auditing Culture: Audit Project Surveys

Responses to employee questionnaires can yield valuable insights on the cultural environment.

Comments Views

​Internal auditors looking to gauge organizational culture can choose from a variety of assessment techniques. Some are innovative, robust, and resource-intensive, while others are fairly simple. Typically, using a combination of techniques provides a more well-rounded picture of the culture.

Some of the most commonly used assessment techniques include:

  • Entitywide employee surveys.
  • Open-ended interviews.
  • Structured interviews, in which a sample of employees is asked the same set of questions.
  • Combining objective data with auditors' perceptions.
  • Focus groups.
  • Self-assessment workshops.
  • In-depth root cause analysis.

One of the simplest tools for auditing culture is an audit project survey — a survey conducted during the course of an audit engagement. There are several advantages to using a survey tool, as well as limitations and challenges that should be considered. Armed with this knowledge, and familiarity with suggested development and implementation practices, auditors may be better positioned to harness audit project surveys as a means of gaining valuable insight on organizational culture.

Advantages

Employee surveys have several advantages over other techniques for evaluating culture, including:

  • Anonymity. If employees know survey results will remain anonymous, they may be more candid than they would in an interview.
  • Potentially Greater Validity. If employees feel safe and believe action will be taken to address their concerns, surveys usually constitute an accurate measure of employee perceptions.
  • Quantitative Results. Most employee surveys I have seen ask respondents to indicate the extent to which they agree or disagree with statements (see, for example, the "University of Minnesota Employee Survey" below). The percentage of employees who disagree or strongly disagree with a statement is an objective fact, and significant disagreement represents strong evidence that something needs to be examined.
  • Efficiency. Audit project surveys provide an efficient way of gathering input from a large sample of employees. Effective project surveys often yield a response rate of 60-70%, and online survey tools make aggregating and analyzing the responses relatively easy. Unless the audited area is unusually small, interviewing and analyzing responses from a comparable percentage of employees would be prohibitively time-consuming.

Challenges and Drawbacks

While the advantages of employee surveys are considerable, internal auditors should be aware of several potential drawbacks. Recommendations for addressing these limitations are also provided.

  • Possible Lack of Candor. Employees may not be candid, in which case positive results will produce false assurance. Although surveys can be anonymous, employees might not believe they are. And if employees fear retribution from their manager, responses are likely to be positive regardless of how they really feel.
  • Potential Blind Spots. Employees may have blind spots about cultural issues, which can affect their assessments. An often used definition of culture is "how we do things around here." When someone joins an organization, he or she wants to fit in and may accept the way things are done without question. Similar to a lack of candor, this will produce false assurance.
    Recommendation. To address both lack of candor and cultural blind spots, auditors should avoid relying solely on survey results. Some people will be more candid in an interview than on a survey. For example, I think of an objection I received when discussing entitywide surveys at a conference in the Pacific Rim. An attendee who worked for a U.S. multinational company that used this type of survey said, "Surveys don't work here. People in this country will never be honest on a survey. They'll tell us exactly what's going on but they would never write it down." I now tell this story when I teach in that country, and the attendees always agree.
    No single tool or technique is sufficient. Auditors need to be aware of limitations that exist in a given location and complement surveys with their own observations, available data that reflects the culture, interviews, and whatever other tools might be useful in that context.
  • Employee Misperceptions. Although surveys can be an accurate measure of employee perceptions, employees can be wrong. I think, for example, of a lead auditor who worked for me when I was an audit manager. She would occasionally come into my office, ask to close the door, and say, "What are you managers thinking? Do you have any idea what the staff is saying about this decision you made two weeks ago?" I'd say, "But Pam, they don't understand why we made that decision," and realize that we needed to tell them. Pam did a great service by alerting us to the staff's misperceptions, which we could then correct.
    Recommendation. Auditors should not present negative survey results as an issue unless they find corroborating evidence. However, if they can't find such evidence, or what they find contradicts the survey results, they should report it to local management as a possible misunderstanding it might want to correct.
  • Ambiguity. Developing survey statements that are clear and unambiguous can be difficult. Take, for example, the statement, "Management is ethical, fair, and open to employee suggestions." This statement asks about three different qualities. A manager might have one or two of these qualities, but not the third. Also, does "management" refer to the employee's immediate supervisor, the head of the organization, or something in-between?
    Recommendation. Auditors can use a couple of methods to prevent survey statement ambiguity. First, they can draw from good models. Examples of effective surveys can be found in internal audit literature, obtained from colleagues, and accessed on the internet. With established models, any initial ambiguity is likely to have already been identified and corrected. Moreover, auditors will be able to approach prewritten survey statements more objectively, and identify any residual ambiguity more easily, compared to statements written by themselves.
    Auditors can also field-test the survey once it's been developed. Before finalizing the survey instrument, they can give it to several people and ask what they thought each statement was asking. This exercise should identify most or all remaining ambiguity.
  • Scope Limitations. Surveys are limited to the predefined issues they include. And obviously, culture encompasses much more than a brief survey can assess.
    Recommendation. Internal auditors can address this concern by asking survey participants for explanatory comments. The University of Minnesota Employee Survey below has only 12 statements, but it asks respondents, "Would you like to tell us anything else about the operations of your (college, department, center, or other term as appropriate)?" Respondents can elaborate on any of the 12 statements or include something else they want the auditors to consider.

Development, Implementation, and Analysis

Audit project surveys should be adjusted to best fit the environment in which they will be applied. Several considerations should be kept in mind when tailoring a survey for use with a particular client or organization, and during survey implementation and analysis.

  • Design the survey carefully. Provide clear instructions for completing the survey, and phrase statements carefully using simple, easy-to-understand language.
  • Ask for level of agreement/disagreement with statements — such as those shown in the University of Minnesota Employee Survey's Likert scale below — and for explanatory comments.
  • Ask managers if they want to add issues they're concerned about. Good managers often wonder what their employees really think about certain decisions they've made or aspects of the environment. This is their chance to get honest feedback that employees might not want to give them in person.
  • If the content might be highly sensitive, consider asking the legal department to review the survey instrument. The lawyers are less likely to object if they are consulted up front than if they see the survey once it's underway. And they might have legitimate concerns.
  • To demonstrate management's support, ask the head of the audited area, as well as the chief audit executive, to sign the survey invitation email.
  • Consider using online survey tools to survey 100% of the population and to facilitate results analysis.
  • Stratify responses by level — for example, senior management, middle management, staff — and compare the differing perceptions.
  • Remember that surveys measure employee perceptions; they must be substantiated to be reported as audit issues. If they can't be substantiated, they still provide valuable information for the manager.
  • Involve the "experts" in interpreting the results. Some audit departments review the stratified results with a focus group of experienced employees who know better than the auditors why employees responded as they did. The confidentiality of individuals' comments, of course, must be preserved.

Regardless of the technique or combination of techniques used, auditors and their stakeholders must keep in mind the objective of culture auditing: to continually enrich stakeholders' understanding of the culture through a blend of qualitative and quantitative evidence; the objective is not to reach final conclusions. Without this shared understanding, internal auditors risk giving false assurance when assessment results are positive and assigning unfair blame when results are negative.

An Important Tool

Project audit surveys can provide key insight on organizational culture. Like other tools used for this purpose, they will not be effective in every situation. But when applied with discretion and in conjunction with other techniques, they can be a valuable asset in the culture auditor's toolbox.


James Roth
Internal Auditor is pleased to provide you an opportunity to share your thoughts about the articles posted on this site. Some comments may be reprinted elsewhere, online or offline. We encourage lively, open discussion and only ask that you refrain from personal comments and remarks that are off topic. Internal Auditor reserves the right to remove comments.

About the Author

 

 

James RothJames Roth<p>​James Roth, PhD, CIA, CCSA, CRMA, is president of AuditTrends LLC in Hastings, Minn., and author of <em>Best Practices: Evaluating the Corporate Culture</em>. He received <em>Internal Auditor</em>’s John B. Thurston award for the article, “How to Audit Culture” and conducts the seminar, Auditing Culture: Challenges and Proven Techniques for The IIA. </p>https://iaonline.theiia.org/authors/Pages/James-Roth.aspx

 

Comment on this article

comments powered by Disqus
  • IIA AuditBoard_Nov 2019_Premium 1
  • IIA GAM_Nov 2019_Premium 2
  • IIA OnRisk_Nov_Premium 3