| Evolving Roles for Auditors in Government Performance Measurement - December 2000 | | Print | |
| Written by Stuart S. Grifel, Stephen L. Morgan, and Paul D. Epstein |
|
Introduction In the past audits were of limited scope with the focus only on determining if government was doing things right. Today’s audit objectives reflect a definition of controls that has been expanded to include all of the methods, systems and functions used to achieve the purpose and mission of an organization. As part of this broader scope, auditors are increasingly taking on the role of champion for performance measurement and performance accountability systems. Jonathan Walters in the recently published book Measuring Up (Walters, Jonathan, 1998, Washington, D.C.: Governing Books p.126) states that “the auditing community in general is clearly starting to lean more toward increased involvement in the whole performance measurement game.” At all levels of government, auditors have demonstrated an ability to use their knowledge and experience to assist their organizations in an advisory capacity in developing and implementing effective performance measurement systems, without compromising their independence and objectivity. This is especially important as government auditors continue to expand their roles outside of the traditional audit function.
In recent years local government auditors have been involved in their organization’s performance measurement systems in roles that range from the more to the less traditional. Traditional roles include: Verifying the reliability and validity of reported performance reports and underlying performance data. Measuring performance during an audit. Auditing the adequacy of existing performance measurement and accountability systems. Most of this article examines five less traditional roles. The first two of these roles are similar to traditional roles, but with a new twist:
Mandates that have stimulated more organizations to develop and implement formal measurement systems have also generated the need for a more elaborate audit process. In 1992, The City Auditor’s Office of Austin drafted a performance measurement and reporting resolution that was approved by the City Council. The resolution required all City departments to implement performance measurement systems and periodically report on performance. It also required the City Auditor’s Office to audit the reliability and validity of the performance reports and to assess the adequacy of departments’ performance measurement systems. Extensive audits of the City’s performance measurement system were conducted in 1994, 1996, and 1998. The 1998 audit served as one of the catalysts for the City’s business performance planning process described in this article. Although the numbers of auditors involved in performance measurement has been growing, a majority of local government audit offices still have not recognized that by providing an assistance role in their organization’s performance measurement efforts, they can add another tool to their menu of value-added services. A fiscal year 1998 survey conducted by the Association of Local Government Auditors (ALGA) found that only 25% of the 58 reporting audit shops reviewed or analyzed submissions by operating departments to verify the reliability of management presentations of performance measure-ment data. This is while 66% of those 58 audit shops indicated that performance measures are being used by their respective organizations (Report on ALGA’s Benchmarking and Best Practices for Fiscal Year 1998, p.8.). This article describes how several local audit offices have begun putting a “new twist” on traditional audit roles to add further value to their jurisdictions. The article then goes on to present still newer roles for auditors in performance measurement, with examples including roles which involve audit staff interaction with citizens, and suggests a potential future role of auditors’ in assessing citizen participation processes. Traditional Roles for Auditors with a New TwistProactively assisting management in establishing and using a performance measurement system A less traditional role for local government auditors is to proactively assist management in establishing and using a performance measurement system. The Office of the City Auditor in Austin has been partnering with management since the fall of 1999 when the City began a major effort to reinvent its performance measurement system to serve as a key component in Managing for Results. Through an intensive business planning process city departments refined their measures so their activities were aligned with the City’s vision, measured, and budgeted accordingly. The City Auditor’s Office was first involved with management in developing guidelines for the new business planning process through an “Editorial Board” made up of representatives from various city departments. Two members of the Auditor’s Office then served as “equal partners” on the Corporate Business Planning Team that was tasked to review the business plans of all City departments. The business plans went through a series of reviews that first looked at the alignment between mission, goals, objectives, and measures. A second review focused on the relevance of each program’s performance measures and suggestions were made by the Corporate Team to add, delete, or revise measures where appropriate. The City’s efforts in managing for results played a large part in its receiving an A- score in the February 2000 issue of Governing magazine. The Office of the City Auditor had an opportunity to be involved in the “front-end” of the City’s efforts to improve its performance accountability and measurement system. The Auditors Office found that this proactive role is preferable to auditing the performance measures after they’ve been developed and having to recommend that they be changed. Now, when auditing a particular program, there is at least a reasonable assurance that the measures that are being audited are relevant. In addition to its role as a member of the Editorial Board and the Corporate Business Planning Team, the City Auditor’s Office of Austin has had a role in providing training on performance measurement and recently trained the City’s Health and Human Services Department managers on using performance measurement data as part of a performance accountability system. This two-day training program provided trainees with multiple examples and hands-on exercises that showed trainees how performance measurement data can be used for management and budgetary decision making, and for strategic and daily planning. Issuing performance measurement reports outside of the audit For the past ten years the Auditors Office of Portland, Oregon has reported to the public on the Service Efforts and Accomplishments (SEA) of key city department (available for download at http://ci.portland.or.us/auditor/audser/html). Although not part of the audit process, the Auditor’s Office checks the accuracy and reliability of the data provided by city bureaus, other cities, and citizens. This information is checked by comparing reported data to budgets, completed financial and performance audits, and other reports and documents obtained from bureaus and other cities. The Auditor’s Office then compiles a “user friendly” report readily available to citizens on Portland’s performance. The Auditors Office of Prince William County, Virginia prepares that county’s Service Efforts and Accomplishments Report (available for download at http://pwgov.org/budan/sear/Sea00/default.html). This report is prepared on an annual basis with the purpose of providing supplemental information to the Board of County Supervisors and county management so they can make more informed budgetary and policy decisions. The Multnomah County (Portland, Oregon) Auditor’s Office is currently implementing the first County Service Efforts and Accomplishments report. The Auditor’s Office believes that, to improve accountability with a SEA report and to ensure the report is meaningful to the public, it must be citizen driven. Citizen engagement in performance measurement and reporting can strengthen accountability by making information more relevant to their interests. The Auditors Office SEA Reporting Feasibility Study states that they “will continue to pursue as many avenues as possible to engage citizens and make the report as relevant and useful as possible.” (January 2000, p. 28. Available for download at http://www.multnomah.lib.or.usaud/sums.html) New Roles for AuditorsGiven the evolutionary nature of the auditor’s role in performance measurement one can foresee a future where auditors play a more active role in working to evaluate measures or benchmarks that citizens and elected officials indicate are of greatest interest to them. These are often measures that focus on issues, or conditions in the community. Suzanne Flynn, Auditor of Multnomah County, Oregon, says that one way to increase accountability is to increase the information that is available to citizens to assess performance. To do this, meaningful and useful reports and mechanisms that engage citizens need to be developed. A recently completed report by the Minnesota Citizens League entitled Engaging Citizens in Achieving Results that Matter: A Model for Effective 21 st Century Governance found that a government can achieve the greatest value from its measurement and improvement efforts when it aligns citizen engagement, performance measurement, and policy and implementation. (Epstein, P., L. Wray, M. Marshall, and S. Grifel, 2000. Available from the Citizens League on the web at http://www.citizensleague.net/cl/SLOAN/cover.html.) Audit offices have interacted with citizens through activities that have included conducting citizen sur-veys and focus groups, helping in the development of community indicators, supporting a multi-govern-ment/ mixed government-private leadership group, and conducting evaluation studies for citizens. Given this progression, local government auditors may one day find themselves serving as independent review-ers of citizen participation processes. Conducting focus groups and citizen surveysThe Auditor’s Office in Kansas City, Missouri has conducted focus groups that included a total of 26 neighborhood and small business leaders from all parts of the city. This input was helpful in discussions with departments in establishing a list of recommended measures. The City Auditor’s Office has also completed special reports specifically designed to develop performance measures in police patrol and investigation and parks and recreation services. In Portland, Oregon, the City Auditor’s Office conducts annual surveys of citizen satisfaction and perceptions, using samples for the whole city and for Portland’s seven “District Coalitions,” which help focus Portland’s extensive neighborhood-based citizen involvement. The Auditor’s Office reports the results, including maps comparing results for the seven districts, in its Service Efforts and Accomplishments Reports. The Auditor’s Office in Phoenix, Arizona has assisted in the development of community performance indicators and solicits input from citizens as they develop performance measures. In developing these measures the Auditor’s Office held focus groups throughout the community. The Auditor’s Office also assisted departments by performing customer satisfaction surveys upon request and assisted in updating the monthly City Manager Executive Report, which includes indicators for every city department. Interpreting measures and conducting evaluation studies for citizens. The Office of the City Auditor of Portland, Oregon serves as staff to the Portland-Multnomah Progress Board. The Board, established in 1993 by the Mayor and County Chair with public and private members, set benchmarks which serve as measures of the condition of the community. In its staff capacity the Auditor’s Office not only tracks the benchmark data, but also has helped citizens with interpretation of measures and has conducted special studies of 3 of the 76 benchmarks. Benchmark studies have been conducted on Children’s Readiness to Learn, Educational Success, and Salmon Restoration. (Studies are available for download from the Portland-Multnomah Progress Board website at www.p-m-benchmarks.org) The introduction to the Children’s Readiness to Learn study states that: These studies provide the Progress Board, other policy makers and the larger community with a better understanding of the forces that affect a benchmark, recommendations about future measurement of the benchmark, research about the best practices for improving the benchmark, and an assessment of the array of services and programs involved in addressing the benchmark. The Board hopes that these benchmark studies produce better measurement and collaboration among citizens.” (1998, p. i). Potential Future Role: Reviewing citizen participation processes related to performance measurement As more citizen participation processes evolve to engage citizens in setting community goals, selecting performance measures, collecting data, or interpreting reported results to suggest policy changes, these processes may become both more influential and more controversial. Auditors may thus be called upon to assure that adequate opportunities were provided for representative, autonomous and “authentic” participation. Authentic participation refers to citizens having a realistic opportunity to make a difference, as described by King, Felty, and O’Neill in their 1998 article The Question of Participation: Toward Authentic Public Participation in Public Administration (Public Administration Review, Vol. 58, No. 4). As citizen participation takes on more importance, participation processes become ripe for challenges by both citizens and public officials who disagree with the results. It may become important, in the future, for an independent authority, such as an auditor, to verify that reasonable steps were taken to attain participation processes that were representative of the community, autonomous from (or at least not controlled by) governing officials, fair, and meaningful (or “authentic”). Such a verification role would not be as unusual for auditors as it may initially seem, as it can be done by establishing criteria for good processes, and comparing actual practices with the criteria. By taking on roles with respect to citizen participation, auditors can “legitimize” citizen engagement as an important part of measuring and improving performance. In Conclusion Since performance measurement itself is still evolving as a tool in many local governments, it remains to be seen as to what extent auditors will be involved with citizens in the performance measurement process. Some auditors have already engaged citizens in matters related to performance measurement, and still new roles, such as reviewing citizen participation processes, are possible in the future. Whatever the future holds, local government auditors will have more and more of an opportunity to play a critical role in assuring that their government’s performance measurement system will be a useful and reliable tool for assuring accountability and managing for results. Editor’s Note: An expanded version of this article has been submitted to the Institute of Internal Auditors for future publication. In addition, information regarding the Portland Multnomah Progress Board was updated. Stuart S. Grifel is an Auditor-in-Charge for the Office of the City Auditor, Austin, Texas. Mr. Grifel has an MBA from Suffolk University in Boston, and an MPA from Baruch College, City University of New York. Stephen L. Morgan, CIA, CFE, CGFM, is Acting City Auditor for the City of Austin, Texas. Mr. Morgan actively participates in the Institute of Internal Auditors serving as the 1995-96 President, Austin Chapter and as the current Chair of the International Government Relations Committee, representing over 12,000 government auditors. Paul D. Epstein is Principal of Epstein and Fass Associates, a New York City-based consulting firm. He co-leads the Citizen’s League research team on citizen engagement in performance management, and is a member of GASB’s performance measurement research team. |
From the Quarterly Editors
-
Read more...
Anytime a group of auditors gathers to discuss the American Recovery and Reinvestment Act (ARRA), words such as "unprecedented" and "new paradigm" often follow. The idea of covering such a groundbreaking topic seemed overwhelming at first. -
Read more...
Congratulations to all those who worked hard to put on an incredible 2010 ALGA Conference in San Antonio, Texas. Even with a dismal economy, many of you saw the value in enhancing your audit skills, networking, and supporting our only local government organization. To all of you, Thank you!
Opportunities for Improvement
-
Read more...
We need to talk about risk, or at least I do. I must finally confess that I have always been uncomfortable about the risk concept. I understand risk and believe it's important to think about in our audits but I also think auditors may miss important findings if they rely too heavily on a risk-based approach.
Past ALGA Quarterlies
-
Summer 2010: ARRA
Spring 2010: Parks and Recreation
Winter 2009: Information Technology
Fall 2009: Social Services
Summer 2009: Public Safety
Spring 2009: Stewardship
Winter 2008: Courage
Fall 2008: Integrity
June 2008: Creativity
From the ALGA President
-
Read more...
A Texas-sized thank you to our incoming president Beth Breier and everyone who volunteered to help make our 22nd annual conference in San Antonio such a success.
AudiTechie Column
-
Read more...
In the spirit of thematic issues of the LGAQ, we are looking at SharePoint technology through the lens of the American Reinvestment and Recovery Act (ARRA). The focus on accountability and outcomes for ARRA funding is unprecedented and technology is facilitating the discussion. SharePoint is a Microsoft product that allows you to customize an internal website to share and manage information.
