Association of Local Government Auditors

In today's world of corporate scandals and highly publicized government control failures, stockholders and members of the public are increasingly demanding improved governance, better controls and enhanced accountability and transparency.

Recent massive government bailouts including the Troubled Asset Relief Program (TARP) and the American Reinvestment and Recovery Act (ARRA) have exacerbated the public's demand for greater accountability and oversight. The citizens of the City and County of Denver put these concepts into practice by overwhelmingly voting to amend the Denver Charter, thereby establishing a unique, robust and independent internal audit model. This transformational change occurred January 1, 2008 effectively altering a structure that had been in place since 1904. As a result of the creation of this unique government audit model, the Denver Auditor's Office is ideally positioned to meet the public mandate for greater accountability, transparency and oversight.

This article highlights the Denver City internal audit model including a description of key attributes and examples of impact. As most public and private sector internal audit functions are not structured in such a manner, the description of the Denver audit model provides Association of Local Government Auditors (ALGA) members with insights into some of the critical, fundamental elements utilized by the City to perform high impact internal audits. We hope our ALGA colleagues will use the Denver model for ideas and possibly as best practice criteria for strengthening their internal audit functions.

The Denver Model

By their vote, the citizens of Denver established one of the most unique and effective government internal audit functions in the nation. The key attribute to the Denver model involves the significant level of the internal audit function's independence. Several provisions contained in the City's Charter establish the internal audit function as truly independent. The following four key components serve as the cornerstone for establishing the independence and subsequent high impact potential of the City's internal audit model.

1. Elected Auditor - The City of Denver has an elected auditor, who constitutionally serves as the second highest elected official for the City. As an elected official, the City Auditor is completely independent from other City elected officials and operational management. Under this model, the auditor is directly and solely accountable to the citizens of Denver. While there are some exceptions, government internal audit functions generally lack complete independence from elected officials and operational management. In many cases, internal audit functions report administratively and functionally to operational management, such as City Managers, Chief Administrative Officers, and Chief Financial Officers and in other cases to elected bodies, such as City Councils or County Boards of Supervisors. Internal audit functions structured in this manner can perform excellent work and have a positive impact, but they lack complete independence which inherently limits their ability to perform unfettered audit work from a public perception.

2. Broad Audit Authority - Per the changes made to the Charter by the people, the City of Denver's internal audit function has widespread audit authority to perform its responsibilities.

• Comprehensive Access - The internal audit function has access to all City records and employees as well as to the records and employees of all entities doing business with the City. Further, City statute provides comprehensive confidentiality protection for internal audit communications, work papers and work products. Prior to the Charter change, the Denver Auditor's Office had no legal authority to access records and employees significantly limiting the efficiency of the audit project management process as well as the ultimate impact of completed audits. In many cases, record requests were simply ignored by operational management. Unlike the Denver model, many public sector audit groups lack the broad and clear legal authority to access all records and employees. The ability to independently access all records and employees in a timely manner is a critical attribute of the Denver audit model.
• No Mandated Audits - As another unique component of the Denver model, the internal audit function independently establishes and executes an annual, risk-based audit plan. More specifically, as a result of the Charter change, previously mandated audits were stripped from Denver Revised Municipal Code (D.R.M.C). These mandated audits generally involved low risk programs and expenditures and required valuable audit resources to complete. For example, a mandate existed for an annual audit of several City petty cash funds. The ability to independently establish its audit plan enables the Denver Auditor's Office to allocate limited resources effectively based on risks identified by professional audit staff. Conversely, audit priorities and projects included on the audit plans developed by many public sector internal audit functions are often heavily influenced by operational management or elected officials.
• Formal Audit Response and Follow Up Requirements - The Denver audit model also includes statutory provisions that require audited entities to respond to all audit findings and recommendations in a formal manner. These requirements serve as an effective accountability mechanism and ensure that internal audit reports are acted upon and problems identified by these reports addressed. Specifically, per statute, audited entities must respond within a certain timeframe, indicate agreement or disagreement with findings and recommendations and include a timeframe for implementing corrective actions. Prior to the Charter amendment, City entities frequently refused to provide responses to audits and in many cases responses that were provided were not "responsive" in that they did not indicate agreement or disagreement with findings and recommendations and did not identify corrective action plans and responsible individuals. As a result, the Denver internal audit function identified the same findings year after year. The issue of chronic recurring audit findings is commonplace among many public sector audit functions because they lack effective audit response and follow up programs. The new Denver model includes a specific legally mandated control for ensuring audit findings are promptly addressed and corrected.

3. Stringent Adherence to Professional Audit Standards - A legal mandate to fully comply with professional audit standards serves as a key element of the Denver audit model. The Charter mandates that the Auditor conduct audits of all City entities in accordance with generally accepted Governmental Auditing Standards (GAS) promulgated by the United States Comptroller General. These standards establish clear definitions and requirements related to the independence of the audit function. The fact that the Denver City Charter now requires the City Auditor to comply with these standards has resulted in the establishment of one of the most structurally independent local government audit functions in the country.  In addition to adhering to GAS standards, Denver's internal audit function also considers standards promulgated by the Institute of Internal Auditor's (IIA) International Professional Practices Framework in the course of work activities. For example, this framework serves as significant criteria for developing our audit plan.

4. Independent Audit Committee - The final substantive change resulting from the Charter amendment was the establishment of a truly independent City audit committee. The audit committee, consisting of seven members and chaired by the City Auditor, is a Charter-level organization that is completely independent from operational management. The Mayor, City Council and Auditor each appoint two members. The Audit Committee performs several critical functions including, among other responsibilities, annually commissioning and monitoring an independent external audit of the finances of the City, known as the Comprehensive Annual Financial Report (CAFR) and receiving internal audit reports. Audit committee meetings are frequently televised providing greater public exposure to internal and external audit findings and audit follow up activities.

Prior to the Charter change, the City audit committee did not have a formal legal standing and its members were appointed by the Mayor, meaning it lacked independence from operational management. Generally, public sector internal audit functions, especially at the local government level, have audit committees structured in this type of manner, if they maintain an audit committee at all.

Modernizing the City's Internal Audit Function

The significant and structural changes made to the Denver internal audit function as of January 1, 2008 have resulted in immediate impact and spawned a new dawn of transparency and accountability over City operations. Provided with the clear legal backing to perform its mission independently, the City Auditor took action to significantly improve internal audit operations, resulting in immediate value and high impact audits.

Strategic Staffing Strategy

A vital component of modernizing the internal audit function involved implementing a strategic staffing strategy. As part of the enhancement of the City's internal audit function, the Mayor's Office agreed to fund three additional, four-member performance audit teams bringing the total number of City audit staff to thirty-four professionals, which is a robust compared to peer Cities. To effectively manage these resources, the Denver Auditor's Office utilizes a dynamic staffing strategy based on the following three components: (1) recruiting, developing and maintaining a highly skilled audit staff; (2) adherence to an integrated audit philosophy and; (3) use of a flexible management matrix approach.

Denver's diverse professional audit staff members hold memberships in a number of professional organizations including ALGA as well as a wide range of professional certifications including: Certified Internal Auditor (CIA), Certified Government Audit Professional (CGAP), Certified Public Account (CPA), Certified Fraud Examiner (CFE) and Certified Information System Auditor (CISA). Additionally, several staff members have received advanced graduate degrees. The 2009 Annual Report describing the Office's staffing structure and staff member qualifications is available on our website at www.denvergov.org/auditor.

Dynamic Audit Plan

The Denver Auditor Office's dynamic and flexible audit plan designed to enhance the impact of audit services is another example of innovation being utilized to deliver high impact internal audits. The audit plan is based on developing a realistic audit horizon of planned high risk audits covering a three-year period. Audits included in the annual audit plan are selected and prioritized using a risk-based approach. The plan includes significant hours for performing specially requested audits and for urgent audit issues that arise throughout the year not originally captured on the plan. This approach provides the Auditor with a great deal of flexibility to address emerging issues in a timely manner and for providing high quality and responsive customer service to elected officials and operational management.

The current annual audit plan reflects a new emphasis on performance auditing, particularly in the areas of program effectiveness and assessing the economy and efficiency of various City departments and programs, as a key mandate resulting from the City Charter amendment. Historically, the Auditor's Office conducted numerous performance audits focused on internal control and compliance objectives. However, the 2009 audit plan includes program effectiveness and economy and efficiency performance audits for areas deemed high risk and many of them emphasize "horizontal" programs and activities that extend beyond individual departments and programs.

Many of these audits focus on the City's general governance structure for managing these horizontal activities to ensure comprehensive control structures are in place and efficient and effective communication processes exist between operating departments. In addition to identifying systemic types of issues involving key City responsibilities and activities, the primary intent of these types of audits is to assess the City's enterprise risk management approach. This audit emphasis is well aligned with the heightened focus nationally in both the public and private sectors towards strengthening and improving organizational governance and internal control environments.

Finally, the plan also includes audits that exhibit the Auditor Office's strategic focus on information technology auditing and fraud prevention and detection. The Auditor Office's 2009 Audit Plan is available on our website www.denvergov.org/auditor.

Formal Information Technology Audit Program

As part of the cutting edge restructure of the internal audit function, the Denver Auditor's Office established the City's first Information Technology (IT) audit team, directed by Certified Information System Auditors (CISA). Local government technology organizations often face the same risks as large corporations. In fact, the risks faced by governments may be higher, as government agencies are sometimes specifically targeted by those with malicious intent. However, governments operate with a significantly smaller budget and the proper functioning of technology controls is critical to protecting our information resources. In fact, in many cases, government organization lack qualified professional staff specifically dedicated to IT auditing activities.

To ensure City IT risks are adequately controlled, the Denver Auditor's office is performing a citywide IT risk assessment as the basis for a detailed risk-based IT audit plan. The risk assessment will identify and prioritize IT performance audits to ensure the confidentiality, integrity, and availability (CIA) of the City's technology infrastructure. Main audit areas include governance, risk, and compliance (GRC), information security management, technology and operations, and disaster recovery. Technical audit steps may include reviews of system configurations, roles and responsibilities, and change control.

The new IT team has also enhanced the Auditor Office's use of technology, including the implementation of audit project management software and electronic work papers, and has utilized Computer Assisted Audit Techniques (CAAT) and data mining software programs to improve data analysis capabilities and results.

Advisory Services Suite of Products

In addition to the development of new performance and information technology audit capabilities, the Denver Auditor's office also implemented a suite of advisory services products. The intent of advisory services is to provide operational management with timely and critical information and analysis without the formality of an audit. Generally, with the exception of Audit Alerts, the Office provides these services at the request of operational management and elected officials. The following are specific descriptions of advisory services:

• Audit Alerts - Audit Alerts provide timely identification of potential risk areas that may prevent the achievement of City objectives. Alerts function as a rapid threading tool for disseminating key information identified by audit work. Those "at risk" entities can then complete a self-assessment and, if applicable, quickly self-correct any related control deficiencies.
• Special Advisory Services - Special Advisory Services provide information on limited reviews or time-critical assessments, investigations or evaluations. The Division generally provides these services at the request of operational management and elected officials. Special Advisory Services further City objectives and goals by providing a reporting vehicle that is flexible, quickly issued, and focused on singular issues.
• Management Advisory Services - Management Advisory Services are activities and reports designed to provide information and analysis related to organizational or programmatic assessments, investigations or evaluations. Also included is the identification of possible solutions or enhancements at the request of operational management and elected officials. Management Advisory Services are activities and products similar to those performed and provided by external consultants.
• Training Services - The Office offers control self-assessment training to City departments and entities. The training assists managers, supervisors, and fiscal personnel to understand management controls, the relationships between those controls, and the risks related to lack of effective controls. This knowledge is imperative and is central to the Strategic Vision of the City.

Evidence of Impact 

Though the Office underwent substantive internal changes during the year immediately following the City Charter amendment, the Denver Auditor's Office was able to produce a series of high impact audits and advisory services during the period that clearly demonstrate the positive potential and impact of the Denver audit model.

Audit Impact Case Study: The City of Denver's Emergency Medical Response System Performance Audit

The Denver Auditor's Office released a high impact performance audit of the City's emergency medical response system (EMRS) in 2008. The audit of this critical and complex public health responsibility is directly influencing significant improvements being made by the City and County of Denver. The ambitious project scope for this program effectiveness performance audit had a definitive impact on improving a critical public health function managed by City government. The audit scope included a review of Denver's two-tiered, multi-entity emergency medical response system. Three separate City agencies along with a large, third party provider (Denver Health and Hospital Authority), who is under contract with the City, have significant roles and responsibilities within the system. Audit work included an assessment of each of these entity's roles and related performance within the system.

The audit incorporated a multi-tiered methodology utilizing traditional performance audit techniques including: interview data, review of complex legal documents and policies and procedures, direct observation of audited activities, a benchmarking survey and best practices research. Additionally, the Office employed IT audit techniques to perform an extensive analysis of the City's Computer Aided Dispatch (CAD) system data used to dispatch emergency medical assistance. The audit team utilized Computer Assisted Audit Tools (CAAT) and highly effective data scrubbing techniques to review and analyze over one million system records for each year reviewed. For 2007 alone, we analyzed 1,066,841 CAD records, comprising 100% of the CAD dataset. The data analysis revealed that the City's times for emergency medical response were significantly longer than industry standards and City requirements and that these times were worsening. The Mayor's Office agreed with the recommendations in the report and immediately initiated implementation activities.

In addition to direct action being taken to address audit findings by City executive management and elected officials, the audit had a significant impact in terms of transparency and accountability related to a public service that is very important to most citizens. Specifically, audit work determined that the fragmented nature of the system along with systemic structural weaknesses and limited oversight reduced transparency and accountability and hindered the City from effectively monitoring and improving the City's overall emergency medical response system. Audit work identified widespread misconceptions and misunderstandings about the City's system especially related to emergency medical response time performance metrics and impact on patient outcomes. By presenting objective and quantifiable audit results, which received widespread media coverage, the Auditors Office provided critical transparency into the process for critical stakeholders and members of the public.

CONCLUSION

The City and County of Denver's internal audit model is truly structured for impact. In an age when citizens are demanding greater oversight, transparency and accountability by government entities, the Denver model should be viewed as a benchmark for both public and private sector entities. The core tenet that makes this model most effective is the true level of independence of the City's internal audit function. This independence results in the execution of a progressive and flexible, risk-based audit plan designed by professional auditors. Further, the results of the City's internal audits are presented to an independent Charter-level audit committee during publicly televised meetings, ensuring that both audit findings and recommendations are acted upon in a timely manner and that the City's residents are well informed of City operations.

Finally, the Auditor's Office has enhanced the model by implementing cutting edge audit technology and practices including full automation of the audit process, the utilization of electronic data tools and analysis, the execution of a dynamic staffing approach and the use of diverse services and flexible reporting tools. As a result, the Denver Auditor's Office is effectively meeting the demands of the City's residents by providing increased oversight, transparency and accountability over City operations.

Kip Memmott is Director of Audit Services for the City and County of Denver, Colorado