It's peer review time. Here comes a group of strangers (professional auditors you do not know very well, if at all) to visit your audit organization, take a random sample of the audit work conducted and examine it against the audit standards you claim to have followed. They also will ask for your audit manual and examine the sample of audit work they selected to determine if the work is done according to the policies and procedures it contains. Upon completion of their examination, they will prepare a written report which renders an opinion on the extent to which audit standards are followed and prescribed audit policies and procedures guide audit practices. You now know the answer to the often asked question: who audits the auditor?
Why, in this world of insecure employment, would any rational auditor even think about submitting to such an ordeal? What kind of self-destructive psychosis is at work that motivates an auditor working in the very political public sector to invite strangers in to review and report on the most deep innerworkings of the function for which that auditor is responsible? How can an action which may jeopardize continued personal employment and, in extreme cases, the longevity of top government officials, be justified? Knowing the "downside", is it worth "selling" top management, legislative bodies and audit committees on the concept that a report card on audit quality is essential, and worse, that they should use scarce resources to pay for it?
At the risk of exposing the auditor's soft underbelly to the cutting edge of the profession, I will attempt to argue for the practice of peer review beyond the simple need to comply with mandated professional auditing standards. Before exploring these issues, let me restate what is called for by professional standards: "Organizations conducting government audits should have an external quality control review at least once every three years by an organization not affiliated with the organization being reviewed." (Government Auditing Standards, page 3-18). This requirement implies that the auditing organization has established and is following adequate audit policies and procedures and furthermore, has adopted and is following applicable auditing standards.
Why, when the profession is as objective as the practice of auditing and the professionals who conduct the work are generally of such high calibre, is a function such as peer review necessary? Unfortunately, history indicates that while individual auditors may be of high calibre, the results of auditing practice point to needed improvements in audit quality, and the concomitant need for peer review. Looking only at audits conducted on government activities, there is sufficient empirical evidence to indicate that significant improvements in audit quality are necessary. The largest continuous aggregation of information about the quality of audit work has been gathered by the President's Council on Integrity and Efficiency (PCIE). Over the past few years, individual Inspectors General in the various federal departments have been quality reviewing audit work conducted on federal financial assistance programs for their departments. The vast majority of the quality review work has been in the form of "desk reviews." However, a significant number of completed audits have been selected for the more comprehensive "quality control review," which includes a review of audit workpapers supporting the auditor's reports.
Semi-annually, the Inspectors General aggregate the information from their individual departments and publish the results. Information about the quality of audit work is subdivided into three categories:
- Reports Issued without changes or with minor changes (reports which required only minor changes or did not require correction of other than minor deficiencies in the audit work).
- Reports issued with major changes.
These reports suffered from one or more of the following flaws:
- Material errors in a major component of the report (e.g., failure to include all federal programs/expenditures in the required supplemental schedules, large or numerous mathematical errors in financial statements).
- Inappropriate scope of audit.
- Workpapers which fail to adequately document some aspect(s) of the study of internal controls or testing of compliance requirements.
- Other report or audit work deficiency which by itself represents a departure from generally accepted government auditing standards, but which does not make the audit report unusable if not corrected.
Reports with significant inadequacies are reports submitted which are so pervasively unreliable that users cannot rely on them. Included in this category are:
- Use of unqualified auditors.
- Workpapers sufficiently inadequate to preclude an assessment of the adequacy of the auditor's work on the study of internal controls or the testing of compliance requirements.
- Lack of a major component of the report (e.g., financial statement(s), opinion).
- Failure to correct substandard work on a timely basis.
- Other gross departure from generally accepted government auditing standards which by itself undermines the credibility of the audit.
The quality review work reported by the PCIE covers audits conducted by non-federal auditors, CPAs, and state and local government auditors. Because the level of review conducted in a "quality control review" is much more comprehensive than a desk review, I presumed its results would have the highest probability of presenting an accurate picture of audit quality. However, on the possibility that the large majority of quality control reviews might have been selected by the various Inspectors Generals because they were identified as "red flags" in desk reviews, I surveyed the seven largest IG offices to ascertain how they selected audits for quality control reviews. While it is true that some of the IG offices use the desk review to identify audits for the more thorough quality control review, the highest percent of audits so identified was 60%. The lowest was 10%. The most consistent criterion used was materiality of the financial assistance to the governmental entity.
Several of the IGs indicated that every large grant for which they are cognizant automatically receives a quality control review. Others indicated that every time there is a new grant or a new auditor, a quality control review will be conducted; still others randomly select a large number of audits for review. The criteria used to select audits for quality control reviews thus is not simply an extension of the desk review process. Because quality control reviews are selected based on a variety of criteria, I believe that they are much more likely to reflect an accurate state of nature regarding quality of audit work conducted on government by CPAs and state and local auditors. Therefore, I elected to concentrate this analysis on the outcomes of the quality control reviews conducted by the PCIE. I have aggregated the two categories "reports issued with major changes" and "reports with significant inadequacies" because, in my view and in the view of many others, they represent substandard audit work.
The following tables disclose, in summary format, the results of these two categories of audit work reviewed over five periods (three years). Differences between the percentages shown in the analysis and 100 percent equal those audits conducted which met professional audit standards. The results were as follows:
PCIE Quality Control Reviews: Substandard Audit Information*
Single and Non-Single Audits*Desk Reviews Excluded | |
Substandard Audit Work - Single Audit
| | CPA Audits | State/Local Audits |
| | Number | % | Number | % |
| Fiscal year ended 1988 | 163 | 51 | 14 | 25 |
| Six months ended 3/31/89 | 52 | 40 | 5 | 21 |
| Six months ended 9/30/89 | 64 | 46 | 8 | 25 |
| Six months ended 3/31/90 | 35 | 46 | 4 | 22 |
| Six months ended 9/30/90 | 37 | 44 | 10 | 40 |
| Total | 351 | | 41 | |
|
| |
Substandard Audit Work - Other Than Single Audit
| | CPA Audits | State/Local Audits |
| | Number | % | Number | % |
| Fiscal year ended 1988 | 87 | 40 | 3 | 25 |
| Six months ended 3/31/89 | 64 | 64 | 3 | 75 |
| Six months ended 9/30/89 | 75 | 56 | 0 | 0 |
| Six months ended 3/31/90 | 135 | 72 | 0 | 0 |
| Six months ended 9/30/90 | 167 | 66 | 0 | 0 |
| Total | 528 | | 6 | |
Of the 879 audits determined to be substandard by the quality control review, 115 (13%) were so poorly done they resulted in referral to the appropriate state regulation board or professional society for consideration of disciplinary action.
At a minimum, these facts are indicative that there is a relative high level of poor quality audit work being conducted on the audited government programs reviewed in this ongoing sample. It is incomprehensible to me that, of the audits subjected to quality control review, the quality of audit work received from CPA firms for single audits resulted in a weighted average of 47.2 percent substandard work. And even more astonishing is that the weighted average for CPA work done on non-single audits in government indicates that 69.3 percent is below acceptable level.
Because the number of quantity control reviews conducted on state and local government audits in these type audits is so small, no empirical conclusion can be drawn from this analysis. However, given that most government auditors are familiar with GAGAS, there is every reason to believe that the quality level is a good deal higher than that found in similar CPA government audit work, a hypothesis echoed by many of the IG staff in discussing general quality of audit work during the survey mentioned above. Moreover, state auditors have been undergoing peer reviews under the aegis of the National State Auditors Association since 1981. Because peer reviews examine conformity with auditing standards as well as compliance with organizational policies and procedures, and the review also examines a statistically valid sample of the whole organization's audit work, peer review is considered a far more comprehensive evaluation than the quality control reviews conducted by the members of the PCIE. Since the inception of the NSAA peer review program, only one of the 56 peer reviews conducted by NSAA teams resulted in a qualified opinion. This is a reasonably clear indication that government auditors are achieving higher audit quality results than their CPA counterparts.
Another troubling factor which is evidenced by the last two PCIE semi-annual reports is that, notwithstanding the pressure for improvement in audit quality as manifested in reports from the General Accounting Office and as represented in the "expectations gap" auditing standards from the AICPA, there is no trend showing improved audit work discernible in the PCIE statistics.
In fact, the level of substandard audit work done by CPAs has substantially increased in non-single audit work in the more recent PCIE reports.
Using this information to highlight the need for quality improvement in the audit work conducted in government is not meant to castigate, but rather to focus on a very serious problem and cajole the profession into improved work quality.
In my own experience, an in house "peer review" conducted as a "dry run" preparation for our external quality review was enlightening. Our organization was firmly convinced that it conducts high quality audit work and meets all applicable GAGAS requirements, but our trial peer review identified a few areas in need of improvement. The benefit derived from our practice peer review was to modify certain aspects of our internal quality review function. We believe that what we learn from such introspection will assist us to grow with the profession and to continue to produce high quality audit work of which we may be proud.
Returning to the opening question: why peer review? The audit function plays an important role in the management of the entity it serves. This is especially true if the forms of audit include both financial and performance audits. In government, the function is arguably more important than in the private sector because the "stockholders" are the public. The auditor conducting work on government programs is not only responsible to conduct an audit in accordance with the applicable standards, but, depending upon whether the audit organization is internal or external, also serves as a mechanism to hold top government officials accountable for their use of the tax-generated resources.
If audits do not meet professional standards, there is little assurance that accurate, complete and/or reliable information will be generated for use in holding top government officials accountable for their managerial behavior or for any of the other uses served by audit work. It was because of this reasoning that both the GAO and the AICPA mandated peer review in their recent revisions to audit standards; GAO in the 1988 revised GAGAS and the Institute in the "expectations gap" standards. It is thus disheartening to learn that current available data shows between 40 and 72 percent of audit work being conducted by CPAs in government, and an unknown but likely much smaller percent of work being conducted by state and local government auditors, fails to meet the overall requirements of professional standards. The AICPA mandated peer review standard appears either not to have either been fully implemented or its effect has yet to become apparent. This indicates that the accountability function of government auditing may still not be as effective as intended. It also indicates that the facts reported by these audit reports may be misleading and unreliable for decision making purposes.
Peer review directly examines whether audit work is performed in accordance with GAGAS and, therefore, may be relied upon by those who depend upon the information contained in audit reports and the financial statements they report upon. Peer review is necessary to identify and correct those instances where quality of audit work is below standard and, because it is periodically required, it forces continual monitoring of auditing efforts. The target should be zero defects in audit work - clearly, the current substandard audit performance rate of CPAs is not even close to the scale of acceptability, much less the goal of zero defects. We can only hope the same is not true for auditors employed by state and local governments.
by Roland Malan of the New York Transit Authority
Peer Review