| Written by Mark Funkhouser,
|
Recently a friend of mine was asked to go to the Balkans to evaluate a program working to resettle and reintegrate ethnic minority people who fled during the Bosnian civil war. She is a social worker with some experience in evaluation and was supposed to be part of a two-person team, the other member of whom was the more seasoned evaluator. At the last minute the war in Kosovo occupied the resources of the organization that was sponsoring the work and she found that she was to go alone. Feeling some pressure, she asked me to review the documents describing the program and see if I had any advice on how to approach the evaluation.
Perhaps the readers of this column might find the thoughts I put together for her to be of interest. These ideas seem to be good things for me, an experienced (some might say, grizzled, battle-hardened, war-weary, shop-worn) auditor, to remember in any crunch-time audit where the pressure is on to do my best work, quickly, and under new or very difficult circumstances.
Risk assessment. In your own mind and in conversations with stakeholders, consider what could go wrong with the program and what measures are in place to prevent the bad things from happening. Consider two elements: the likelihood of occurrence and the magnitude of the potential harm. If the combination is small or if the preventive measures seem strong, then forget about it. If the preventive measures seem weak or the combination of likelihood and magnitude seems large do some kind of checking to see whether the bad things have occurred. Checking may simply be asking stakeholders whether the things have happened. If they say no, say how do they know? If they say yes, the things have occurred, then ask them to describe details.
Systematic examination. In the interest of fairness and due diligence, make sure that you include all groups of stakeholders in interviews if at all possible. Make sure that, if you can, you systematically make inquiries or do your other work. Every other site, every third person, the three largest ... whatever. The issue is to be as fair as you can, not be arbitrary to an unwarranted degree, and be able to explain to people why you did what you did.
Take photos, record notes of conversations and observations and record feelings, emotions and impressions in a daily journal. Do not try to do all or even most of your analysis at the scene. Save that for later. Gather information and do as much as you can to allow you to remember what you heard and saw. This will also allow you to put things in context, not arrive at too early a closure on specific issues and to not be unduly swayed by the first person you talked to or the last, or the most emotional thing you saw.
Efficiency, effectiveness and equity. Efficiency means, in essence, serving the most people and thereby doing the most good you can with fixed resources. Effectiveness means that the help that is provided is meaningful in addressing the problems the people face. Equity means that access to services is fair and that service decisions are based on merit.
Findings. The elements of a finding are: condition, criteria, cause, effect and recommendation. When you find things that are not as they should be, consider the elements of a finding as a way to conceptualize and communicate what you believe to be the case, what should be happening instead, why things have gone wrong, what the consequences are, and what should be done to improve the situation.
Go with your strengths. Consider your strengths as a person and as a professional and then focus your work in ways that play to your strength. If you go about this in ways that are the most satisfying to you and that allow you to do the things that you do best you will be more successful. This is not the time to try to broaden your repertoire of skills. You can do this and you are all that any one has to do it at the moment. Immerse yourself in it and have fun.
|
Funkhouser on Auditing