1. Be active, not passive, in the design and
implementation of an evaluation of your program.
2. Work in partnership with the evaluator. Make sure the evaluation meets
your needs.
3. Don't be cowed by all the numbers and formulas. Trust your common sense. Ask
questions.
4. Be aware that different evaluators will use different language. Take
the time to crosswalk to your use of language.
Advice from:
Organizational Resources
References
The Short Answer
(1) Performance accountability
and performance or program evaluation both make use of performance measures.
Evaluation is part of
performance accountability.
(2) Different
purposes: Evaluations provide a structured,
disciplined analysis of how well a program works, so that managers and funders can make judgments about
how and whether to change, continue or terminate a program and whether it is worth replicating.
Evaluations sometimes tackle the challenge of differentiating the effect a
program has on client outcomes from other "outside" factors.
(3) Performance
accountability is a management and oversight process that makes use of evaluation findings,
and many other sources of information to manage the program, track
performance, report to stakeholders and improve performance.
Full Answer
They go out to parties on weekends.
Performance accountability,
performance measurement and evaluation are all terms of art. Like other
terms in this guide they do not have "standard" definitions, and
different writers will define and use the terms differently.
(1) Performance accountability
and evaluation both make use of performance measures, but
evaluation is part of
performance accountability. Evaluations provide a structured,
disciplined, analysis of how well a program is working or has worked.
Evaluations sometimes tackle the challenge of differentiating the effect a
program has on client outcomes from other "outside" factors. Evaluations are
often commissioned so that funders can make judgments about whether a program
should continue to be funded, whether it should be changed, and whether it is worth replicating. Performance
accountability is a management and oversight process that makes use of evaluation
findings,
and many other sources of information to manage the program, track
performance, report to stakeholders and improve performance.
(2) The evaluation world is divided into at least two armed camps. First
there are the evaluators who take their tradition from the audit world. They
wear white coats. They do not "help" the program while it is
operating for fear they will contaminate the findings. They pass judgment as
objective outsiders. Their reports often come out two years after the program
has ended telling what was wrong and why it failed.
Then there are the
"self-evaluation" or "empowerment evaluation" folks.
They see their role as partners with the program, providing feedback and
helping make midcourse corrections. They use the same statistical
methods as the first group, but see evaluation as a continuous process which
helps improve the chance that the program will succeed. This guide holds a
distinct preference for the second philosophy. Self evaluation methods are at
the heart of performance accountability. They both use data to steer programs and
improve performance.
(3) Rigor: Evaluations are
rigorous, designed to stand up to academic scrutiny. Accountability is a
real world process, pragmatic and sometimes political, using whatever information can be obtained,
going on gut instinct when no information is available, doing whatever it
takes to make the program work.
(4)
One time vs. ongoing data collection and reporting. Evaluations are
often structured as one time events. Often evaluations produce a single
report. Performance accountability, on the other hand, requires the
collection and use of data on a regular basis, and regular continuous
reports.
(5) Approach to data collection: Evaluation data collection is often quite
detailed, to the point that it sometimes burdens the program. Performance
accountability, as presented in this guide, looks first for existing data
before collecting new data. And it looks for the smallest possible
data set which can be used to run the program. Both can use similar data
collection methods including 100% reporting and sampling. Performance accountability
makes use of all data available (anecdote, hunch, instinct).
(6) Expense: Evaluations can be expensive, partly because they are often
performed by consulting firms or institutions which charge a lot of money to
do the work. Performance measurement and accountability can and must be done
by program staff themselves. This requires time, but usually less expense.
Most managers should be able to answer the 7
Questions on a periodic basis without new staff resources.
(7) Do you need both? Often you do. All programs need some form of
performance measurement and accountability. Evaluations can take data based judgments
about performance to a higher more rigorous level. They can answer
questions (such as "Is this program worth replicating?") in a more
complete and disciplined way. And often research evaluations are the only way to get concrete information on
cause and effect relationships.
