Home    Index of Questions    Index of Tools    Index of Topics    Case Studies
Search   Active Schematic   Complete Guide   Resources   Comments

The Short Answer

(1) Start small. Don't do it for the whole enterprise in a single budget cycle. Try it out with your best managers. Work out the bugs so it is actually useful to them. Then...

(2) Work from the bottom up.  Grow it up through your organization. 

(3) Work from the top down. And model behavior. Use performance measurement and accountability  in the interaction of the head of the organization and the next level down.

(4) Eventually, use a common sense, minimal paper process at every intersection between a supervisor and subordinate. (See the 7 Questions) And use it in the budget process.

Full Answer

(1) In any organization the best thing to do is to start small. Prove that something can be successful and most importantly useful and build out from there.

(2) The history of performance measurement is that one day the chief executive comes charging out of the shower all fired up about performance measurement, gets to the office, declares that starting tomorrow every unit of the agency will have performance measures. About 3 months later you've generated a thousand pages of paper, most of it completely useless. Eventually the system collapses of its own weight, until the next executive comes charging out of the shower. Do not create a thousand pages of useless paper. Work to create one page that actually helps managers run their programs and improve its performance.

(3) Work top down and bottom up. (The following sections are taken from "A Guide to Developing and Using Performance Measures."

Building a Performance Measurement System from the Bottom Up

Whatever else may be true of performance measurement systems, they almost always display too much, not too little data. Typically, for each sub-sub-program, 10 or more performance measures are shown. As we move from sub-program to program to agency levels, the number of displayed performance measures grows exponentially. We provide executive and legislative branch decision makers with a sea of data, and no particular way to sort out what is important from what is not.

While it makes sense to build performance measurement systems from the bottom up, this does not mean we must adopt the undisciplined practice of using unlimited numbers of performance measures. The first and most important feature of a good performance measurement system is the use of a common sense approach to seeing the forest for the trees.

The first task is to contain the data explosion at each step in the construction process. For each level of performance, we could identify the 2, 3 or 4 most important performance measures. Measures not selected here can and should be used, but don't need to appear in the management or budget document. The four-quadrant sorting bin can be used to help select primary measures at each step in the process.

Using this approach, each level of the performance document or budget has the same amount of performance information organized in roughly the same way. Agency X monitors its performance on 3-4 primary measures. Program X monitors its performance on 3-4 primary measures. And so forth. If you want more detail, go to the level below (or to the data identified, but not selected) In an agency with three levels (agency, program, subprogram) it works like this:

1. For each subprogram

Identify all the "candidate list" of performance measures available in the four quadrants above.

Pick the most important 2, 3 or 4 primary measures. These should generally come from the right-hand quadrants. (See section C below for additional criteria for selecting primary measures.)

Create baselines with forecasts for these measures.

2. For each program repeat this process using the performance measures of the program's subprograms as the candidate measurement list.

3. For the agency as a whole, repeat this process using the agency's program level performance measures as the candidate list.

In the course of this work it is not uncommon to find programs, and even whole agencies, for which there is very little good data. When this is the case, the data selection process is not about picking the best of good data candidates, but finding any good data candidates. There are rarely any easy answers to this problem. But it is important to proceed with development of performance measurement with what you have and work to improve the system over time. It is sometimes possible to create data, based on sampling techniques (by reading a limited number of case records, for example) as a short term substitute for later data system development.

A related problem has to do with the relative scarcity of quality measures in data system reports. Most agency data systems count quantity not quality. Here, one relatively simple suggestion might help. Consider the role of "composite" performance measures, that is, performance measures which are created by calculating the ratio of two existing quantity measures. For example, many agencies count the number of safety or compliance violations among the programs they supervise. By themselves, the raw count of violation totals don't mean much. But by calculating the ratio of program components with reported violations to total program components, a useful measure of quality can be created. Most good quality measures, whether currently reported or proposed, take the form of composite measures.

Building a performance measurement system from the top down
or a word about that rare occasion when a top-down approach makes sense

One of the most common mistakes in the use of performance measurement in management and budgeting is the tendency to implement performance measurement all at once on a grand scale. "Starting next week, every manager of every program and sub-program must begin reporting on performance." Mountains of paper are produced. Little of it is used for anything. People come quickly to resent the intrusion of these new time-consuming and largely useless tasks. And the system is eventually abandoned.

There is nothing wrong with having performance measures for every component of an agency. But consider a different way of getting there. Imagine that the agency director asked each of the people who report directly to her or him to bring a few performance measures with them to their next meeting. This could take the form of the four quadrant chart filled in with one entry in each quadrant. They could discuss three things:

What does this data tell us about performance?

What more would we like to know? (for example, comparison to last year, last month, 1, 2 or 5 year trends, maybe forecasts of performance...who knows where this will lead?)

Are these the right/best performance measures? The four- quadrant chart could be used to add or drop performance measures in these first meetings.

This process could, over a few months, lead to the creation of a regular performance report to be reviewed at each meeting. Over time the performance measures could become the basis for agreeing on agency or even personal goals for performance (and in the most advanced scenario, could be used for performance "contracting" between the program manager and the agency head.)

By starting the process this way (or using this method to build on and existing performance measurement system), two very important messages are sent:

Performance measurement is part of day-to-day management. It is not some back-burner, humorless, tedious and irrelevant exercise; and

Top management is modeling behavior for the rest of the organization.

This is why the top-down approach makes sense in this case. This allows, even encourages, the senior management to use this same process with the people who report to them, and to build downthrough the organization. (This is not the way the management books tell you to do it, but it probably works better.)

Still another reason why working from the top down makes sense is that the performance measures of individual programs and subprograms should be tied to the most important performance measures for the agency as a whole. If it is done right, working top down will give people a sense of what top management sees as important, without making this an inflexible and domineering perspective.

The best work on performance measurement will be iterative, top down and bottom up. But top-down work of any sort has taken such a beating in the management literature that we sometimes don't recognize the times when it has a legitimate and important place. This is one of those times.