Results-Based Accountability™ Advice



How do we create a baseline (trend line) for an indicator?

The Short Answer

1. Baselines have two parts: an historical part which shows where we've been, and a forecast part that shows where we are headed if we stay on our current course. 

2. The historical part should have as much data as you can get. Aim for three to five years. If you have no historical data, then start the baseline where you are and build the history part over time.

3. Build one or more forecast scenarios using partners knowledgeable about causes and forces at work. Use statistical techniques as a tool not an answer. See the long form and short form techniques below for creating a forecast. 


Full Answer

(1) What are baselines: A baseline is a multi-year display of data with two parts: an historical part which shows where we’ve been, and a forecast part that shows where we are headed if we stay on our current course. Baselines allow us to define success as doing better than the baseline or "turning the curve." 

(2) Baselines serve two purposes. First they allow us to ask and answer the question: AIs this future OK? @ If the percentage of 3rd graders reading at grade level [7] has been declining for several years, Ais it OK for it to continue to decline? @ ALook where we'll be in two years if this continues! @ Most processes of serious change start with the members of the community saying AThis is NOT OK. We can do better. @ Baselines with forecasts allow you to have this discussion.

  Second, baselines allow us to assess progress in terms of doing better than the baseline. This allows us to Acount @ as progress when we have slowed the rate at which things are getting worse, before we fully turn around and go in the right direction. This stands in contrast to the usual definition of success: that things get better right away. This is often unrealistic when trends have been headed in the wrong direction for a long time. It takes time to turn the curve on such a trend line. If we do not use baselines to measure success, we set ourselves up for failure by creating unrealistic expectations of quick fixes.

  The box below displays the history part of a sample baseline prepared for the training sessions for Prop 10 commissioners sponsored by the State Commission, the California Endowment and the Foundation Consortium.


(3) Forecasting is an art, not a science. It is necessary to get the right people around the table who can talk knowledgeably about the forces affecting the baseline and what are likely scenarios for change in the future. It often helps to have a statistical expert as part of the team, But this person should not be allowed to dominate the process. The best forecasting is not about technical statistical analysis. It involves people who know Awhat =s happening on the street @ and who can create two or three believable scenarios of the likely future. Statistical folks can help this process by analyzing trends and presenting data. But do not relinquish control of the forecasting work to the statisticians. Forecasts should reflect the consensus view of key partners about where we are heading. 

 (4) TECHNIQUE: Creating a forecast for a baseline. The hard part of baselines is, of course, the future. How good are economists or meteorologists at  predicting the stock market or weather. Uncertainty is an unavoidable part of this part of the work.

Shortcut : Consider two simple ways to organize a discussion of forecast: Ask the following questions and at each step ask “why?”:

(1)   Do we think the trend will continue in the same direction?

(2)   Will it go in that direction faster, slower or about the same?

(3)   Do we think the trend will flatten out? When will it flatten out and at what level?

(4)   Do we think the trend will change direction? When? What will happen after?

  Systematic: Here is another more structured way to do much the same thing:

  Create a table with the months across the top and rows as follows:

Result: Stable community with adequate housing      
Indicator: Number of vacant houses/properties
Quarter: -4 -3 -2 -1 Present +1 +2 +3 +4
A. Begin quarter (3 months) 20 21 22 23 24 25 27 30 32
B. Changes during quarter
1. Increment rate of growth/decline
 a. Historical pattern of increase at 1 per 
1 1 1 1
2. Changes due to demographics
 a. Older population moving out 
     at faster rate
0 1 0 1
3. Changes due to economic conditions
a. Economic conditions get worse over
    the next yr.
1 1 1 1
4. Other changes
C. Forecast Path without action plan 21 22 23 24 25 27 30 32 35
D. Changes due to action plan
1. Fix-up 5 houses per quarter -5 -5 -5 -5
E. Forecast Path with action plan 22 20 17 15

  The group decides the factors to be considered in section B and the estimated value of each. So for example, a chart on forecasting vacant housing might look like this:  

Story: Robert Reichauer, former director of the OMB described 4 factors that they considered in forecasting the surplus or deficit. (NPR interview 7/18/2000). Conceptually their technique is very close to the process described above. The four factors were:

  • Performance of the economy (assume average for the next 10 years)
  • Tax policy  (assume status quo)
  • Entitlement policy (assume status quo)
  • Growth in discretionary spending (assume same rate as inflation)

(5)  Tell the story behind each baseline, or the story for all the baselines together. What are the causes, the forces at work? Why does this picture look the way it does? Why are only 67% of our children immunized at age two? Why are so few of our 3rd graders reading at grade level? Why is it getting worse or improving so slowly? This is the Aepidemiology @ part of the work. Each part of the story is a pointer to action in the next stage. If one of the reasons 3rd grade reading scores are going down is that parents don =t do a good job helping children build reading skills, then this is a pointer to parent education, support for library reading programs, campaigns to get reading material into the homes of families with young children etc. Be prepared for different stories and serious debate. As different partners add their perspectives, a rich picture will emerge about what is happening in the community and why children are, or are not, healthy and ready for school.

Remember : When considering the story behind the baselines, make sure, again to involve as many partners from as many different and diverse communities as possible. It is particularly important to involve parents in this thinking process. Those we are trying to help, often know more about the causes and likely successful approaches than the professionals.

  In doing this work, it may be useful over time to develop the indicator baselines and the story behind the baselines into a periodic report card on Achildren healthy and ready for school. @ Making this report card part of a larger report card on child and family well-being will help link Prop 10 work to other efforts in the county to improve results for children and families.

  Another type of story which needs to be told is the cost of bad results. What is the price we pay when children are not healthy and not ready for school? Such costs show up in many places: as remedial education costs in the school system, as health care costs in the public and private sector, and later in such things as teen pregnancy and juvenile crime. Showing the costs of bad results helps make the economic case for additional investments in children and their families. [8]




The Web



©1997 - 2010 by FPSI Mark Friedman
All Rights Reserved