Responses to Questions... by Mark Friedman

Index of Bulletin Board Questions Latest entry: 12/8/02

 

This is a place where anyone (anyone at all) may pose a question, and I will post my best shot at an answer. Hopefully these will be questions about Results-Based Accountability, but other topics will be entertained. I will do my best to respond promptly. This space may also address questions which come up during site visits, workshops and coaching sessions. And I may just make up some questions so I can answer them.

 

We, here at the Raguide Central Office, hope that this Bulletin Board will stimulate debate and will help connect people working on similar challenges. To further the business of connecting people, it would be helpful if you could include contact information, and indicate if it's OK to show this information with your question. In some cases, the questions will be edited or restated so they make sense to a broader audience. Let me know if you have a problem with how your question is edited, or have comments about the answer, or have a follow-up question.

And I welcome other "answers" to these questions, which I will post, subject, of course, to my absolute, final, and sometimes ruthless editorial discretion.

---

We start with some questions and answers from previous correspondence.

Post date: 12/8/02 
Question date: 3/22/02
Questions from: Vermont Carol Maloney
25 St. Paul Street
Montpelier, VT  05602
phone  - (802) 229-4564
email - vtpartners@vtlink.net

 

QA1.1 What about Short term, Mid term, Long term ( Very long term, Eternity)?: Where/how does the RBA model help delineate these different outcome time lines, commonly referenced in logic models and other frameworks?

Here’s the “simple” answer first: Results are end conditions and almost by definition long term (and insofar as they are about all children, possibly eternity). With regard to indicators, I do not use ST/LT (temporary acronym) because I think we have too often in the past used these labels as excuses for not doing things with any sense of urgency. If you remember the Boston story of turning the curve on juvenile homicides, it would have been easy for the group at the table to say that this is really a long term effort and we shouldn’t expect too much too soon. But instead, with about 15 to 18 months work, they accomplished what some would have said was impossible, a 2 and one-half year period with NO juvenile homicides. Many places have turned around childhood immunization rates much faster than anyone thought possible. So I am trying to eliminate a reason to go slow, to be less ambitious, to settle for half hearted effort. The formulation that I offer instead is: “Identify the indicators that truly tell you whether you are getting the results you want, and work to turn the curve on those indicators just as fast as you can.” Several times, people have come up to me and said something like: “This is too hard. Instead of these long term results can’t we find shorter term, easier things to work on?” Of course, strategies will all be about nearer term actions and objectives. But if we apply this need for easier and nearer to the notion of end results, it becomes a slippery slope. We find easier and easier, closer and closer things to work on and eventually we are working on readily achievable things, but they are not very important. The fundamental idea of results and indicators is that we honestly face up to what’s really important, not what readily achievable.
Now to some of the nuances: There will be many “secondary” indicators that may (should) show progress before the headline indicators change. Logic models create chains of such measures. “If we do a good job on mentoring then we should see better grades which will lead to higher graduation rates which will lead to less poverty.” This kind of thinking is useful in two ways. It can help us identify secondary indicators (those that don’t make the headline short list) to watch and use to track progress. And perhaps more importantly, logic model thinking can be used to test our ideas about what works. In the part of the process where people are coming up with ideas about how to turn the curve(s), the question that should always be posed is “Why do you think that would work?” And causality chains are one way to answer that question. (As you may remember, logic models work in the opposite direction as results based decision making, typically: action affects cause (story behind the curve) affects indicator leads to the result.). And so logic model thinking should be another tool in the tool kit to help people craft and test their ideas about strategies to achieve results.
Finally, the notion of short tem, mid term and long term is a useful way to structure the actual action plan. (See 2.13) Rather than use ST/MT/LT I recommend the more practical: this year, next year, 3 to 10 years, which tends to mirror how budget and other decisions actually get made. If you look at the criteria for choosing actions from a long list of possible what works ideas, you will see “specificity, leverage, values and reach.” This last criteria, “reach” is about staging work over this kind of multi year period. (The use of the word “reach” here is a deliberate reference to Lizbeth Schorr’s “Within Our Reach” wherein she argued that we really know a lot about what works, and our failures are most often failures of will, not knowledge.)

QA1.2 One group or many doing the work of moving from talk to action?: Many different groups have created a set of "results," Would you recommend a single group walking through the talk to action process? OR, can different groups take each subsequent step and further the plan?

This is ultimately a “political” question more than a technical one. What approach would work best with the people you are working with in your process? The Results-Based Accountability framework is a method of deciding on and taking action. The notion of people around the table committed to doing this is central to the work. But which people around how many tables? Any action plan must have structure and coherence to have any chance of working. In fact I define strategy as a “coherent set of actions that have a reasoned chance (again logic model idea) of turning the curve.”
In order to create such a coherent plan you need to balance the chaos of coming up with good ideas and tapping every possible source of energy in the community with the need for some orderly process that leads to action quickly and can be tracked and improved over time. So I would try to create a space (physical and otherwise) where such a strategy can be created. I recommend the idea of a map room (like the Churchill War Rooms “map room” we discussed). In this space a group of people (maybe constant, more likely changing/evolving) meet regularly both to craft strategies and actions and also to see if things are working. But, in addition to this, I would let a thousand flowers bloom. I have come to believe (and this is something of a leap of faith) that there is ultimately more power in messy organic processes than in tightly controlled autocratic ones – provided (and this is important) that there is some place where a single groups makes sense of it all.
So in some ways I am suggesting that you do both things. Allow many to work on this. But also identify a single group to run the map room.

QA1.3 Logic Model – Results Decision making: Which process to use?: What is the primary difference between "the whole distance from results to what works" and "turn the curve exercise?" As I've mentioned, I've ultimately got to develop a logic model that includes immediate, and long range outcomes. The outputs and activities are likely to generate some heated discussion (with budget cuts these issues seem to be very contentious). I'm wondering which process, if they are substantively different, would work best. Will they both get me the details I'll need to complete a logic model?

First, your question about the difference between the two exercises: The “Whole Distance” exercise on www.raguide demonstrates the entire thinking process quickly in a large group setting. The “Turn the curve” exercise allows small groups to learn by doing. In both cases real work is done. Whole distance deals with the all results at once. Turn the curve focuses on one result and one indicator.
If you use the results model to develop a strategy, then you can use logic model thinking to test the strategy (see above) and/or to present the strategy if that’s the way your audience is expecting to hear it. (Although it is almost certain that your audience will easily understand any RBA presentation. Logic models have no lock on logic and RBA is very logical.) You run a certain risk, however, developing a strategy one way and presenting it another, of having the worst of both worlds. I have seen some very bad processes where people tried to mix and match frameworks. They would usually have been better off picking one or the other.
Budget cuts will always involve heated discussion. I was a budget director for many years and have the scars to prove it. Budgeting is about choices. RBA is about creating better choices, not dictating a formula for deciding which ones to make. So how can we do the best we can with what we have to achieve these results? Remember that no-cost, low-cost ideas, non-programmatic actions, and partners’ contributions are enormously important to this thinking process, and can help take the edge off the fight for money. In the end, healthy disagreements are just that, healthy. And by always going back to the results, people can find (again) their common ground, and differences about resources etc. are put in perspective as differences about means, not ends.
Finally, the best way to learn this stuff is to practice it. If you can, find a small group of friendly faces who will let you work with them before the higher stakes meetings.
See also 3.9
Another set of thoughts about Logic Model
The difference between RBA and Logic Model frameworks is partly about what question(s) they are trying to answer. The question answered by RBA is: "What set of actions by our partners can measurably improve the well-being of children and families?" The logic models you are looking at generally are designed to answer the question: "How does this program or set of program activities lead to improved conditions of well-being?" The starting point for RBA is the results. The starting point for logic model work is a program, service or set of actions.
Now here’s why this difference is important. Starting with programs (services or actions) and testing the logic of their contribution to improved customer results and ultimately population results is a useful thing to do. It is in fact something that is naturally done when considering “what works” in RBA. But I believe that it is insufficient as the overall framework, because it starts with programs, and that approach is too narrow. Much of logic model work grows out of the old way of thinking that is program-centric, assuming that programs are the solution to everything. RBA, because it starts with the end conditions we hope to achieve, leads to a much larger and richer set of solutions than just programs. (Jolie Pillsbury has a nice way of describing this as a Copernican revolution in social policy. We once thought that results revolved around programs at the center. And we now know that programs revolve around results at the center.)
Logic model thinking can be useful in program design, and in helping people figure out how and why their program works. It can also help figure out how to explain how a program works and why it should be funded. But it can also be very tedious, time consuming and paper intensive. I have more than once heard people tell of working on their logic models literally for years. RBA is a much faster and more direct way to do the work. All of the logic model detail is not necessary when you can use RBA to go directly to customer outcomes in less than an hour and begin using them immediately to improve performance.
In summary, I would use RBA to decide what to do and use a logic model to test why you think it has a chance of working. You should then easily be able to fill out any funder’s application form from this base. Or, using the crosswalk you can do it one way and translate to the other. What I would not try to do is create a hybrid, since you risk ending up with something that makes less sense than either approach by itself.