Past Facilitated Processes

August 2005, Atenas, Costa Rica (Tropical Forest and Agriculture Ecosystem)

In collaboration with SFS staff, EMI facilitated a second SFS center in the development of an evaluation plan. The Center for Sustainable Development Studies focuses on research related to sustainable agriculture and improved management of protected areas in Costa Rica. This workshop, which was conducted in both English and Spanish, built on existing collaborations between SFS and local decision-makers by actively including representatives of the Ministry of the Environment and national park staff. In this and the previous workshop EMI applied a streamlined process of identifying evaluation questions and indicators that would allow for site-specific indicator development but also a consistent framework across centers. 

We used the Costa Rica workshop to test the cross-cultural utility of the EMI evaluation process, and as an experiment in “training the trainer,” by sharing facilitation duties with the SFS Dean of Programs.  Through discussions after the workshop and in Ann Arbor two months later, we identified key challenges for other people in facilitating this process and discussed ways to structure training programs to deal with these challenges.

July 2005, South Caicos Island, British West Indies (Marine Ecosystem)

The School for Field Studies (SFS), a organization that offers field-based undergraduate study-abroad program, was seeking a process to evaluate the effectiveness and impact of each of their five centers' 5-year research plans (5YRP). These plans are designed to organize the center's research, education and outreach activities, including planning, conducting and disseminating research to assist local communities and decision-makers in solving resource management problems.

Through a two-day on-site workshop, EMI facilitated development of an evaluation plan for the Center for Marine Resource Studies in the Turks and Caicos Islands. This center works together with local fishermen and the Department of Environment and Coastal Resources (DECR) on issues related to the over-exploitation of reef fish and the potential ecological and socioeconomic impacts of tourism. Participants commented that the process, especially situation mapping, helped them gain a clearer and shared understanding of the 5YRP. At the same time, this workshop served to train higher-level SFS staff in how to facilitate the evaluation process at other centers.

June 2005, Sacramento, California (Coastal and Marine Ecosystem)

As a follow-up to the Packard Science & Conservation Program workshop, EMI worked with CCMI, one of the Program’s regional initiatives. Its coordinator had participated in the Packard workshop and was familiar with the process, but wanted EMI to facilitate CCMI in creating a logic model, which could be provided to Packard. The group identified the need to “go deep,” to “refine, reshape, and validate” their objectives and approaches.

CCMI had a rich set of existing planning and reporting documents, and so EMI prepared a draft map from these to jumpstart group work. Over 1½ days, we revised the map and developed “story lines,” which helped participants clarify and communicate the logic of the program, and developed a draft evaluation framework. We helped CCMI present the draft map and story lines to the Packard Foundation staff, and discussed this approach to logic modeling and evaluation.

To date, CCMI is working on prioritizing and refining their questions and indicators and specifying targets and trigger points for indicators. These will then be worked into a model that combines the traditional logic model with the threats/assets component of the situation map.

March 2005, Seattle, Washington (Fisheries, River/Coastal Ecosystem)

Members of the Cedar River HCP implementation team at Seattle Public Utilities (SPU) began working with EMI’s evaluation process on their own using the Measuring Progress Guide, which had been recommended to them by a co-worker at SPU. They sought advice over the phone from EMI about how to use specific elements of the process, then sought more in-depth facilitation from EMI.  Their goals was to complete an adaptive management plan that would meet legal commitments, and would also provide a “balanced and objective approach…to step back and see how we are doing.” Although SPU had existing research and monitoring programs underway, they lacked a framework that would allow them report on their overall effectiveness.  For EMI, the effort was an experiment in extending our evaluation process deliberately into the realm of adaptive management.

While participating in the Puget Sound training workshop (see trainings), the group developed a draft situation map. We then worked on refining the map and compiling and prioritizing evaluation questions and indicators.  To date, they have developed implementation and effectiveness questions and measures and are awaiting feedback from their HCP Oversight Committee before specifying logistics of implementing their evaluation plan.

An interesting addition by this team to the assessment matrix was a column on “Likely causal mechanisms” of a change in an indicator to help interpret change and distinguish the impact of the group’s efforts from others.

Ecosystem-Based Management Science Subprogram
November 2004, Los Altos, California (Coastal/Marine Ecosystem)

This Packard program, which supports research and regional initiatives in ecosystem-based management of coastal-marine systems, was seeking a process to measure program effectiveness and impact. Prior work had produced a draft logic model and metrics, but both needed refinements, and program officers wanted to try the EMI process as a way to move forward.  Over two days we refined a situation map based on existing documents and developed a set of evaluation questions and indicators.  To organize and prioritize these, we developed overarching questions.  The mapping effort highlighted threats and assets that had not been captured in the existing logic model, and pointed to the importance of information dissemination as a somewhat undefined change strategy.  The evaluation process also pointed to the need for a parallel evaluation planning process at the Program level, to highlight interconnections and put various evaluation efforts into common form.

Since the workshop, Packard program officers used the evaluation products to refine their original logic model and metrics. 

September 2004, Catalina Island, California (Chaparral and Sage Scrub Ecosystem)

The director of the Catalina Island Conservancy wanted to develop her organization’s evaluation processes as part of an overall re-visioning process of the organization. The Conservancy had a need to bring together the monitoring and evaluation work they had done in “bits and pieces” into a “big picture” that included “biological needs and socioeconomic context.” They also sought to clarify a shared vision across Conservancy staff as well as residents of the island.
Over two days EMI worked first with a broader audience (see trainings) to introduce the process, and then with key staff of the Conservancy to refine a situation map based on their extensive planning documents and to develop a comprehensive evaluation framework. 
Since the workshop, the Conservancy has lost two key leaders limiting the buy-in and capacity to continue developing the evaluation plan for now. Nonetheless, they have used the map as an informational tool for potential new employees, and discussed using the map as part of a public input process.

August 2004, Ashland, Oregon (Forest Ecosystem)

The Applegate Restoration Demonstration Project, one element of a broad set of activities underway in the Applegate Valley Adaptive Management Area, aims to demonstrate a range of strategies for restoring forest health and reducing fire hazards. One member of this collaborative attended the RCC Bozeman training and requested that EMI help the group develop indicators to document the social, economic, and ecological implications of different restoration strategies. 

Rather than starting from scratch to develop a situation map, EMI drafted a map from planning documents already created by the group.  During the workshop, participants had the opportunity to revise the map, discuss strategic planning questions, and draft evaluation questions and indicators.
Following the workshop, an indicator team (which included only one of the workshop participants) developed a framework of indicators for the project, This new framework did not directly use the workshop map or evaluation questions, but did use other tools presented in the workshop, including worksheets specifying data needs, sources and uses.

April 2004, Elko, Nevada (Rangeland Ecosystem)

Having participated in the RCC Bozeman training workshop, the president of NNSG sought EMI’s help to develop an evaluation framework that would help her group measure progress of its sage grouse conservation strategy as well as weigh its strategic options in general.  As a collaborative of agencies and ranching, conservation, and business communities, NNSG was also especially interested in assessing the effectiveness of the group’s process, including leadership and collaboration.

Over a 1½-day workshop EMI facilitated the group’s work at developing a situation map, identifying priority activities, assessing its organizational strengths and weaknesses, and creating a draft set of evaluation questions and indicators.  Through group discussions NNSG then prioritized four key overarching evaluation questions and are using these to guide strategic planning discussions. As with CRGA, the head of NNSG remarked that, “Evaluation has matured our thinking about what our next steps should be.”

March 2004, Invermere, British Columbia (Montane Valley Ecosystem)

CRGA requested that EMI help the group systematically assess their efforts to maintain a connected network of open space and wildlife habitat in the Upper Columbia Valley. Specifically they had a need “to be able to be clear with funders about where we’ve achieved objectives, where we haven’t, and why.” Because two CRGA members participated in the RCC training in Bozeman, they were able to jumpstart the process by leading other members in developing a draft situation map.  EMI then helped CRGA complete a refined map as well as a draft framework of evaluation questions and indicators over a two-day on-site workshop and follow-up contact.

Due to turnover of a key CRGA team member as well as overall limited capacity (this largely volunteer group board has only one part-time paid staff), they have not been able to implement their plan to date.  However, it appears that simply going through the process helped them to link evaluation and planning. As one participant said, “Evaluation is an aspect of whatever we do now. We never thought about it [before]…It’s a much more tangible concept for us. "