Evaluation Research and Publications
EMI faculty and staff are committed to learning from and influencing the practice of ecosystem management. With a core focus on evaluation and adaptive management, we engage in research to better understand the needs and capabilities of people to practice evaluation and adaptive management. We also remain self-critical about our own work in evaluation and continue to assess the effectiveness of our training and facilitation programs and tools. Below, we highlight examples of past and ongoing research initiatives in evaluation and adaptive management.
- How do organizations evaluate and what role does evaluation play in program success? Since 1995, EMI researchers have conducted a longitudinal survey of ecosystem management projects across the United States. The goal of this research has been to report on how projects have undertaken ecosystem management and to educate practitioners on the types and breadth of experiences seen. One key research question concerns how organizations evaluate their progress and what role evaluation can play in a project’s success. The results of our initial survey in 1995 of 105 projects across the United States were published in the book: Ecosystem Management in the United States (Island Press, 1996). The findings from our second survey in 2000 are available in the report Recent Trends in Ecosystem Management. Click here for the evaluation-related research findings from our most recent survey in 2003.
- What lessons have we learned from case studies of practitioners engaged in evaluation? In addition to our longitudinal study, EMI researchers have conducted two sequential studies to understand the level of outcome-oriented evaluation and assessment carried out in a sample of collaborative ecosystem management projects. A first investigation looked at 5 typical cases in the Great Lakes states. A second examination specifically sought cases that had engaged in fairly extensive evaluation. From both, we developed an understanding of the challenges of evaluation and the level of evaluation and monitoring work occurring at each site. This research has also helped us explore ways to expand the capacity of sites to evaluate their progress.
- How have other authors written about evaluation and what tools have they made available? To develop tools that would assist in evaluation, we researched and synthesized the rich academic and applied literature on the goals of ecosystem and community-based efforts, and the use of indicators to measure progress towards desired outcomes. From this research, we drafted three working papers – dealing individually with issues related to ecological systems, human communities, and collaborative processes. Click here to access these three working papers.
- What have we learned from other organizations applying the EMI planning process? EMI trainers utilize the EMI four-stage evaluation planning process,Measuring Progress, to help build the capacity of practitioners to evaluate their own organizations and programs. To test the value of our process, we held nine evaluation training workshops for organizations that work in an ecosystem management context. The trainings occurred across North America and reached participants from diverse organizations in the public, private, and non-profit sectors. In the summer of 2005, we invited participants to complete an anonymous online survey to learn of their long-term impressions of the workshop and how they plan to use or adapt EMI’s tools and concepts.
As a related investigation, a Master’s student at SNRE conducted a detailed study of the impact of our evaluation trainings and facilitation. Click here for more information on this study.
- What is an effective process for developing an evaluation and adaptive management project? Researchers at EMI have spearheaded numerous initiatives to learn about the challenges and benefits of evaluation, the evaluation tools and resources available to practitioners, and to highlight examples of organizations that have utilized successful and pragmatic approaches to evaluation and adaptive management. This research has laid the foundation for our four-stage evaluation planning process, Measuring Progress, a process that our trainers use to facilitate evaluation workshops and long-term processes. A text version of Measuring Progress is available to download. Click here to learn more about Measuring Progress and related worksheet templates and tools.
This research has also provided the foundation for our Evaluation Sourcebook, a text-based compendium of examples of evaluation questions, indicators, and data sources relevant to measuring progress in ecological, socio-economic, and organization/process themes. Click here to learn more about the Evaluation Sourcebook.