How is the EMI Evaluation Process Different?

To develop the EMI evaluation planning process, researchers at EMI interviewed several leaders of ecosystem and community-based projects to solicit their views on the benefits and challenges of evaluation and their ideas for tools to overcome the challenges. Researchers also reviewed numerous articles/books on evaluation, particularly with respect to the evaluation of ecological and social outcomes and process/organizational characteristics. Some of these resources provide comprehensive planning and assessment guidance for organizations with overlapping environmental and socio-economic goals. The EMI researchers drew heavily from these and other bodies of existing knowledge to develop their evaluation planning process; the EMI process, however, is also purposefully different from these resources in the following ways:

• Our process focuses on ecological and social outcomes, and the process characteristics that enable environmental outcomes to be achieved.  Most of the organizations that have piloted evaluation processes focus primarily on biodiversity conservation as an overriding objective.  A more integrative process was necessary because most ecosystem management projects focus on a broader set of ecological objectives as well as social outcomes (such as building community economic health and maintaining local culture) and process improvements (such as building communication mechanisms and collaborative networks);
• Our process utilizes a scaleable, multi-metric approach that encourages projects to define measures of success that focus on outcomes, changes in threats or available assets, and activities or strategies. When tracking success on goals is difficult or takes a long time, an evaluation of whether you have managed to reduce threats or follow-through on strategies can provide intermediate measures of success.
• Our process introduces and describes evaluation as an integral component of a project’s management process and not as a stand-alone task or one undertaken at the “end of the pipe.” Hence, we seek to improve a project’s management processes by defining evaluation as a set of activities that are integral to ongoing management and that can be implemented by project staff and volunteers on their own so that they are not dependent on outside consultants or auditors;
• Our process recognizes that evaluation translates into adaptive learning only if the results of monitoring are tied back into the decision-making processes of an ecosystem management project. Hence, our process encourages groups to be more effective at carrying out the full evaluation process, particularly the steps that lie beyond defining a situation map (objectives, threats, strategies) and defining indicators, including defining information collection plans, trigger points, and ways to tie learning back to decision making;
• Our process tries to be sensitive to the limited resources and skeptical outlook toward evaluation held by many ecosystem management projects and the various organizations that participate in them.  Hence, we have sought to define a process that can adapt to available on-site resources (information, expertise, staffing).