Overview Training Facilitation Tools & Resources Research & Publications

Search

Site Map

Home

Challenges of evaluation

Why is evaluation so challenging?

  • Problems with objective setting

    • Objectives are rarely clear &/or agreed upon
    • Objectives may be too vague to measure
    • Project objectives may conflict with each other
    • Few absolutely “correct” directions
  • Natural and social systems are complex
    • Ecological processes are not well understood
    • Systems and strategies operate at multiple scales – time and geographic
    • Change is difficult to interpret
    • Hard to demonstrate causality
  • Projects at different stages cannot be held to the same standards
  • Projects may have neither motivation nor capacity to evaluate progress
    • Evaluation is uncomfortable/judgmental
    • Evaluation can lead to the need to change direction, which is hard
    • Limited time and human power:  Better to do, than think about doing.
  • As a result, our tendency is to …
    • Measure success of strategies/activities
    • Use process improvements as proxies for environmental improvements
    • Measure what is easiest to measure
    • Focus on data collection, not decision making
    • View evaluation as a process that is external to a collaboration (and conducted by others)
    • Avoid evaluation, or do it poorly

 

How do you deal with these challenges?

  • Integrate evaluation into the process of managing your project
  • Consider the motivation and capacity of project members
    • Keep it simple
    • Ensure that evaluation produces recognizable benefits to the project
      • Link evaluation to decision making
      • Celebrate learning
    • Build capacity; Access area resources
  • Tailor evaluation to your needs
    • Target measures to phases of the project
    • Focus evaluation and data collection on specific questions that are relevant to your project’s decision making
      • Understand the system/situation components and interconnections
      • Clarify objectives
      • Frame specific measures of success and comparisons
  • Use multiple measures of success
    • Ecological, social, process
    • Ultimate outcomes, near term outcomes, strategy implementation
    • Subjective, objective; Coarse, fine
  • Be strategic in choice of indicators
    • Look for keystone/integrative indicators
  • Overcome data problems
    • Take advantage of existing data
    • Coordinate others’ monitoring activities
    • Partner with area colleges or NGOs with data collection or analysis expertise
    • Seek long term funding for monitoring from EPA or a state agency in order to sustain the effort

Home | Site Map | Search | © 2009 Ecosystem Management Initiative. Terms of Use

Challenges of Evaluation" has been developed by Steven Yaffee, Sheila Schueller, and Elizabeth McCance. Copyright � 2005 Steven Yaffee, Sheila Schueller, and Elizabeth McCance, University of Michigan.