Although there are no “right” or “wrong” evaluation criteria, there are better and worse ones, or at least more useful and less useful ones. The characteristics of good evaluation criteria are (adapted from Keeney and Gregory, 2005):

• Accurate and Unambiguous, meaning that a clear and accurate relationship exists between the criteria and the real consequences.
• Comprehensive but concise, meaning that they cover the range of relevant consequences but the evaluation framework remains systematic and manageable and there are no redundancies.
• Direct and ends-oriented, meaning they report directly on the consequences of interest and provide enough information that informed value judgments can reasonably be made on the basis of them.
• Measurable and Consistently Applied to allow consistent comparisons across alternatives. This means the criteria should be able to distinguish the relative degree of impact across alternatives. It does not exclude qualitative characterizations of impact, or impacts that can’t be physically measured in the field.
• Understandable, in that consequences and trade-offs can be understood and communicated by everyone involved.
• Practical, meaning that information can practically be obtained to assess them (i.e., data, models or expert judgment exist or can be readily developed).
• Sensitive to the Alternatives under consideration, so that they provide information that is useful in comparing alternatives.
• Explicit about Uncertainty so that they expose differences in the range of possible outcomes (differences in risk) associated with different policy or management alternatives.

While not a strict requirement, it is good practice to check that the criteria are also additive – or more formally, preferentially independent (Keeney, 1992). This means that they contribute independently to the total performance of an alternative. When criteria are preferentially independent, simplified decision modeling tools can legitimately be used. And because preferential independence is almost always implicitly assumed, it is best to make sure the assumption is valid to avoid errors of logic. If it is not valid, then more complex analysis is required. There will be trade-offs to make in selecting criteria. For example, the most “direct and ends-oriented” criteria tend to be less “operational’ as they are difficult to estimate or model. The most “accurate” may not be understandable to non-technical decision makers.

Key Ideas

• Good evaluation criteria share a number of characteristics
• Evaluation criteria should accurately reflect differences between alternatives
• All participants should be able to interpret the evaluation criteria in the same way