|
In SDM, evaluation criteria are used to characterize the
degree to which different alternatives are expected to meet
objectives. They are used to:
- compare alternatives accurately and consistently;
- expose trade-offs including trade-offs among different
degrees of uncertainty;
- generate productive discussion about better
alternatives;
- prioritize information needs;
- communicate the
rationale for and improve the transparency of decisions.
It isn't easy to define good evaluation criteria that are
widely agreed upon by stakeholders, experts and decision makers.
However, the up-front investment pays off in streamlined
decision making, for two principal reasons:
- because data, modeling and expert judgment processes are
focused on producing decision-relevant information;
- because large numbers of very complex options can be
consistently and efficiently evaluated by multiple decision
makers.
Brainstorming evaluation criteria
Each lowest level objective in the hierarchy will need an
evaluation criterion. There are a number of things to consider
in selecting or designing good criteria which we cover in the
next section. However, the first step is simply brainstorming an
initial list of candidate criteria. It's easiest to know what
criteria will be useful if you have a clear idea of the
alternatives under consideration. (In fact most often you can't
really identify useful criteria at all without this knowledge!)
Therefore, a good approach involves:
- Brainstorm a list of alternatives. Ask, what are all the
possible ways we could achieve these objectives? You don't
need the details of the alternatives at this point, but you
do need to identify the full range of them;
- Sketch out a consequence table, with the objectives listed
in the rows and a few sample alternatives shown in the
columns.
- Ask, "What specific metric could we use to report the
impact of these alternatives on this objective?" Or
"What specific information would you like to see to be
able to evaluate the impact of these alternatives on this
objective?"
- At this point, write down all the possible responses. It
is possible that the responses will reveal that some people
have different interpretations of the objectives. This is
good. One of the key reasons for structuring objectives is
to build common understanding and improve communication.
Refine the objectives if you need to.
For complex systems (biological, economic or social), it will
be helpful to have or to develop influence diagrams to show the
major factors affecting the objective. Useful evaluation
criteria are often found in the influence diagram.
Next>> |
Evaluation criteria are required for comparing and
making tradeoffs between alternatives