Generally, proxies tend to be overused, so be on guard. But they can also be extremely useful. A practical approach involves:
- Use proxies at the scoping stage (they are generally much easier to estimate than the corresponding natural criteria).
- Based on preliminary analysis, eliminate, if possible, some proxies from further consideration (because they are insensitive to the alternatives for example, or closely correlated with another criterion).
- Be explicit about the range of possible and/or presumed relationships between the proxy and the endpoint/objective.
- When using one proxy to represent multiple objectives, make sure there are no hidden trade-offs.
- If trade-offs involving a proxy are pivotal to the decision, use a structured expert judgment process to better understand the relationship of the proxy to the endpoint/objective.
Conclusion
- Use conceptual models such as influence diagrams to help identify appropriate criteria and focus information gathering and model building efforts.
- Use natural criteria if you can. Consider expert judgment to estimate them if modeling is not available.
- Use constructed scales as a second choice. If they are carefully constructed, they are surprisingly effective.
- Use proxies freely at the scoping stage. Use them cautiously at later stages if they are key to final choices about difficult trade-offs. They are surprisingly problematic.
- There are no right or wrong criteria, but there are good ones and bad ones. Know the limitations of the criteria you use.
Key Ideas
- Be cautious when using proxies: there are a number of pitfalls associated with them
- Natural attributes make the best evaluation criteria, followed by constructed attributes, and then proxies.