Step 4: Identifying valuation criteria

Valuation criteria provide the frame of reference used to determine the merit or worth of a given object of evaluation. The valuation aspects considered relevant must be made measurable, i.e. operationalised. In addition, for each operationalised criterion, a criterion point (threshold value) should be established. This defined ‘turning point’ where the valuation changes from positive to negative or from positive to very positive, etc. can be described either using a number or using a textually formulated target state or intermediate state. Operationalised criteria should be determined before the start of data collection, but at the very latest before the start of data analysis. This ensures that the weighting of criteria and criteria points is not retroactively adjusted to accommodate data collection and analysis or to serve the interests of certain groups (e.g. for the purpose of justification). This is particularly important if the focus of the evaluation is to reach a fundamental decision or ensure accountability.

Establishing criteria for an appropriate valuation of a given object of evaluation is often particularly challenging. Under no circumstances should valuation criteria be set in a manner that is ad hoc, purely intuitive, methodologically uncontrolled or incomprehensible.

The following list contains four generic sources of criteria that can be used as a basis for the selection of criteria:

1. the frame of reference used by actors to orient the object of evaluation (e.g. a mission statement of an organisation or a social science theory).
2. the social consensus or discourse on law, ethics and social values.
3. a logic model enabling criteria to be established for concept consistency, implementation compliance and effectiveness.
4. Evaluation literature containing various criteria that may be transferable to the planned evaluation.

At least one operationalised criterion needs to exist for each evaluative and causal question used in the evaluation. Ideally, the definitive list of criteria (scope and weighting) is created with the involvement of the various stakeholders. The criteria should also be established in a comprehensible manner, taking into account the context and the specific characteristics of the object of evaluation.

The use of operationalised criteria and criteria points for valuation is essential for fundamental decisions and accountability purposes. The decision on exactly who determines the valuation criteria and what role the evaluators will play in this process must be determined for each evaluation. A major challenge in the evaluation process is striking the right balance between different valuation requirements and creating transparency about them.