A decision-making model as an assisting tool in designing evaluation strategies for integrated rural development programmes

The study objective is to delineate, apply and revise a decision-making model aimed at helping evaluators plan and carry out effective and meaningful evaluations of multisectoral development projects.
To reach this objective, in the first part of the study a literature review is carried out to categorise existing project evaluation techniques and to identify crucial criteria for evaluation design.
On the basis of the literature review, a first version of the decision-making model for evaluation design is developed. Such a model aims at planning effective and meaningful evaluations with regard to the context, applying an elimination / selection strategy across a set of criteria in a stepwise process. The steps are:
1) Criteria setting;
2) Context analysis within the criteria framework;
3) Selection of the techniques, in order to discard the least suitable techniques and accept the most appropriate methods on the basis of the criteria and of the context;
4) Integration of selected techniques in a structured evaluation design and analysis of the feasibility of the design on the basis of the available resources; if the available resources are not sufficient, some parts of the planned evaluation have to be “pruned”, maintaining the techniques that better respond to the evaluation objectives and the project characteristics.
In order to test it, the model is applied to the Emilia-Romagna Integrated Mediterranean Programme, a multisectoral integrated development programme which was carried out in the Italian Region of Emilia-Romagna and whose goal was to favour endogenous development of the disadvantaged Apennines area. Among others, a measure of the Programme included the evaluation of the Programme itself by an external expert.
The final part of the study compares the theoretical evaluation design, resulting from the application of the model to the Emilia-Romagna Integrated Mediterranean Programme, with the design of the actual Programme evaluation that was carried out.
This comparison shows the strong points of the model:
a) Great attention to the client’s needs and expectations;
b) Application of inside analysis together with outside analysis on the ground of the context examination;
c) Possibility to choose from a reservoir of techniques in order to find the best match between evaluation objectives, project characteristics and evaluation methods;
d) Logical soundness and ease of understanding of the sequence of the aspects to be evaluated: performance-effectiveness-impact-efficiency.
Some weaknesses of the model are also revealed:
a) The effects of project activities on the natural environment receive inadequate attention;
b) The internal and external coherence of the project is not considered as a necessary element to assure project success;
c) No indication is provided on the kind of information that should be obtained and on the characteristics of the evaluation results.
Finally, on the basis of the conclusions drawn from this comparison, the model is reviewed and improved in three areas:
1. The impact on the natural environment;
2. The analysis of synergy;
3. The nature of the information and of the evaluation results.
The model that is obtained is not a definitive scheme to be strictly followed, but rather a framework, a “dynamic” tool for evaluators that can be adapted to different contexts and to evaluation team’s personal experiences. Applications of the model to different programmes and situations should lead to its further development and improvement.