Development cooperation takes place in complex contexts. This presents a methodological challenge for those designing evaluations to be appropriate to their object. To meet this challenge we are refining relevant methods and standards.
In our work at the institute, we seek to achieve high scientific standards in designing evaluations and choosing and applying appropriate methodologies and to develop innovative methods for our assignments.
We collect data through a range of methods, selected according to the questions asked and in line with the triangulation approach. By combining different methods, we can ensure that the methodological weaknesses of one instrument are offset by the strengths of other instruments. Data collection methods are selected so that they complement each other, giving us a broad information picture. By employing several methods and interpreting and analysing the data through consensus approach, we are able to achieve multiple validation of our findings (data and researcher triangulation). The choice of data collection methods is not only about guaranteeing validity but is also aimed at keeping the burdens they impose on stakeholders proportionate to the expected benefits of the evaluation exercise.
The whole approach is designed to ensure that the effectiveness, efficiency and sustainability of development interventions are adequately determined so that we can use our findings to make assessments and recommendations that have been systematically substantiated and can be clearly understood.
Research on methods
Based on a growing number of DEval evaluations, the Competence Centre for Evaluation Methodology started working on core methodological questions. One key challenge centres around how to open the programmatic black box and uncover the causal mechanisms between the intervention and the observed changes on outcome.
DEval is guided in its work by international quality standards for development evaluation. We attach particular importance to the evaluation principles set out by the Development Assistance Committee of the Organisation for Economic Cooperation and Development (OECD DAC). These principles relate to the following aspects of development evaluations: a free and open evaluation process, evaluation ethics, partnership approach, coordination and alignment to partners, capacity development, and quality control.
DAC's quality standards for development evaluation reflect this changing approach and set best practices for development evaluation, by which DEval is also guided. Depending on the object and objective of an evaluation, we accept, as far as possible, the DAC standards as our central criteria and apply them to our own evaluations. These criteria can be summed up as: relevance, effectiveness, efficiency, development impact and sustainability.
By conducting evaluations, DEval helps to refine basic standards. To this end, it uses – among other things – a framework format for complex, strategically relevant evaluations that is based on high, internationally accepted quality requirements. One example of this is the process description of DEval's evaluations for reference groups. The workflow chart clarifies the role and function of reference groups in DEval's evaluations under idealised circumstances.