In consultation with the stakeholders involved, DEval chose a theory-based approach in order to embed "evaluative thinking" and the creation of preconditions for impact evaluation in the programme, starting right from the planning stage. This approach consisted of jointly developing a programme theory as well as programme- and project-level impact models. As a result, it was possible to develop a common understanding of goals and underlying causal mechanisms of the programme as a whole and of its individual, constituent projects.
For each intended change in the impact model, indicators were developed, from which information needs were derived and consideration given to which existing data could be used to satisfy these needs or – if no such data was available – with which instruments it might be collected.
This information is primarily needed in order to record the baseline situation and then to measure changes over time as activities proceed, but also to serve as a basis for evidence-based decisions in the planning and steering of the programme and the projects. To gather this data, DEval adopted a mixed-methods approach involving the collection of primary and secondary data. A nationwide standardised survey of SMEs, a standardised questionnaire for bank staff and semi-structured qualitative interviews with key individuals (i.e. those directly and indirectly involved in implementation, affected persons and independent experts) were conducted. A comprehensive literature review was carried out to augment the data collected with supplementary information from secondary sources.
As the second step, to carry out the impact evaluation itself a longitudinal design was chosen – beginning with a baseline study – to ensure that comparable data was collected systematically before, during and after programme implementation. Naturally, the data and information collected can also be used for the purposes of monitoring and programme steering.