The first phase of this cooperation between AidData and DEval relied on AidData’s 2014 Reform Efforts Survey as the primary source of data, which was conducted by a team of researchers from the College of William & Mary in the summer of 2014 and included the careful construction of a sampling frame of approximately 55,000 senior policymakers and practitioners across five stakeholder groups from 126 low- and middle-income countries and semi-autonomous territories: host government officials, development partner staff, civil society and non-governmental organization (CSO and NGO) leaders, private sector representatives, and independent experts. 6,744 of the individuals who received an invitation to participate in the survey actually participated.
Given that the particular interest in evaluating the perceptions of domestic decision-makers and stakeholders who worked directly with official German development agencies in low- and lower-middle income countries, development partner officials and independent country experts were excluded from the analysis, reducing the sample to 4,455 survey participants. Of these, 1,227 survey participants indicated that they had firsthand experience working with at least one of the three official German development agencies included in the survey questionnaire —German embassies, GIZ/GTZ, or the KfW Development Bank (KfW).
These individuals’ responses were used to measure the perceived performance of official German development agencies in affecting the reform process in low- and middle-income countries based on three indicators:
- The usefulness of German policy advice provided to in-country policymakers and practitioners;
- The influence of German development cooperation on a partner government’s decision to pursue reforms; and
- The helpfulness of German development cooperation during a partner government’s reform implementation efforts.
The analysis examined variation by stakeholder group, region, and policy cluster in order to identify possible comparative strengths and weaknesses of Germany’s development cooperation in the eyes of its partners.
Building on the findings from the first study, the second evaluative study was based on analyses of the 2017 Listening to Leaders Survey and supplemented with selected country studies: Almost 2,400 people took part in the 2017 Listening to Leaders Survey. It focused on aspects such as partner-country policymakers and practitioners perceptions of donors' influence on the pursuit of particular policy initiatives or the usefulness of the support for implementing these initiatives. In a first step the data from the 2017 Listening to Leaders Survey were analysed similarly as the data from the previous study but not only for Germany’s official development cooperation but for various international donors. In a second step, country case studies were conducted with various stakeholders, in order to explore how policymakers and practitioners understand the terms “influence” in agenda setting and “helpfulness” in policy implementation. The country case studies also build a richer narrative around the presumed explanatory factors through which donors can become more influential and more helpful, and unearth additional factors that were not identified ahead of the case studies.