Creating preconditions for impact measurement in the context of the programme for sustainable economic development in Myanmar
At the end of 2012, the German government entered into bilateral development cooperation (DC) with Myanmar. In order to plan measures appropriate to the objectives and to be able to measure their impact later, DEval carried out a baseline study for the DC priority of sustainable economic development. It also provided advisory support for the planning process as a further contribution to improved impact orientation. The data collection was carried out between 2014 and 2015 and the project was completed in 2016.
Findings of the Baseline Study
- The economy of Myanmar is characterised by small and medium-sized enterprises (SMEs) with fewer than 50 employees. While most representatives of SMEs expressed satisfaction with the market conditions and took an optimistic view of the future, experts assessed the private sector as uncompetitive in relation to international competition.
-
The positive assessments given by the entrepreneurs reflect the fact that they are unable to assess the competition outside their own country. The Myanmar government should hold firm to the reform process that has been initiated and ensure on the political level that reforms benefit the private sector, especially smaller companies, so that they are equipped to meet the demands of international competition.
- Myanmar's financial sector must urgently be modernised and reformed.
-
The banking sector continues to be dominated by state-owned banks, but private banks are becoming increasingly active. Currently the financial sector lacks both a basic regulatory framework based on international standards, and training programmes for banking staff. Extensive capacity development will be required before the central bank and the private banking sector can fulfil their function as the backbone of the economy and play their part in the international financial market. SMEs in particular often resort to and prefer informal sources of finance, seeing them as no less advantageous than formal sources. In principle, however, SMEs showed interest in using new and different financial services if these were useful and available to them.
- The country's education sector is highly fragmented; cooperation between companies and vocational education and training institutions is rare.
-
To strengthen the quality of the vocational education and training system and improve competitiveness of Myanmar employees, appropriate standards and certifications for qualifications must be developed. These should be aligned with international standards. As a rule, the entrepreneurs interviewed are not familiar with vocational training and prefer to hire unskilled workers from the immediate environs. The added value of qualified workers and vocational education and training must be recognised among employers and trainees if these are to contribute to improved employment conditions and better qualified SMEs.
- The system of quality infrastructure is outdated; national standards do not conform to the international requirements.
-
With the opening of markets and the expected rise in import and export activities, internationally recognised quality labels are becoming increasingly important. A functional and reliable quality infrastructure system is therefore indispensable for the development of the entire economy, and especially for the development of small and medium-sized enterprises.
The data collection was carried out between 2014 and 2015 and the project was completed in 2016. This is a summary of the results and recommendations; you can find the complete results and recommendations in the report.
Goals of the Cooperation
DEval was asked to support the process of planning the first measures within the priority area of “sustainable economic development” by contributing expertise in “evaluative thinking”. In this regard, the overarching goal of involving DEval was to strengthen the impact orientation of the programme and thus to create better preconditions for impact measurement and evaluation on the programme level.
Three areas were concentrated on during the cooperation:
- Supporting the planning of a coherent programme by jointly developing impact models on the module and programme levels;
- A comprehensive baseline study which could be used as a basis for later (impact) evaluation of the programme;
- Where necessary, supporting the development of monitoring systems or other instruments for the collection of (monitoring) data.
Background
After more than two decades of rule by a military regime, Myanmar – formerly Burma – began taking steps towards a democratisation process from 2011 onwards. As a consequence, many multilateral and bilateral donors – including Germany – decided to resume development cooperation with Myanmar after discontinuing it for many years. The first technical cooperation (TC) measures began as early as the end of 2012. Further technical and financial cooperation (FC) projects were added in 2014 and 2015. Both TC and FC are active in the priority area of sustainable economic development, among others, particularly in the sectors of vocational education and training, private sector development, financial sector development and quality infrastructure.
The Federal Ministry for Economic Cooperation and Development (BMZ) recognised the resumption of German-Myanmar bilateral cooperation in 2012 as a unique opportunity to devote greater attention to aspects such as coherence and impact orientation. The framework conditions in Myanmar were considered to be exceptional for the following reasons:
- At the time of recommencing DC, there were no existing donor activities of any kind in the field of development cooperation. Accordingly, it was possible to analyse the situation in the country without any interventions whatsoever as a baseline, with a view to repeating the same analysis a few years later and then comparing and measuring which changes have occurred as a result of which measures (impact measurement and attribution).
- Many partner countries have a diversity of pre-existing, historically evolved portfolios and cooperations. At the time of re-engagement in Myanmar, however, the German portfolio could be planned, designed and comprehensively coordinated from scratch. The aim of this was to draw on experiences from other contexts for the planning and implementation of a coherent, impact-oriented programme tailored to the partners’ needs.
- In the spirit of the Aid Effectiveness Agenda, development partners thus had the opportunity to coordinate their priorities and measures from the outset and to plan and implement them coherently.
Methods
In consultation with the stakeholders involved, DEval chose a theory-based approach in order to embed "evaluative thinking" and the creation of preconditions for impact evaluation in the programme, starting right from the planning stage. This approach consisted of jointly developing a programme theory as well as programme- and project-level impact models. As a result, it was possible to develop a common understanding of goals and underlying causal mechanisms of the programme as a whole and of its individual, constituent projects.
For each intended change in the impact model, indicators were developed, from which information needs were derived and consideration given to which existing data could be used to satisfy these needs or – if no such data was available – with which instruments it might be collected.
This information is primarily needed in order to record the baseline situation and then to measure changes over time as activities proceed, but also to serve as a basis for evidence-based decisions in the planning and steering of the programme and the projects. To gather this data, DEval adopted a mixed-methods approach involving the collection of primary and secondary data. A nationwide standardised survey of SMEs, a standardised questionnaire for bank staff and semi-structured qualitative interviews with key individuals (i.e. those directly and indirectly involved in implementation, affected persons and independent experts) were conducted. A comprehensive literature review was carried out to augment the data collected with supplementary information from secondary sources.
As the second step, to carry out the impact evaluation itself a longitudinal design was chosen – beginning with a baseline study – to ensure that comparable data was collected systematically before, during and after programme implementation. Naturally, the data and information collected can also be used for the purposes of monitoring and programme steering.
Team
- Dr Stefanie Krapp Former Head of Department, DEval
- Heike Steckhan Evaluator
Contact
Amélie Gräfin zu Eulenburg
Phone: +49 (0)228 336907-930
E-mail: amelie.eulenburg@DEval.org
![[Translate to Englisch:] Cover des Berichts](/fileadmin/_processed_/8/c/csm_DEval_SME_Report_2015_EN_526b54c772.png){kind=small}