The impacts of Covid-19 interventions on health-related outcomes
Low- and middle-income countries are particularly at risk during the Covid-19 pandemic due to shortages of skilled health personnel, technical healthcare equipment and infrastructure, and lack of funds to implement preventive measures. Robust evidence on the effects of health and pandemic control interventions can guide the cost-efficient and ‑effective use of funds used for measures against COVID-19 and its secondary effects.
Even though a vaccine against Covid-19 has been developed, it will still be some time until a majority of the population can be immunised and until there are clear treatment protocols. Therefore, slowing the spread of the virus is still essential to reduce deaths and to avoid overloading healthcare systems. Low- and middle-income countries (LMICs) are particularly at risk due to shortages of skilled health personnel, technical healthcare equipment and infrastructure, and lack of funds to implement preventive measures.
Robust evidence on the effects of health and pandemic control interventions can guide the cost-efficient and ‑effective use of funds used for measures against COVID-19 and its secondary effects. However, the novelty and rapid transmission of the disease have so far left little time for rigorous impact evaluations. The first relevant scientific evidence has mainly relied on non-experimental designs such as simulation exercises, event studies including before-and-after studies and quasi-experimental studies, which compare outcomes of pandemic control interventions with non-intervention or a so-called “control” group.
Which pandemic control measures are effective in reducing the spread of Covid-19?
Early policies and recommendations sought to halt the spread of the disease through the reduction of social interactions. The cancellation of public events and gatherings seem to be the most effective measure according to a study by Askitas, Tatsiramos and Verheyden (2020). In the US, states that implemented shelter-in-place orders had 44 per cent fewer Covid-19-related fatalities (Dave et al. 2020).
For LMICs, however, more nuanced approaches are needed to balance the costs and benefits of minimising social interaction. In many LMICs weak social security systems, fiscal incapacities for substantial government transfers, little household savings, and limited teleworking capacity, force people to run the risk of infection and participate in face-to-face labour market interactions. To mitigate the economic effects of blanket lockdowns, Alon et al. (2020) argue in favour of stay-at-home orders and social transfers solely for the elderly.
However, “Stay home, stay safe” recommendations are challenged by limited access to clean water and hygiene facilities in low-income households, combined with overcrowded, multigenerational living arrangements. Here, cluster-based responses are one possible solution. Oshitani et al. (2020) argue that most transmissions were spurred by a small proportion of cases, which led to cluster formations (e.g. in enclosed or crowded spaces). Tracing contacts of infected people can help to avoid clusters and therefore has the potential to mitigate transmission. However, effective contact-tracing needs reliable data. In an ongoing large-scale RCT, Avdeenko et al. (2020) track the spread of the disease over time and space in rural Pakistan using frequent, short phone interviews, and test whether geographical clusters can be identified.
To address the trade-off between health risks and economic burden, non-pharmacological interventions such as hand hygiene, wearing face masks, and social distancing could help people resume their economic activities safely. The effectiveness of face protection and social distancing has been shown in a meta-analysis of 44 observational studies, mainly conducted with health-sector personnel. Including respiratory infectious diseases such as SARS and MERS as well as Covid-19, Chu et al. (2020) provide evidence that eye protection, face masks, and social distancing of over one metre do not fully eliminate but effectively reduce the risk of infection. Mitze et al. (2020) use a synthetic control method approach to show that the mandatory wearing of face masks in Jena, Germany, slowed the daily growth of new infections by 40%.
Social norms, motivation and habit play important roles in adherence to safe hygiene behaviour (Powell-Jackson et al., 2020; Curtis et al., 2011). Awareness raising can be a powerful tool to promote preventive healthcare behaviour (Dupas 2011). A pre-pandemic meta-analysis showed that text messages (SMSs) can be useful to induce behavioural change (Orr and King, 2015). In the context of Covid-19, Falco and Zaccagni (2020) sent SMSs in Denmark to remind people to stay at home. The messages increased recipients’ intention to not leave the house, but did not translate into action. Banerjee et al. (2020) conducted an RCT in West Bengal, India, on the effectiveness of Covid-19 information campaigns. They show that an additional SMS from a prominent public figure emphasizing the importance of complying with the rules notably increases the desired behaviour. Study participants’ mobility reduced, handwashing and mask-wearing increased, and more symptoms were reported to local health facilities. The harm public messaging can do is examined by Ash et al. (2020), who show a causal link between Covid-19-sceptic programmes on Fox News and non-compliance with social distancing rules. Now that a vaccine has been developed, further awareness campaigns will need to quickly address potential fears and misconceptions that could prevent take-up.
Finally, evidence from earlier studies on other viral diseases such as Ebola and HIV can inform about potentially successful responses to Covid-19. What seemed to matter then was timely strengthening of health systems, e.g. maintaining essential health services, access to infection-prevention and ‑control measures, testing and treatment, especially for displaced populations, fast and flexible production of protocols, timely shift of resources, establishment of new communication technologies and innovative community engagement, and standardized data collection (Jefferson et al., 2011; Etkind et al., 2020; Lau et al., 2020; Jefferson et al., 2008).
When interventions need to lead to the desired behavioural change quickly and cost-effectively, what matters is the nuances of implementation – i.e. content, target groups, and method and intensity of delivery. One important tool for implementing interventions at scale is to use multiplier effects from social learning and peer effects (Dupas, 2011). An example from HIV testing showed that people are more likely to inform themselves about their test results if their neighbours do so (Godlonton and Thornton, 2012). So-called “spillover effects” were also identified by Banerjee et al. (2020), in whose study non-participants adopted preventive healthcare measures when surrounded by treatment group participants.
The analysis of potential unintended or even negative effects is an integral part of impact evaluation. With attention exclusively focused on preventing and treating Covid-19, take-up and quality of other health services decreased or were disrupted. Immunisation rates fell significantly, including for diseases such as polio (Nelson, 2020) and increasing child and maternal deaths (Roberton et al., 2020). With increased social isolation, vulnerable population groups – e.g. migrants, the elderly, the mentally ill – experienced higher levels of mental illness (Brooks et al., 2020; Gunnell et al., 2020; Rajkumar, 2020), domestic violence rose (Bullinger, Carr and Packham, 2020), and stigmatization and human rights violations increased (Riley et al., 2020).
Outlook and Recommendations
Identifying successful approaches to reduce the spread of Covid-19 and, potentially, future pandemics will shape the trajectory of countries worldwide in the short and long run. These approaches may vary, from being targeted and specific to being more encompassing and integrative (i.e. considering the role of One Health initiatives). Impact evaluations can help uncover the impacts of pandemic response strategies and guide future investment. Yet rigorous evidence remains scarce.
First, current scientific evidence on containing Covid-19 health consequences reports short-term effects, which may change or even vanish in the long term. Second, while lessons can be learned from earlier pandemics, Covid-19 evidence so far is mostly based on studies from high-income countries. The same interventions may yield differing outcomes when implemented in different cultural, socio-economic, and demographic contexts. Finally, rigorous studies require an ethically sound design, a credible control group, time for effects to unfold, and reliable and relevant data. The need for rapid evidence and limited testing capacity can undermine these goals.
More replication studies in diverse settings and further rigorous evaluations of more targeted and innovative approaches are thus important. Reliable impact evaluations will require high-quality data, from either administrative sources or large-scale primary data collection. Antibody tests will further improve data quality and certainty about effective approaches. High-quality data will also allow the interventions’ cost-efficiency and -effectiveness to be studied, i.e. considering compliance rates and the sustainability of the effects. Carefully designed evaluations can also help to anticipate widening and dangerous healthcare gaps for other diseases.
Once the new studies have produced the hoped-for causal evidence, meta-analyses and systematic reviews can synthesize findings and provide valuable evidence to guide policy.
This post is a condensed version of a policy brief by the BMZ and DEval published and last updated in September 2020. Since then, several new studies have emerged, supporting the here presented evidence (e.g. Takaya et al. 2020; Fetzer 2020; Fetzer and Graeber 2020).
Alon, Titan, Minki Kim, David Lagakos, and Mitchell VanVuren (2020), “How Should Policy Responses to the COVID-19 Pandemic Differ in the Developing World?” w27273, National Bureau of Economic Research, Cambridge, MA. doi.org/10.3386/w27273.
Ash, Elliott, Sergio Galletta, Dominik Hangartner, Yotam Margalit, and Matteo Pinna (2020), “The Effect of Fox News on Health Behavior During COVID-19”, SSRN Electronic Journal. doi.org/10.2139/ssrn.3636762.
Askitas, Nikos, Konstantinos Tatsiramos, and Bertrand Verheyden (2020), “Lockdown Strategies, Mobility Patterns and COVID-19”, IZA Discussion Papers, no. No. 13293. www.iza.org/publications/dp/13293/lockdown-strategies-mobility-patterns-and-covid-19.
Avdeenko, Alexandra, Jakob Gaertner, Marc Gillaizeau, Ghida Karbala, Laura Montenbruck, Giulia Montresor, and Atika Pasha (2020), “Rapid Data for Rapid Action Initiative”. www.poverty-action.org/recovr-study/rapid-data-rapid-action-initiative.
Banerjee, Abhijit, Marcella Alsan, Emily Breza, Arun G. Chandrasekhar, Abhijit Chowdhury, Esther Duflo, Paul Goldsmith-Pinkham, and Benjamin A. Olken (2020), “Messages on COVID-19 Prevention in India Increased Symptoms Reporting and Adherence to Preventive Behaviors Among 25 Million Recipients with Similar Effects on Non-Recipient Members of Their Communities”. NBER Working Paper Series, no. w27496, National Bureau of Economic Research, Cambridge, MA.
Brooks, Samantha K., Rebecca K. Webster, Louise E. Smith, Lisa Woodland, Simon Wessely, Neil Greenberg, and Gideon James Rubin (2020), “The Psychological Impact of Quarantine and How to Reduce It: Rapid Review of the Evidence”, The Lancet Vol. 395/10227: 912–20. doi.org/10.1016/S0140-6736(20)30460-8.
Bullinger, Lindsey Rose, Jillian Carr, and Analisa Packham (2020), “COVID-19 and Crime: Effects of Stay-at-Home Orders on Domestic Violence”, w27667, National Bureau of Economic Research, Cambridge, MA. doi.org/10.3386/w27667.
Chu, Derek K., Elie A. Akl, Stephanie Duda, Karla Solo, Sally Yaacoub, Holger J. Schünemann et al. (2020), “Physical Distancing, Face Masks, and Eye Protection to Prevent Person-to-Person Transmission of SARS-CoV-2 and COVID-19: A Systematic Review and Meta-Analysis”, The Lancet Vol. 395/10242: 1973–87. doi.org/10.1016/S0140-6736(20)31142-9.
Curtis, Val, Wolf Schmidt, Stephen Luby, Rocio Florez, Ousmane Touré, and Adam Biran (2011), “Hygiene: New Hopes, New Horizons”, The Lancet Infectious Diseases Vol. 11/4: 312–21. doi.org/10.1016/S1473-3099(10)70224-3.
Dave, Dhaval M., Andrew I. Friedson, Kyutaro Matsuzawa, and Joseph J. Sabia (2020), "When Do Shelter-in-Place Orders Fight COVID-19 Best? Policy Heterogeneity Across States and Adoption Time", NBER Working Paper Series, no. w27091, National Bureau of Economic Research, Cambridge, MA.
Dupas, Pascaline (2011), “Health Behavior in Developing Countries”, Annual Review of Economics 3: 39.
Etkind, Simon N., Anna E. Bone, Natasha Lovell, Rachel L. Cripps, Richard Harding, Irene J. Higginson, and Katherine E. Sleeman (2020), “The Role and Response of Palliative Care and Hospice Services in Epidemics and Pandemics: A Rapid Review to Inform Practice During the COVID-19 Pandemic”, Journal of Pain and Symptom Management Vol. 60/1: e31–40. doi.org/10.1016/j.jpainsymman.2020.03.029.
Falco, Paolo, and Sarah Zaccagni (2020), “Promoting Social Distancing in a Pandemic: Beyond the Good Intentions”, a2nys. OSF Preprints. Center for Open Science. ideas.repec.org/p/osf/osfxxx/a2nys.html.
Fetzer, T and Graeber, T. 2020. 'Does Contact Tracing Work? Quasi-Experimental Evidence from an Excel Error in England'. London, Centre for Economic Policy Research. cepr.org/active/publications/discussion_papers/dp.php
Fetzer, T, 2020. "Subsidizing the spread of COVID19: Evidence from the UK's Eat-Out-to-Help-Out scheme," CEPR Discussion Papers 15416, C.E.P.R. Discussion Papers.
Godlonton, Susan, and Rebecca Thornton (2012), “Peer Effects in Learning HIV Results”, Journal of Development Economics Vol. 97/1: 118–29. doi.org/10.1016/j.jdeveco.2010.12.003.
Jefferson, Tom, Ruth Foxlee, Chris Del Mar, Liz Dooley, Eliana Ferroni, Bill Hewak, Adi Prabhala, Sree Nair, and Alex Rivetti (2008), “Physical Interventions to Interrupt or Reduce the Spread of Respiratory Viruses: Systematic Review”, The BMJ Vol. 336/7635: 77–80. doi.org/10.1136/bmj.39393.510347.BE.
Lau, Ling San, Goleen Samari, Rachel T. Moresky, Sara E. Casey, S. Patrick Kachur, Leslie F. Roberts, and Monette Zard (2020), “COVID-19 in Humanitarian Settings and Lessons Learned from Past Epidemics”, Nature Medicine Vol. 26/5: 647–48. doi.org/10.1038/s41591-020-0851-2.
Mitze, Timo, Klaus Wälde, Reinhold Kosfeld, and Johannes Rode (2020), “Face Masks Considerably Reduce COVID-19 Cases in Germany: A Synthetic Control Method Approach”, Institute of Labor Economics (IZA), Discussion Paper Series no. IZA DP No. 13319 (June).
Nelson, Roxanne (2020), “COVID-19 Disrupts Vaccine Delivery”, The Lancet Infectious Diseases Vol. 20/5: 546. doi.org/10.1016/S1473-3099(20)30304-2.
Orr, Jayne A., and Robert J. King (2015), “Mobile Phone SMS Messages Can Enhance Healthy Behaviour: A Meta-Analysis of Randomised Controlled Trials”, Health Psychology Review Vol. 9/4: 397–416. doi.org/10.1080/17437199.2015.1022847.
Oshitani, Hitoshi, and The Experts Members of The National COVID-19 Cluster Taskforce at Ministry of Health, Labour and Welfare, Japan (2020), “Cluster-Based Approach to Coronavirus Disease 2019 (COVID-19) Response in Japan—February–April 2020”, Japanese Journal of Infectious Diseases 30 June. doi.org/10.7883/yoken.JJID.2020.363.
Powell-Jackson, Timothy, Jessica J.C. King, Christina Makungu, Nicole Spieker, Susannah Woodd, Peter Risha, and Catherine Goodman (2020), “Infection Prevention and Control Compliance in Tanzanian Outpatient Facilities: A Cross-Sectional Study with Implications for the Control of COVID-19”, The Lancet Global Health 8 (6): e780–89. doi.org/10.1016/S2214-109X(20)30222-9.
Rajkumar, Ravi Philip (2020), “COVID-19 and Mental Health: A Review of the Existing Literature”, Asian Journal of Psychiatry 52 (August): 102066. doi.org/10.1016/j.ajp.2020.102066.
Riley, Taylor, Elizabeth Sully, Zara Ahmed, and Ann Biddlecom (2020), “Estimates of the Potential Impact of the COVID-19 Pandemic on Sexual and Reproductive Health in Low-and Middle-Income Countries”, Int Perspect Sex Reprod Health 46: 73–6.
Takaya, S., Tsuzuki, S., Hayakawa, K., Kawashima, A., Okuhama, A., Kanda, K., Suzuki, T., Akiyama, Y., Miyazato, Y., Ide, S. and Nakamura, K., 2020. Nightlife clusters of coronavirus disease in Tokyo between March and April 2020. Epidemiology & Infection, 148.
- The views and opinions expressed in the blog articles are those of the authors.
These do not necessarily reflect findings and recommendations from DEval evaluations. Such results and recommendations can be found in our evaluation reports, policy briefs and press releases.