The impacts of Covid-19 interventions on health-related outcomes

Even though a vaccine against Covid-19 has been developed, it will still be some time until a majority of the population can be immunised and until there are clear treatment protocols. Therefore, slowing the spread of the virus is still essential to reduce deaths and to avoid overloading healthcare systems. Low- and middle-income countries (LMICs) are particularly at risk due to shortages of skilled health personnel, technical healthcare equipment and infrastructure, and lack of funds to implement preventive measures.

Robust evidence on the effects of health and pandemic control interventions can guide the cost-efficient and ‑effective use of funds used for measures against COVID-19 and its secondary effects. However, the novelty and rapid transmission of the disease have so far left little time for rigorous impact evaluations. The first relevant scientific evidence has mainly relied on non-experimental designs such as simulation exercises, event studies including before-and-after studies and quasi-experimental studies, which compare outcomes of pandemic control interventions with non-intervention or a so-called “control” group.

Early policies and recommendations sought to halt the spread of the disease through the reduction of social interactions. The cancellation of public events and gatherings seem to be the most effective measure according to a study by Askitas, Tatsiramos and Verheyden (2020). In the US, states that implemented shelter-in-place orders had 44 per cent fewer Covid-19-related fatalities (Dave et al. 2020).

For LMICs, however, more nuanced approaches are needed to balance the costs and benefits of minimising social interaction. In many LMICs weak social security systems, fiscal incapacities for substantial government transfers, little household savings, and limited teleworking capacity, force people to run the risk of infection and participate in face-to-face labour market interactions. To mitigate the economic effects of blanket lockdowns, Alon et al. (2020) argue in favour of stay-at-home orders and social transfers solely for the elderly.

However, “Stay home, stay safe” recommendations are challenged by limited access to clean water and hygiene facilities in low-income households, combined with overcrowded, multigenerational living arrangements. Here, cluster-based responses are one possible solution. Oshitani et al. (2020) argue that most transmissions were spurred by a small proportion of cases, which led to cluster formations (e.g. in enclosed or crowded spaces). Tracing contacts of infected people can help to avoid clusters and therefore has the potential to mitigate transmission. However, effective contact-tracing needs reliable data. In an ongoing large-scale RCT, Avdeenko et al. (2020) track the spread of the disease over time and space in rural Pakistan using frequent, short phone interviews, and test whether geographical clusters can be identified.

To address the trade-off between health risks and economic burden, non-pharmacological interventions such as hand hygiene, wearing face masks, and social distancing could help people resume their economic activities safely. The effectiveness of face protection and social distancing has been shown in a meta-analysis of 44 observational studies, mainly conducted with health-sector personnel. Including respiratory infectious diseases such as SARS and MERS as well as Covid-19, Chu et al. (2020) provide evidence that eye protection, face masks, and social distancing of over one metre do not fully eliminate but effectively reduce the risk of infection. Mitze et al. (2020) use a synthetic control method approach to show that the mandatory wearing of face masks in Jena, Germany, slowed the daily growth of new infections by 40%.

Social norms, motivation and habit play important roles in adherence to safe hygiene behaviour (Powell-Jackson et al., 2020; Curtis et al., 2011). Awareness raising can be a powerful tool to promote preventive healthcare behaviour (Dupas 2011). A pre-pandemic meta-analysis showed that text messages (SMSs) can be useful to induce behavioural change (Orr and King, 2015). In the context of Covid-19, Falco and Zaccagni (2020) sent SMSs in Denmark to remind people to stay at home. The messages increased recipients’ intention to not leave the house, but did not translate into action. Banerjee et al. (2020) conducted an RCT in West Bengal, India, on the effectiveness of Covid-19 information campaigns. They show that an additional SMS from a prominent public figure emphasizing the importance of complying with the rules notably increases the desired behaviour. Study participants’ mobility reduced, handwashing and mask-wearing increased, and more symptoms were reported to local health facilities. The harm public messaging can do is examined by Ash et al. (2020), who show a causal link between Covid-19-sceptic programmes on Fox News and non-compliance with social distancing rules. Now that a vaccine has been developed, further awareness campaigns will need to quickly address potential fears and misconceptions that could prevent take-up.

Finally, evidence from earlier studies on other viral diseases such as Ebola and HIV can inform about potentially successful responses to Covid-19. What seemed to matter then was timely strengthening of health systems, e.g. maintaining essential health services, access to infection-prevention and ‑control measures, testing and treatment, especially for displaced populations, fast and flexible production of protocols, timely shift of resources, establishment of new communication technologies and innovative community engagement, and standardized data collection (Jefferson et al., 2011; Etkind et al., 2020; Lau et al., 2020; Jefferson et al., 2008).

When interventions need to lead to the desired behavioural change quickly and cost-effectively, what matters is the nuances of implementation – i.e. content, target groups, and method and intensity of delivery. One important tool for implementing interventions at scale is to use multiplier effects from social learning and peer effects (Dupas, 2011). An example from HIV testing showed that people are more likely to inform themselves about their test results if their neighbours do so (Godlonton and Thornton, 2012). So-called “spillover effects” were also identified by Banerjee et al. (2020), in whose study non-participants adopted preventive healthcare measures when surrounded by treatment group participants.

The analysis of potential unintended or even negative effects is an integral part of impact evaluation. With attention exclusively focused on preventing and treating Covid-19, take-up and quality of other health services decreased or were disrupted. Immunisation rates fell significantly, including for diseases such as polio (Nelson, 2020) and increasing child and maternal deaths (Roberton et al., 2020). With increased social isolation, vulnerable population groups – e.g. migrants, the elderly, the mentally ill – experienced higher levels of mental illness (Brooks et al., 2020; Gunnell et al., 2020; Rajkumar, 2020), domestic violence rose (Bullinger, Carr and Packham, 2020), and stigmatization and human rights violations increased (Riley et al., 2020).

Identifying successful approaches to reduce the spread of Covid-19 and, potentially, future pandemics will shape the trajectory of countries worldwide in the short and long run. These approaches may vary, from being targeted and specific to being more encompassing and integrative (i.e. considering the role of One Health initiatives). Impact evaluations can help uncover the impacts of pandemic response strategies and guide future investment. Yet rigorous evidence remains scarce.

First, current scientific evidence on containing Covid-19 health consequences reports short-term effects, which may change or even vanish in the long term. Second, while lessons can be learned from earlier pandemics, Covid-19 evidence so far is mostly based on studies from high-income countries. The same interventions may yield differing outcomes when implemented in different cultural, socio-economic, and demographic contexts. Finally, rigorous studies require an ethically sound design, a credible control group, time for effects to unfold, and reliable and relevant data. The need for rapid evidence and limited testing capacity can undermine these goals.

More replication studies in diverse settings and further rigorous evaluations of more targeted and innovative approaches are thus important. Reliable impact evaluations will require high-quality data, from either administrative sources or large-scale primary data collection. Antibody tests will further improve data quality and certainty about effective approaches. High-quality data will also allow the interventions’ cost-efficiency and -effectiveness to be studied, i.e. considering compliance rates and the sustainability of the effects. Carefully designed evaluations can also help to anticipate widening and dangerous healthcare gaps for other diseases.

Once the new studies have produced the hoped-for causal evidence, meta-analyses and systematic reviews can synthesize findings and provide valuable evidence to guide policy.

This post is a condensed version of a policy brief by the BMZ and DEval published and last updated in September 2020. Since then, several new studies have emerged, supporting the here presented evidence (e.g. Takaya et al. 2020; Fetzer 2020; Fetzer and Graeber 2020).