The overload in Covid-19 resources – navigating the evidence jungle

There has always been a gap between what is known from evidence and what is implemented in development practice. There are multiple reasons for this, with information overload and high search and screening costs potentially being at its core. These obstacles are amplified in the current crisis, due to the enormous number of Covid-19 resources produced in a very short period of time.

We have tried to blaze a trail through the jungle of Covid-19 evidence – searching, screening, assessing, and structuring the resources produced by well-known organizations working on development or relevant evidence. This included more than 50 different stakeholders, such as development agencies, research institutions, and multilateral development organizations. We encountered a number of difficulties: the sheer number of websites containing Covid-19 resources, and discriminating between which websites to consult and which to ignore. It is challenging to understand the content and functioning of each resource, and  the evidence base and quality of the resources are often impossible to assess. Frequently, the target audience (policy-makers, researchers, project implementers) is not specified. It seems far-fetched to expect political decision makers and project implementers to succeed in this chaotic information search.

To provide some guidance, we came up with:

A list of stand-out-trees among the Covid-19 evidence jungle, i.e. some resources that we consider particularly helpful for decision makers in policy and practice as well as for researchers.

A list of recommendations for producers of Covid-19 resources that might help them to grow more stand-out-trees – recommendations to make the jungle more accessible and increase its effectiveness.

Let’s start with how to grow a stand-out-tree within the Covid-19 evidence jungle.

- Clearly flag what type of evidence your Covid-19 resource is based on. For example, it could be based on opinion, rigorous impact evaluations, or systematic reviews – or be rooted in analysis of macroeconomic data.

- Link the original sources/evidence if you use secondary sources. Here’s a previous entry in this DEval blog series as an example of how to approach this, and another one from the Campbell Collaboration.

- Indicate your target audience. Knowing for whom this is relevant can speed up information search for everyone.

- Present information in an understandable, non-complex, and maybe innovative way. With every additional tree in the jungle, the quality of this so-called knowledge brokering becomes more important.

- Each resource in the Covid-19 jungle has been set up with the best of intentions. However, members of the evidence community need to coordinate the provision of key resources on a single platform, and to jointly advertise this platform to ensure that synergies are harnessed and duplications avoided. Though some coordination attempts have been made, notably the work by the OECD Covid-19 evaluation coalition and a number of other collaborations out of which some are presented below, we see a need for stronger joint action towards a one-stop-shop of Covid-19 resources.

Now let us turn to the existing stand-out-trees in the Covid-19 evidence jungle. This list is non-extensive – other valuable resources are out there.

- This J-PAL blog series is a great example of concise blog posts rooted in rigorous evidence, linking all its sources. It summarizes rigorous evidence on different socio-economic topics and currently has a focus on Covid-19-relevant evidence. For example, this blog post summarizes existing knowledge on how to increase uptake of healthy behaviours and to improve the delivery of health products and services. Other topics include cash transfers, mobile money, gender-based violence and schooling.

-Similarly, this blog series by 3ie is another excellent example of how to present existing evidence in an easily understandable way. It features Covid-19-related blog posts on a wide range of topics, such as handwashing, ways to improve maternal and child health, and vaccinations, as well as challenges and opportunities in data collection during pandemics. For example, check out this evidence-summarizing blog post on how social media can be used for community health outreach during the pandemic.

- This evidence collection by evidence aid features Covid-19-relevant systematic reviews, including topics such as prevention of transmission, treatment of infected people, and socioeconomic consequences. The collection also provides plain-language summaries for all reviews.

- The Africa UN Knowledge Hub for COVID-19 provides a collection of information and resources, including country data, country responses, and knowledge products on potential impacts of mitigation efforts relevant to Africa.

- The COVID-END has devoted one part of their platform to decision-makers (the other one is aimed at researchers). Apart from a collection of selected resources and practical tips and tools for those supporting decision-makers with respect to evidence use, it includes a taxonomy of decisions, particularly with respect to different mitigation measures that could be taken.

- The IPA RECOVR Hub pools various research-relevant information, such as ongoing research projects and current funding opportunities. It also demonstrates one of the few coordination efforts, as multiple partners are contributing to this hub.

- The Development Impact blog of the World Bank had a two-part piece on how to conduct a phone survey in times of Covid-19 (see Part I and Part II). It reflects a high level of expertise and gives detailed and very practice-oriented guidance on the use of phone surveys.

- Similarly, ODI has collected an extensive list of resources on different challenges and best practices for collecting primary data in a Covid-19 era.

- Oxford University’s policy tracker measures and ranks the strength of government responses to Covid-19 along 17 dimensions for more than 160 countries. This may prove to be a useful resource for measuring impacts of Corona responses.

- The Global Partnership for Sustainable Development Data (itself a commendable example of concerted action) has created a list of useful Covid-19 resources with a focus on datasets, visualizations, maps, and tools. They use nice, to-the-point descriptions for each resource.

- The C3.ai data lake has integrated multiple data sets, sourced from all over the globe, in a unified and frequently updated data model, making data analysis much easier.

While the content of the resources above is tailored to the Covid-19 pandemic, evidence that was created before the pandemic might be relevant to decisions in the current crisis, too. These two further sources systematically and thematically synthesize findings from single-impact evaluations as well as systematic reviews:

- The 3ie evidence hub hosts 3,745 impact evaluations, 730 systematic reviews, and 20 evidence gap maps that you can browse by a range of filters such as sector, country, or funding organization.

- Similarly, the Campbell Collaboration repository holds systematic reviews and evidence gap maps that you can also filter by sector and country, among others.

We hope these stand-out-trees will help you navigate the jungle!