BUILDING RESILIENT EDUCATION SYSTEMS: EVIDENCE FROM LARGE SCALE RANDOMISED TRIALS IN FIVE COUNTRIES

BUILDING RESILIENT EDUCATION SYSTEMS: EVIDENCE FROM LARGE SCALE RANDOMISED TRIALS IN FIVE COUNTRIES

More than two billion people live in countries affected by emergencies that frequently disrupt education such as: conflict, disease, natural disasters, and climate events. Education systems need to withstand these shocks, all of which routinely close schools. During these emergencies, alternative models are needed to deliver education. However, rigorous evaluation of effective educational approaches in these settings is challenging and rare, especially across multiple countries.

Following on from Youth Impact's work on distance education during the Covid-19 pandemic in Botswana, researchers asked would this approach scale to new settings, with government, and other education in emergencies? This project used large-scale randomised trials to evaluate the provision of education in emergency settings across five countries: India, Kenya, Nepal, Philippines, and Uganda.

This project tests multiple scalable models of remote instruction for primary school children during COVID19, which disrupted education for over one billion schoolchildren worldwide. Focusing on a method called ‘connectEd’, a phone call tutorial programme used to deliver high quality education through mobile phone calls, this project replicated and scaled the Youth Impact work from Botswana in five randomised controlled trials in India, Nepal, Kenya, the Philippines, and Uganda.

World map pointing to countries where ConnectEd are working

In total, over 16,000 households were enrolled in this multi-country evaluation. In each household, a primary school student and a caregiver were identified to receive the intervention. Students were enrolled in grades 3 to 5, with one exception in Kenya where students were in grades 1 and 2. The trials took place between December 2020 and July 2022.

The evaluation tested a simple education in emergencies programme of targeted 1-on-1 instruction delivered over mobile phones– a high-access, low cost and scalable way to reach students and their
caregivers when school is out of session – to provide various educational interventions over the
course of eight weeks. This approach was based on 2 underlying principles: 1) platform: reaching students on a widely available platform (mobile phones) and 2) pedagogy: using the demonstrated effectiveness of targeted instruction.

Treatment groups

One treatment group received a set of SMS messages with numeracy content provided weekly.
A second treatment added weekly one-on-one 20-minute phone call tutorials. Phone calls covered foundational numeracy and aimed to target instruction to student learning levels via light-touch, high-frequency assessments conducted on a weekly basis. Using the assessment, instructors led students through foundational numeracy practice problems to help them master addition, subtraction, multiplication, and division.
Each country also had a control group

In addition, researchers tested several scalable delivery models, including non-government organisations and government teacher delivery, with the implementer type randomised within 2 countries.

Despite the different contexts within this study, the results show that the effectiveness of phone call tutorials can scale across countries. Researchers find consistently large and robust effect sizes on learning, with the weekly phone and SMS program causing a 0.32 Standard deviation increase in learning. These effects are highly cost-effective, delivering up to four years of high quality instruction per $100 spent, ranking in the top percentile of education programs and policies. The SMS arm had a positive effect on learning but unlike the phone call amr, SMSs alone were not effective across all contexts.

In a subset of trials, researchers randomised whether the intervention was provided by non-government organisation instructors or government teachers. Results here show similar effects, indicating scalability within government systems. These results reveal it is possible to strengthen the resilience of education
systems, enabling education provision amidst disruptions, and to deliver cost-effective learning
gains across contexts and with governments.