Free to choose: Introducing Technology for Foundational Learning in Pakistan (Joint with Juan Baron, Isabel Macdonald, Tomoya Murakawa, Zainab Qureshi)
Prior research has found varying returns to education technology (EdTech). These studies almost universally mandate the use of technology when it is introduced. In this study, we find that the approach to introducing EdTech has a large impact on learning outcomes. We explore these dynamics in the context of a large-scale foundational learning program in public primary schools in Pakistan. The novel curricular program incorporated both targeted instruction and structured pedagogy, and some teachers were also randomly assigned access to a low-cost, smartphone application (tech tool) designed to reduce the administrative burden of classroom management. Test results show that the 8-week program raised student learning overall by 0.12 SD across math, English, and Urdu. Learning gains, especially among low-performing students, were highest when the tech tool was offered to teachers but framed as optional. In contrast, mandating the use of the tech tool (0.10 SD) had significantly lower student learning relative to the Optional tech (0.19 SD) version of the program. The differences are driven by schools with female teachers and students (schools are gender segregated), where the Mandated treatment arm performed even worse (0.02 SD) relative to the Optional arm (0.17 SD). These findings suggest substantial heterogeneity in teacher returns to EdTech, and support a self-selection approach for introducing new technology.
Inducing Positive Selection through Performance Pay: Experimental Evidence from Pakistani Schools (Joint with Christina Brown)
Attracting and retaining high-quality teachers has a large social benefit, but it is challenging for schools to identify good teachers ex-ante. This paper uses teachers’ contract choices and a randomized controlled trial of performance pay with 7,000 teachers in 243 private schools in Pakistan to study whether performance pay affects the composition of teachers. Consistent with adverse selection models, we find that performance pay induces positive sorting: both among teachers with higher latent ability and among those with a more elastic effort response to incentives. Teachers also have better information about these dimensions of type than their principals. Using two additional treatments, we show effects are more pronounced among teachers with better information about their quality and teachers with lower switching costs. Accounting for these sorting effects, the total effect of performance pay on test scores is twice as large as the direct effect on the existing stock of teachers, suggesting that analyses that ignore sorting effects may substantially understate the effects of performance pay.
Subjective versus Objective Incentives and Employee Productivity (Joint with Christina Brown)
A central challenge facing firms is how to incentivize employees. While objective, outputbased incentives may be theoretically ideal, in practice they may lead employees to reduce effort on non-incentivized outcomes and may fail in settings where effort is weakly tied to output. We study the effect of subjective incentives (manager-discretionary performance evaluation) and objective incentives (test score-based) relative to no incentives for teachers using an RCT in 230 Pakistani schools. First, we show that subjective and objective incentives both increase test scores and have similar magnitude effects. However, objective incentives decrease non-test score student outcomes relative to subjective incentives. Second, we show that teachers’ effort response is very different under each scheme, with attendance increasing under subjective and teaching quality decreasing under objective. Finally, we rationalize these effects through the lens of a moral hazard model with multi-tasking. We use within-treatment variation to isolate the causal effect of contract noise and distortion and show that these channels explain most of our reduced form effects.
Grade Repetition and Household Responses in a Low Income Setting (Joint with Ethan Matlin and Gabrielle Vasey)
Millions of students around the world are retained and repeat a grade each year, yet we may not fully understand the consequences of this choice. Especially in low-income countries, where the majority of repeaters reside, and the decision is often at the teacher’s discretion, it is important to understand how parents, students, and schools react to this negative information shock. In this paper, we use rich matched household-child-school panel data from Pakistan to study the dynamics of the beliefs, investments, and outcomes of these three agents following a student not being promoted to the next grade. We find that following grade repetition, parents revise downward their expectations, beliefs, and investments. Students are discouraged by the retention, and decrease their beliefs in the value of study effort. Conversely, schools do not play a large role, and teachers have no negative bias towards students who are repeating. Overall, we find negative effects of repetition: repeaters score -0.27 to -0.44 standard deviations worse in math, English, and Urdu and are 7.1 percentage points more likely to drop out than their peers.