2013 Training Course - Impact Evaluation and Analysis of Development Interventions

Overview

Over the course of the past ten years, the number of impact evaluations related to development interventions has increased substantially. In fact, under the leadership of leading academic institutions in the US and Europe, empirical studies based on robust comparisons of treatment and control groups have received considerable financial support from the donor community and led to dozens of publications in leading academic journals in the areas of agriculture, education and health..

At the heart of this movement has been a widely-shared idea that impact evaluations can contribute to development effectiveness by helping to determine “what works and what does not”, and that this research can be used to influence policy. Yet, despite an estimated 250 on-going or completed impact evaluation studies in Africa conducted by NGOs, the World Bank and universities, only a handful of these involve researchers from African universities. Why?  And how can this be changed?

This course intends to provide African-based researchers with knowledge of the latest developments regarding both technical and topical aspects of impact evaluation. The course is also meant to help build a high-level network of local impact evaluation practitioners and partners of within the region.

course information

The course will be organized into two* one-week courses in Dakar, Senegal. Each course will be further divided into three two-day modules focused on a particular strand of literature that is of current relevance for impact evaluations in the region, and taught by a leading academic in the field. Sector-specific topics include agriculture, education, finance, health, infrastructure and safety nets. In each module, the training will cover theoretical and empirical areas in that literature, apply that learning by replicating a real-life impact evaluation, and a discussion of a participant’s research and how these techniques could be applied.

*A follow up course will be offered in Fall 2013.  We ask that you attend both the Spring and Fall courses.

The Spring course will be organized as follows:

• Fifteen years of impact evaluations in Africa: themes, teams and geography
• Impact Evaluations and the conduct of research in African research centers
• Objectives of the AIEN
• The tools of the AIEN
• A network of high-level Africa-based researchers involved in the design and conduct of impact evaluations
• A network of world-class professionals involved in Impact Evaluations, supporting AIEN with training and mentoring
• A set of ad-hoc tools to support proposal designs, data analysis, and paper writing
• Looking forward: 2013 and 2014 plans for the AIEN.

• Overview of Impact Evaluation --What is it?  How can it be used? What are the different methodologies? (RCT, RDD, DD, PSM, IV) (This will be an interactive session)
• The Econometric Framework for Impact Evaluation: Different Treatment Effects (ATT, ITT, LATE). This includes a practical case study from Niger, and mentions ITT and LATE, which Ruth will cover later
• Randomization: What is it?  What are the different approaches?  How can it be used? 
• Case study on using RCTs in their work
• What's power got to do with it?  Estimating sample size in RCTs and other evaluations

• Social Protection Programs: The goal is to set up the idea of social protection, with a broad overview of programs and more specific details on CCT programs.  I'll talk specifically about what we know about what works and then what is happening in Africa in CCT programs or other social protection (pension in South Africa).
• Targeting in Social Protection: How do we get benefits to those who need them, and how does that affect methods to evaluate social protection programs?  How to target programs well (do we know)?
• Regression Discontinuity Design- From targeting, what it measures (LATE), why it is akin to a localized experiment, and examples of how regression discontinuity has been used to evaluate CCTs?
• Example based on the CCT evaluation of Comunidades Solidarias Rurales in El Salvador

• Managing threats to analysis (2 hours)
• Attrition: attrition and differential attrition
• Spillovers
• Partial Compliance: Estimating Intention to Treat, Local Average Treatment Effects
• Partial Compliance: Encouragement design
• Examples from randomized control trials in index insurance (1 hour)
• Case-study exercise 1 (1.5 hours)
• Increasing power through repeated measures (45 minutes)
• Examples from randomized control trials in agricultural markets  (45 minutes)
• Case-study exercise 2 (1.5 hours)

Participants of the training will be given access to a dedicated space within AGRODEP website, with access to relevant material (papers, datasets, problem sets, blogs, conference announcements and others), as well as a discussion forum for group members to share experiences and information and to receive feedback on on-going work. Further training sessions and group meetings shall also be available for the upcoming years.

ONLINE APPLICATION

This training course requires Stata background knowledge. All applicants need to pass the online Stata Proficiency test and submit the application form below by April 12.

• Online Stata Proficiency test
• Application form

If you would like to practice using Stata before taking the proficiency test, please review the modules below.  Information included covers Stata use for beginners, linear regressions, bivariate regressions, and panel data.  You will need to know this information to successfully complete the test.

• Training Module 1: Introduction to Stata
• Training Module 2: Basic Data Management, Graphs, and Log-Files
• Training Module 3: Linear Regressions
• Training Module 4: Bivariate Regressions
• Training Module 5: Panel Data Regressions

Organizer

Tanguy Bernard is a Research Fellow within the Markets, Trade and Institution Division of IFPRI, based in Senegal. Over the recent years, his research has mostly focused on producers’ organizations, their existence, their membership, their activities, and their performance in linking African smallholders to input and output markets. His works relies heavily on primary data collection and experimental tools, with a geographical focus on Senegal, Burkina Faso and Ethiopia. Prior to his current position, Tanguy Bernard was a research officer at the Agence Française de Développement, prior to which he was a post-doctoral fellow at IFPRI, based in Ethiopia.

Instructors

Jenny C. Aker is an Assistant Professor of Economics at the Fletcher School and Department of Economics at Tufts University. She is also a Non-Resident Fellow at the Center for Global Development , a member of the Advisory Board for Frontline SMS and the Boston Network for International Development (BNID) . She also serves as the Deputy Director of the Hitachi Center for Technology and International Affairs.  After working for Catholic Relief Services as Deputy Regional Director in West and Central Africa between 1998 and 2003, Jenny returned to complete her PhD in agricultural economics at the University of California-Berkeley. Jenny works on economic development in Africa, with a primary focus on the impact of information and information technology on development outcomes, particularly in the areas of agriculture, agricultural marketing and education; the relationship between shocks and agricultural markets; the determinants of agricultural and environmental technology adoption; and impact evaluations in the area of social protection (cash transfers and vouchers). Jenny has conducted field work in many countries in sub-Saharan Africa, including Benin, Burkina Faso, Burundi, Chad, DRC, The Gambia, Ghana, Guinea, Kenya, Liberia, Mali, Mozambique, Niger, Nigeria, Rwanda, Senegal, Sierra Leone and southern Sudan, as well as Haiti and Guatemala.

Alan de Brauw is a Senior Research Fellow in the Markets, Trade, and Institutions Division of IFPRI. He has a Ph.D. in Agricultural and Resource Economics from the University of California at Davis; prior to joining IFPRI, he was an Assistant Professor of Economics at Williams College. His research has focused on understanding the evolution of rural labor markets in a developing economy and the effects of migration on source households; he has also conducted randomized and non-randomized evaluations of conditional cash transfer programs and agricultural interventions. His work has been published in journals such as the Journal of Development Economics, the American Journal of Agricultural Economics, the British Journal of Nutrition, and Economic Development and Cultural Change.

Ruth Vargas Hill is a Senior Research Fellow in the Markets Trade and Institutions division of IFPRI. She joined IFPRI in 2007 as a post-doctoral fellow in the Director General’s Office. Ruth has 9 years experience conducting research on rural markets in East Africa and South Asia, more recently focusing on formal and informal markets for insurance. She has worked on the design and implementation of index-insurance projects in Bangladesh, Ethiopia and India. Her work in Ethiopia and Bangladesh focuses on designing group-based index insurance schemes which combine group saving and lending with the purchase of formal insurance products. She also conducts research on market institutions and has been working with firms in Tanzania and farmers groups in Uganda to identify and implement interventions that improve the functioning of markets. Prior to joining IFPRI Ruth worked at the World Bank. She received a PhD in economics from the University of Oxford in 2005.

Training materials

Presentations

• Non-Randomized Evaluations: Regression Discontinuity Design
• Examples: Impacts of Comunidades Solidarias Rurales in El Salvador and BISP in Pakistan
• Social protection: program design and evaluation
• Learning about Agricultural Microinsurance from randomized control trials
• Impact Evaluation and Analysis of Development Interventions: Increasing power through repeated measures
• Impact Evaluation and Analysis of Development Interventions: Managing and minimizing threats to analysis

Data files 

• Data do files

Related Papers

• RDD
  • Robust Data-Driven Inference in the Regression-Discontinuity Design
  • Using the Regression Discontinuity Design with Implicit Partitions The Impacts of Comunidades Solidarias Rurales on Schooling in El Salvador 
  • Optimal bandwidth choice for the regression discontinuity estimator
  • Regression discontinuity designs: A guide to practice
  • Regression Discontinuity Designs in Economics
  • Manipulation of the Running Variable in the Regression Discontinuity Design: A Density Test
  • Causal inference with observational data
• Social Protection
  • Cash Transfers and Child Schooling
  • Must conditional cash transfer programs be conditioned to be effective? The impact of conditioning transfers on school enrollment in Mexico
  • Grandmothers and Granddaughters: Old‐Age Pensions and Intrahousehold Allocation in South Africa
  • Targeting of Transfers in Developing Countries
  • Targeting Food Security Interventions When “Everyone is Poor”: The Case of Ethiopia’s Productive Safety Net Programme
  • Cash transfers, conditions, school enrollment, and child work: Evidence from a randomized experiment in Ecuador
  • Conditional cash transfer
  • Targeting of Transfers in Developing Countries
• Partial Compliance and Impact Evaluation Design
  • Barriers to Household Risk Management: Evidence from India
  • Who You Train Matters: Identifying Complementary Effects of Financial Education on Migrant Households
  • Why Don't the Poor Save More? Evidence from Health Savings Experiments
  • Beyond baseline and follow-up: The case for more t in experiments
  • Social security health insurance for the informal sector in Nicaragua: a randomized evaluation
  • Demand for Weather Hedges in India: An Empirical Exploration of Theoretical Predictions