Monthly Archives: June 2012

Evaluating the impact of stimulation and micronutrients on early childhood development

The preliminary results and analysis of this study was presented at the “Promises for Preschoolers: Early Childhood Development and Human Capital Accumulation” conference on 25 June in London.

The first few years of life are a period of intensive brain development. It is in this crucial phase that we form cognitive and social skills that last through an entire life span.  Preliminary results from an impact evaluation in Colombia suggest that regular visits from a well-trained but local “home visitor” had significant impacts on children’s cognitive development.  Moreover, this approach produced a significant impact on the receptive language and some behavioural indicators of children who were relatively older at the start of the intervention.  The same results also indicate some important effects on parenting practices.

It is an accepted truth that well-designed interventions targeted during the early years can have long lasting impacts. However, moving from such a realisation to the delivery of sustainable and cost-effective policies is a complex process. The behaviour of parents, the links of the family unit with the community, and child rearing practices that improve interactions with the parents and lead to more and better investments in child development must all be taken into account.

We designed, implemented and tested a programme in Colombia with the explicit aim of offering a blueprint for such a scalable intervention.  We wanted to address these challenges as well as generate data that allowed us to learn in a deeper and more scientific fashion.

An innovative approach to early childhood development

This Colombian programme was targeted at poor households with children between 1 and 2 years of age. The intervention was conducted in 96 small towns and covered around 1,500 households in total. The households we studied were beneficiaries of a large Colombian cash transfer programme called Familias en Acción.

Integral to this intervention was a stimulation programme, which consisted of weekly home visits from a “home visitor” to a mother and child that lasted an hour and that were carried over a period of 18 months. A key innovation of the programme was that the home visitors were drawn from a network of local women who were mediators between beneficiaries of the cash transfer programme and the government. They were thus both influential and well connected within their own communities. The home visitors received training and advice from mentors with backgrounds in psychology or social work. Having home visitors based in the communities helped minimise costs and facilitated implementation. This innovation also carried with it the promise that local communities would feel empowered to take ownership of the intervention going forward.

The home visitor interacted with mother and child on the basis of a well-structured but flexible curriculum. The curriculum focuses on cognitive and linguistic development and includes age-appropriate activities eg songs and nursery rhymes, toys (constructed from everyday domestic materials), books and puzzles. More generally, there is an emphasis on fostering interaction between mother and child on a continuous basis, and on integrating two-way conversation, praise and play into regular day-to-day activities.

The second arm of the intervention also included a micronutrient supplementation component: provision of iron, zinc, Vitamin A and Vitamin C in levels appropriate for long-term intake by children.

A key goal of this study was to evaluate the effect of the programme on children’s cognitive, linguistic and motor development. We also studied the way household behaviour changed in reaction to the stimulation and micronutrient supplementation.  While we found that the home visitor element produced significant improvements in both cognitive development and the quality of the home environment, we detected limited impacts of micronutrient supplementation.

Orazio Attanasio, University College London, Centre for the Evaluation of Development Policy (EDePo) at Institute for Fiscal Studies (IFS)

Emla Fitzsimons, Centre for the Evaluation of Development Policy (EDePo) at Institute for Fiscal Studies (IFS), London, UK

Sally Grantham-McGregor, University College London

Costas Meghir, Centre for the Evaluation of Development Policy (EDePo) at Institute for Fiscal Studies (IFS), London, UK and Yale University

Marta Rubio-Codina, Centre for the Evaluation of Development Policy (EDePo) at Institute for Fiscal Studies (IFS), London, UK

Evidence to policy: bridging gaps and reducing divides

Evidence-based policymaking is important but not always straightforward in practice. The complex reality of policymaking processes means that the availability of high quality research is a necessary, but not sufficient, ingredient for evidence informed policy.

At the recent 12th Annual Colloquium of the Campbell Collaboration, all the vibrant and in-depth discussions conveyed one clear message: we need to get better at bridging the gap between research and policy. Over 150 researchers and policymakers gathered in Copenhagen for the three-day event, which combined training in systematic review methods with plenary sessions focusing on policy and practice. The takeaway messages from the sessions offer a clear and sharp call to action for researchers.

Produce relevant and timely research

As researchers, we often focus too much on the supply side – on producing high quality and academically stimulating research. But producing relevant research means a greater focus on the demand side – working closely with commissioners and users of research to ensure that our output meets their needs.

For instance, Ruth Stewart and colleagues at the newly established Johannesburg Centre of the Collaboration for Environmental Evidence are working towards addressing the demand side and setting research priorities. Her team is running a consultation exercise involving a range of governmental and non-governmental stakeholders across sub-Saharan Africa. This exercise will be followed by country-specific workshops focusing on context specific priorities.  3ie’s Policy Window similarly uses upfront user engagement in identifying topics for primary studies.

Know the rules of the political game

Researchers need to be aware of the political narrative and ensure that evidence reaches policymakers when they are most receptive. According to Baroness Estelle Morris, former Secretary of State for Education and skills in the UK Parliament, evidence is most useful and influential when the political narrative is still being formed or reviewed rather than once a political stance is taken or campaign rolled out.

In fields in which policy is highly politicised, we need to be mindful of the challenge of evidence prompting a political U-turn. We also need to be aware of how research findings are portrayed in the media. Careless criticism of the political agenda may damage the relationship between a researcher and a politician. When bridges are burned, even high quality evidence can end up abandoned on the sidewalk.

Translate and institutionalise knowledge

Another key message of the colloquium was that we need to spend more time and resources on evidence translation.  We spend a lot of time perfecting the methodologies of our research. But we should also place greater emphasis on finding ways to get evidence to feed into policy, meaning getting better at communicating research using formats accessible to users. User friendly summaries and policy briefs are perhaps used most frequently, but there is a need to go a step further.  We need to stop thinking about knowledge translation as being just about packaging of information and move towards institutionalising the use of evidence in organisations and systems.

There are some valuable lessons to be learnt from sectors that have a longer tradition of research for evidence based policy than the international development sector. The Evidence-Based Policing Matrix is an example of an innovative knowledge translation tool used in crime and justice. Developed by Dr. Cynthia Lum and colleagues, the matrix provides a visual representation of the evidence of the effectiveness of a range of policing interventions.  The clusters of studies are visually represented as ‘realms of effectiveness’ that allow us to make policy related generalisations.

The team has further developed an approach to implement this evidence tool in practice through the Matrix demonstration project which works with police agencies. This matrix translates and institutionalizes research findings for the day to day practice of police agencies.

At 3ie, we are also working on developing similar knowledge translation tools with our evidence Gap Maps. 3ie’s Gap Maps graphically consolidate what we know about ‘what works’ in particular sectors by drawing out evidence from systematic reviews and impact evaluations.

Engage with a broader range of evidence

In line with the need to conduct more policy-relevant research, there appeared to be an increasing acceptance of the need to engage with broader types of evidence, including a range of quasi-experimental study designs for evaluating intervention effectiveness, as well as qualitative evidence.

Professor Mike Saini from the University of Toronto, highlighted how qualitative synthesis could potentially answer a different (and often complementary) set of questions than quantitative syntheses of effectiveness. Such an approach starts with the research question and adopts the method(s) which is best suited for answering it.

Qualitative evidence can, for instance, help us better understand issues related to process and implementation of complex interventions. It can also improve our understanding of barriers and facilitators of intervention effectiveness.

At 3ie we are working hard to ensure we fund and conduct research that is both high quality, relevant, accessible and timely. To achieve this ambitious goal, we are increasingly working with our constituency of policymakers and practitioners to identify research priorities and to develop effective dissemination and policy-influence plans. The work of our colleagues from the Campbell Collaboration and other organisations will surely be an inspiration for 3ie’s work.