Cappa – MICS

DR. CAPPA: Thank you, Joan. Thanks a lot to the organizers for giving me the opportunity to be here today. What I would like to do in my presentation is to provide an overview of UNICEF work for the collection, compilation and analysis of data on early childhood development. I was very pleased to see that a few of the presenters before me used the data that UNICEF produced or made reference to what UNICEF has done methodologically or also for the analysis of the data. I would like to have an opportunity to go a little bit deeper and provide you information on what exactly we do in the area of data collection and data analysis.

Let me start by giving a little bit of information on our main data collection instrument. UNICEF supported Multiple Indicator Cluster Surveys. It is a household survey program that was designed by UNICEF to support countries in the collection of data on children and women specifically in low and middle-income countries to allow these countries to collect the evidence that is needed for planning purposes, policy purposes, and advocacy purposes. But obviously, this is a global good that UNICEF is providing.

What we do actually is we support the data collection in the country and we support the analysis compilation of the data. These are not UNICEF surveys. These are surveys that UNICEF supports in low and middle­ income countries and I will give you a little bit more information later on how we do that. But it is a very important point that we care about. It is not about a single organization going to countries collecting data and taking the data away. It is building capacity and ownership through support in terms of capacity building, development of tools, et cetera.

The surveys being developed by UNICEF covers more than 100 indicators going from nutrition to health, ECD, mortality, child protection, education, disability, HIV and so on. These surveys really allow countries to get information on a variety of topics at the same time. These are not just traditional demographical and health surveys. Of course, there is an important health and demographic component to them, but they have been conceived with the child at the center. The objective is to gather information on multiple dimensions of a child's well-being and to get information on the child and the context, the family, the communities where children live.

Data are collected and can be disaggregated by background characteristics of the child and the caregiver to allow for the analysis of inequalities. We talked earlier. I really appreciated the presentation about inequalities that was delivered this morning. With this type of data, each single indicator can be disaggregated to look at disparities within countries between boys and girls, girls from ethnic groups and minority ethnic groups and majority of the population across families of different socioeconomic backgrounds so disaggregation is possible not only at the child level, but also at the family level. Rich families versus poor families. Families living in remote areas versus families living in urban settings. It allows for analytically to really look at the child disparities across countries because these are standardized measures that have been tested cross nationally, but also within countries. With DHS, with the Demographic and Health Surveys, it is going to be mentioned tomorrow with Sunita's presentation.

Multiple Indicator Cluster Survey represents the largest sources of data on their well-being on children and their families in low and middle-income countries. Five rounds of surveys have been conducted since their inception in 1995 so different rounds approximately every five years. As you can see, they were linked to major international goals. The most recent round of survey is what we call the MICS5 survey. They are currently ongoing. They will be able to produce data for more than 40 countries. They are going to be key sources of data not only to assess the MDG and to see whether countries have reached the Millennium Development goals, but they are going to be essential to set baseline for post-2015. For most countries, we are going to have already data that have been collected throughout the year since 1995 for topics that are currently not MDGs. We are going to be able to say data has been collected in ECD for ECD indicators. UNICEF and partners are going to be able to support the collection in the future so monitoring ECD indicators is possible because it has been done already. It is going to be very important from the past MDG discussion.

These are the countries where a MICS survey has taken place since the inception. This is since 1995. MICS survey has been conducted in more than 100 low and middle­ income countries and more than 240 surveys have been collected. These are nationally representative surveys, statistically sound. You can see it covers most of the countries for which we had said earlier we do not have enough research and enough evidence. From very small countries, to large countries, countries in war, countries that are recovering from past emergencies and so forth. In some of these countries, MICS surveys are the only source of statistical data on children.

This shows the variety of topics that are currently covered in MICS surveys. I highlighted in black some indicators and topics you might find particularly relevant for the discussion we had today. Although for me from the early childhood developmental standpoint, probably all of them are relevant, but I would just like to highlight some of the topics for which currently the data are collected in a MICS survey from specific early to measure about breastfeeding, anthropometry. We collect childhood development indicators that I will present later information on education including preschool attendance, on violent discipline, on the experience of violent and non- violent disciplinary practice used with parents with children age one and above. A very comprehensive way, if you want, to collect information well-being on children. We also have a module on child functioning and disability that we are currently developing with the Washington group on disability statistics is being tested in ten countries so far. We hope to be able to launch it in the next round of MICS.

The methodology that we used. First, we develop the questionnaire, the data collection tools, and we do the proper development through literature review, testing both at the cognitive level as well as the field-testing level validation and then analysis of the properties of the tools. These tools are developed by UNICEF in consultation with expert interagency groups.

But the development of questionnaire indicators is just one small component of a much bigger picture because if we develop the tools and we do not have the mechanism for the data collection, we do not provide a country with technical assistance in collecting the data, these indicators are not going to be used. It is important that when we think about indicators' development, we think about how we are going to be able to support countries in collecting this data that will allow them to report on the indicators that the international level we have agreed upon.

The implementation of the MICS survey is done by the countries. It is done by government organizations with the involvement of different ministries with the support and technical assistance from UNICEF through our network of countries and regional offices. For each country, there is a MICS focal point in UNICEF. We provide technical assistance to the government counterpart to undertake these surveys.

We do capacity building throughout the entire process because we train the national statistical office in survey design, in data processing, in the data collection. We assist them with the sampling. We assist them with data analysis and dissemination of the findings and writing up the reports. We don't do that. Countries do this and we provide technical support.

I would like to zoom in specifically into the ECD indicators that we cover in MICS with the understanding that, as I said earlier, there are many other indicators first time that we started introducing ECD questions in that are relevant to ECD that are already covered. The MICS was in 2000. Since then countries have collected information in the different rounds so we have the availability of trans-data.

In the first set of questions we introduced, it was about the attendance to early childhood development education. But we constantly improve our methodology and expand it. During the third round of MICS in 2005 and 2006, we introduced new questions that looked at the availability of books and playthings including toys done by kids themselves. I was happy to hear the presentation about Zambia before. We don't just cover about toys that are manufactured in the US. We ask about any toy built by children as well, the availability of books. The adult's engagement in learning activity with children, the interaction of the adult with children including fathers. We look at how many children are left in the care of siblings, a point that was raised before by Julie.

And then during the third round of MICS in 2010, we developed an index of early childhood development that looks at outcomes. These are the number of countries for which as of today, we have released publicly today on the different ECD indicators. The one indicator for which we have the largest number of countries with data as of today is the early childhood education attendance. But as you can see, there is an average of 50 countries, low and middle-income countries, for which we have data for all the other indicators. This is as of yesterday.

In the next six weeks, we are going to have 20 more surveys that are going to be released because countries are now finalizing the data processing, the presentation, and the results so we are going to have a large number of new surveys coming in the next few weeks.

Just to see how MICS does in terms of the other sources of data. UNICEF has maintained global databases on a large number of indicators, about 120 indicators. We maintain global databases. We include sources coming from different surveys, not just MICS. We compile data coming from MICS, DHS, RHS surveys, LSMS. Depending on the indicators, we also rely on administrative data. We do a process of obviously quality check. But we maintain global database from different sources. This shows what type of data sources we have for three core indicators: underweight, child discipline, and ECD. You can see that for underweight, it is really -- the situation is split. We have MICS survey, but we have also an important number of DHS surveys in our database and other survey, 42 actually of the total of 80 plus comes from other sources. But when it comes to child discipline, the most important ECD, MICS survey is the leading source of data because other sources do not cover the topics in a standardized manner. In the last three minutes and 28 seconds and I would like to provide an overview of some of the data. I apologize. It is going to be rushed. It is going to be quite superficial. But UNICEF is going to be releasing a major ECD report next year that will have in-depth analysis of all the data that we have collected so far. I want to tease a little bit your interest and show you some of the statistics. These are the proportion of children, 36 to 59 months so children below the age five, who attend some form of early childhood development program. As you can see, we have countries here. About 50 countries are displayed. You can see most of the country. Less than half of children are actually attending some form of early childhood education program. There are huge disparities not only across countries, but also within countries if you look at the weighted average in terms of school preschool attendance. For the poorest quintile population, only 6 percent of children are attending some form of ECD compared to 25 percent of children in the richest quintile.

The disadvantage of poor children is also visible in terms of the material that they have at home for learning. This graph shows the percentage of children under five so zero to four so under five, for which there are three or more children's book available in the homes. Again, every sort of book for children. As you can see, some countries have very low levels overall. But even in countries where the levels are high, there are huge disparities between the richest children and the poorest children when it comes to availability of books. Children are disadvantaged in terms of education attendance, but also availability of learning material in the house.

UNICEF has developed, as I was saying, during the fourth round of MICS a measurement of early childhood development. You have this brochure that is available outside together with this publication. You can pick it up and explain how we develop these early childhood development index that measure whether children are developmentally on track on four different domains. Then we have an aggregate measure that give you a sense of the percentage of children while developmentally on track overall based on the four different domains. These are the data that we have. This is a subset of countries. As you can see, in most countries, more than half of children are developmentally on track according to this measure. Obviously, as you can see, the low spectrum. There are countries like Chad, Sierra Leone, Mali, Central Africa and Republic, Congo, Cameroon. But here again, when we look at the disaggregation, eye level of wealth, we can see that there are huge disparities. In Nigeria, for instance, children from the richest, 20 percent of the population who are displayed here in purple are twice more likely to be developmentally on track than children from the lowest quintile. And the same can be seen across countries including those countries where you have higher proportion of children overall that are developmentally on track. This is disaggregation by domain. We said there are four domains. We can isolate those four domains to see where children are lacking behind. As you can see from this graph, literacy and numeracy, which is one of the four domains, is where children are in a way falling apart. This is the domain where the majority of children are actually not on track. They are unable to identify letters, numbers and so on. When it comes to physical domain on the other end, most children are developmentally on track across a variety of countries.

This completes my presentation. But again, you will find a little bit more information on what we do and the type of data that we have in the publication that are available outside. Thank you.

Care For Child Development Home Visit

Community Health Volunteer (CHV): Hello

CHV: Let’s go inside. I can see you are doing great.

Mother: We are all fine. How about yours? CHV: No complaint. We are doing great too. What is the name of your child?

Mother: She is called Rehema.

CHV: How many months is she?

Mother: 2 months.

CHV: We are in the rainy season, season of mosquitoes. You should sleep with your child under the mosquito net because it is a bad season.

Mother: I sleep with her under the mosquito net.

CHV: So do you play with your child? You talk to her or laugh with her? I would love to see how you play with her, while she is so young. How do you play with her?

Mother: I play with her by picking her up, by tapping her cheek and sometimes I love to make her laugh.

CHV: I would love to see how you play with her.

Mother: She wants to breastfeed. She is hungry

CHV: Does she want to breastfeed?

Mother: yes, she does.

CHV: the other one wants to play with her parent too. How do you play with them both?

Mother: When I am playing with the younger one the other child comes closer so that we all play together. I laugh with my companion sister. My daughter kisses her, and she taps her on the cheek. You just came when she wants to sleep.

CHV: It is ok.

Mother: He was playing well a while ago, but now she feels sleepy.

CHV: I would advise you to play with her. When she wakes up you can take a walk with her.

Mother: There is a time when she wakes up she plays, sometimes she wants to breastfeed, she wants to be bathed after finishing, she plays for a while, she goes back to sleep.

Vargas-Baron – ECD Measurement – Implications for ECD and ECI Policy

DR. VARGAS-BARON: I’m going to admit to you that I feel very humbled before the challenge. This is a huge question, a huge challenge for all of us, to think of what is needed around the world from all the countries with regard to early childhood development and early childhood intervention policy, and various types of programmatic implications that are involved. I actually began my work in measurement, in Colombia in 1972, to date myself, working with Dave Wyker and others, in the precursor to Sally McGregor’s work in Jamaica. So we were using the GRIFIS, and it was (indiscernible) and Charlie Suburg did some work with Jose Obduliomoda that resulted in some articles that I highly recommend that you read for the history of the development of the field, it’s a very interesting area.

Then later I began working on the CDAS, which is the Child Development Assessment Scale. And it too would pretend to be able to (indiscernible) use but it was not as rich as some of the other scales and it requires a lot of manipulatives and it would be quite challenging.

The instruments that were presented to you today really form a type of overlapping patchwork quilt. And yet there are many gaps in the quilt. And there are some experiences that weren’t presented, and I would like to pay homage to my friend Eliana Woolman who has done superb work in Chilie. You haven’t heard about it, but she actually was able to assess the levels of child development in population based assessments, and discovered the high levels of developmental delay by zone, which is region, of Chile, was formidably important, it ended up with a high impact in policy, and it formed what is now today the Chile Crece Contigo that continues on. And I wish to applaud her work. And she shows us how to do it. I wish that she would have presented because it’s an important contribution.

And also the work of Cecilia Brinebower who was a colleague in the work, who then carried it to Nicaragua and Honduras in population surveys that were national (indiscernible) population surveys like the DHS or the MICS. And I think that we have really a lot of elements.

And if anything I will call for opening up the collaboration, bringing together more people to work around this so that we can learn from the best practices of all the work that has been done, that were presented today and other things, and have a richer outcome for purposes of child development and policy. I’m going to speak today from the vantage point of national policy planning for ECD and ECI. And I’m going to look at this in an intersectorial way.

We are working toward achieving now the SDGs and also achieving ECD and ECI objectives. Some nations will try to achieve and measure improved child development using future measurement capacities of the SDGs, but not all will. So we’re going to have again a patchwork quilt of efforts around the world. ECI and ECD communities are in any case very challenged to improve their national measurement capacities, and they’re trying to do so, oftentimes devoting very little money to that effort.

Nations will continue to place their own needs first. Be very clear about that. We may see the global need for comparison, but actually nations will be rather parochial, they’ll look at what do we need to improve the development of children in our countries. Hopefully they will adopt some of the SDG indicators when we develop them. And those measures that lie behind the indicators are extremely important. Most likely nations will develop their own targets, they won’t use international targets. As of June 2014, 68 nations had one or more ECD policy instruments, now it’s up about 74-75. Ten countries were reliably reported to have them, and 23 were preparing them. 60 percent of those 68 nations were lower and middle income countries, which I think is very telling.

There is a tremendous drive that is real to work in the area of ECD and now increasingly ECI. Not all of the ECD policy instruments however included in indicators. Some consultants and people who worked with those countries did not include indicators. They’re more policy statements than they are really strategic plans or full-blown policy. Some of them have many indicators.

Now with regard to the types of indicators they tend to be predominantly operational or process indicators, mainly inputs and outputs that are essential for policy implementation. Again they’re being very self-interested in wanting to implement what they’re doing. They have fewer outcome indicators. And especially I find this to be the case in this region. (indiscernible)under the Soviet Union basically measured inputs and outputs and never outcomes, which is part of I think the reason for the downfall of the system, they didn’t know what the impact of their decisions were. So one of the things that’s very important is to inculcate, to develop a capacity to put into place systems for measuring outcomes, and then use those outcomes for purposes of planning.

The MDGs per se lacked an ECD goal. However seven, not just two, seven of the eight goals had ECD related indicators. When you look at the indicators list, which is rarely published, you will find them. In a book I wrote in 2005 you will find them listed there. So we used those under the MDGs as proxies for a lot of the work we do. For example stunting is one of the major ones. Many MDG indicators were measured at both national and global levels, such as infant mortality, that probably will not be the case with a lot of the SDGs. The SDGs present an opportunity for us, if we’re wise, if we’re strategic, to have effective ECD and ECI outcome indicators. Now actually ECD is under more areas of the SDGs than you realize.

Poverty, there are four different areas: young children living in poverty, services for children in poverty, resources, nutrition and food security. I won’t read them all, but there are two under there. Health and wellbeing, there are two. Education, there are four. Gender, there is one. And then water and sanitation. Water and sanitation is critically important for ECD. This is where you get your waterborne diseases that are so terrible. There is a direct relationship between water and sanitation quality and child development quality. Reduce inequality within and between nations, there’s one on child rights, which is for us. And then safe, resilient, and sustainable settlements, there we go, for children and women, and they’re mentioned. Peaceful and inclusive societies, the 16.2, but also 16.9.

So we do have an opportunity if we look at it very carefully and study it to have an influence in those area. We may not do it on the first round of indicators, but we know from the experience of the MDGs that it wasn’t all done in the first round, it took about four years under the MDGs to get the full list of indicators out and used around the world. So don’t get so worried about gee we’ve got to do it this year, it’s going to be an evolutionary process. Key issues. ECD specialists realize, and others don’t, that the quality early childhood development means so much more than appears in its write-up. And I’d like to point out something that has become a real problem, because we all want learning. Ten years ago I fought for education to be learning for all, and I lost, and now we’ve got a learning emphasis in education for all, which delights me. But child learning is not the same as child development, it’s a subset of child development, and we have to take that broader view of child development.

So if we only measure child learning we’re not going to be measuring child development. ECD is not just one to two years of pre-primary or school readiness, it just simply isn’t, and we have to say that over and over. And people in the UNESCO Institute of Statistics, and World Bank, and others are going to tell you no, you have to use the time proven measures that are basically input and output measures of coverage and how many teachers. This isn’t enough, and we have to fight that. Then ECD includes child health, nutrition, the sanitation area, I insist, education rights, and protection. With regard to countries, national, regional, and municipal needs will come first. They want to know about their fragile birth status. They know that children have a whole series of areas that are important to be assessed at that level. Level of child development from infancy by age bands is wanted, just as Eliana did for Chile. They knew they needed to look at what is the level of our development in early age bands, nine to 12, 18 to 24, et cetera, et cetera, so we have a tremendous need to really get into that period. I was thrilled to see the emphasis in the presentations today on the zero to three, as well as the 36 months and over.

Malnutrition and chronic illness, I cannot emphasize the importance of that enough, and we know from the Lancet study, finally the message got through, that really stunting at the moderate to severe level is a proxy for developmental delay, and I use that constantly because it really is the case.

Disabilities and atypical behaviors now are becoming of interest not only to ourselves but to countries because they realize they don’t have two to four percent, they really have around 40 percent, 50 percent, when you add up all the different types of children’s situations that cause developmental delays.

Then disabilities and atypical behaviors, we had the wonderful presentation of the autism area, that’s one of the areas of atypical behaviors. These are very important to be included, and countries know that, policy makers know that now. High risk status and child protection, ethnic and linguistic groups, moderate to severe poverty, you know the list.

And then I was very pleased that we had presentations on ethical and rights considerations, the CRC, CRPD, Child and Family Rights, confidentially informed consent issues. The general comment seven wasn’t presented but I want to call your attention again to that. Read it again. It is very much for us today as we work on this.

Some nations are measuring only as I mentioned input and output indicators, and we need to put much more emphasis on outcomes. The outcome indicators are essential for measuring effectiveness, quality, equity, transparent accountability. Most nations that I have worked in are very interested in having good indicators and measures for ECD and ECI.

The problem is they don’t have the requisite expertise, and we really have a technical assistance role, I hate to use the term, it’s a technical cooperation role, but countries do need more knowledge in not only what indicators, measures, and targets, and how to set them, but also in how to validate instruments. Instrument validation is critical. We can’t just develop instruments, we have to validate them. I won’t go into how, but it just is very important.

Then the ECD MIS and monitoring and evaluation database development is absolutely critical, and that’s the weakest part. We need to think about how will the measurement be used within national SDG plans. They’re going to be done. Will they include ECD or not? Then within existing ECI, ECE, ECD, and IE inclusive education policies we need to be thinking about this, and then within new ones, and I think amendments are there needed, within new ones we have a need for updating and getting additional ones.

I want to tell you some additional challenges. We have a very chaotic list of SDGs. And we have a new series of quick and dirty indices that are being developed. One of them I’ll mention, the social progress index and the People’s Report Card, and this is completely left out ECD and marginalized populations. Many more of these indices also are doing the same thing. So we have to work very hard to promote our area at this time.

And we need to be involved in the national SDG plans. That to me is going to be critical. We’re not ready for it, though. And it is going to take us I think at least five years to develop a good system of indicators, rather than thinking we have to do it tomorrow we should take our time and do it well, it has got to be done well.

We should include policy makers and policy researchers in the effort. This should not only be done by measurement people. I love the way IDELA has taken the results and has shown the results to policy makers and gotten them to make changes, and indeed that’s what Eliana did.

We need a research agenda, I have a series of suggestions that have to do with this that cover areas that you can read there in another list here, but I’m down to one minute. And you’ll notice that I cover all the areas, even birth registration and service access, because we really have to get new ones. We also need targets for government support for ECD and ECI at all levels. Remember, governments have to pay for most of this.

And the final point I have is that leadership is needed for ECD, ECI, and measurement for example, and this is from Enrique Peñalosa, the new mayor, returned mayor really, of Bogota, Colombia. Children are a kind of indicator species. If we can build a successful city for children we will have a successful city for all people. I love that. Thank you.

Raikes - Measurement Birth to 8 years

Good morning everybody. Thanks so much for the chance to talk about measurement. I was joking earlier that if we still had bards that we should have a measurement bard here who’s sort of singing us songs about measurement. So as Hiro said I’m going to be talking about measurement for kids birth to eight, and particularly in light of the sustainable development goals.

So in terms of the main points and themes I’m first going to give you a little bit of background on some of the details of the sustainable development goals and the context that that provides for measurement. And fundamentally I think that we as an early childhood community have some decisions to make, and that as we’re facing this new agenda measurement becomes one set of very important decisions that we have for positioning early childhood in the global dialogue. Then I will also be talking about creating equity through measurement in line with the theme of this meeting of what does it mean to promote equity through measurement, and what are some of the steps that we need to take, and then how do we realize the contribution. So in terms of setting the stage for measurement, so early childhood development has made its global debut in the sustainable development agenda. And it’s mentioned I think as came up yesterday in more than one place, it is in education, the target specifically focused on early childhood sits within the education goal. It’s target 4.2. The words in this target are not negotiable. So this is the target that we’ve got.

The amount of time and energy that it took to even get to this language was something that we all aged many years in a short period of time. So the challenge here for goal for target 4.2 will be creating a holistic enough dialogue and making sure that people understand that development starts at birth. Also then integrating non-formal pre-primary programs, parenting programs, health and nutrition, integration of all of those. Those will not necessarily be intuitive to those who first see the language of 4.2. So that will be one of our first challenges.

It’s also the case that these goals are interlinked with one another. So early childhood has implications across many of the other goals, which means that then it will be upon us as a community to reach out to the other people working in the other sectors to help them see what the role of early childhood is in the work that they’re doing, and for us to figure out then how we can best support one another’s efforts.

Overall this is such an important era for early childhood development. And when I hear from member states and people who really don’t know anything about early childhood, what’s really remarkable to note is how much understanding there is of the importance of early childhood even compared to five years ago. So we have this incredible opening and this opportunity with many new program models, growing evidence of the importance of the randomized trials, and measurement as one important element to track the trends and investments at a population level.

So I’m going to pause just for a second on population level measurement. And so what I’m saying there is that it’s measurement that really is designed to provide a picture of an entire population and not to diagnose individual children. And when member states and other conveners get together that’s really what they’re talking about when they say they want to know what’s happening in early childhood, it’s really population based measurement.

So in light of that, what do we see now that’s different about SDG measurement? The agenda is very different and the measurement will be very different as a result. The first point is that whereas the MDG goals were all very specific and very measureable and very global, there’s now a huge emphasis on regional and national monitoring. So there will be a small set of global indicators, but there’s also a big push coming from member states saying we have measurement systems, we work within our own regions, we don’t need somebody to tell us what type of measures we should be using. The second is that there’s an emphasis on outcomes that cuts across all of the targets in education, and it’s not only access. And then quality becomes very central. I think there’s growing awareness that you can’t just have access, and then instead you have to really be focused on the quality of services.

It’s also the case that there are many expansive terms all throughout the agenda that are not defined. And then how those terms are defined then of course becomes a really critical piece of measurement. We have many new measurement innovations that we’ll be talking about later in this session, and more coming all the time. And at the same time we have resources from governments that are very limited in terms of investments in measurement. That’s not something that governments often want to spend a lot of time and money on.

And then the data have to serve these multiple users. We’ve got globally comparable is one factor at Hiro mentioned earlier, and then we also have got to be able to use data for improvement. We’re at a point right now that we can’t really waste too much time of collecting data at a population level if we can’t figure out exactly what we want to do with it and how we want to use it to improve services for kids.

Finally, we’ve got the data revolution, and I think the session this afternoon will be very interesting from that standpoint as well. We’ve got technology, we’ve got a lot of new methods, we’ve got potential new partners, all of whom can help us get to the next level in ECD measurement.

So I wanted to show you this measurement framework for children birth to eight that we published in partnership with Kimber and her team, with Terran Dewlit and Pia Rebello Britto. And I think that this is a very simple representation of what I think here collectively in this room we agree upon, that contextual factors have an enormous influence on child development, that inputs are important such as promising practices, that outputs, coverage of programs is really important, which in turn lead to child development.

So our challenge now is how we translate this into feasible measurement at scale. And so if we all think that this is true, what do we now need to do to make sure that that shows up in all of the measurement systems that countries and regions in the global system are using?

The good news is that we have lots and lots of experience to draw upon. In addition to the measures that we’ll be discussing in the afternoon, and some of them are listed on here, the IDEALA, we’ve got Eva who will be talking about the IDELA, we have lots of regional efforts of population based measurement at or near the start of school.

I think politically there’s much greater acceptance that I hear at least of measuring kids’ development as they enter the primary school system as opposed to other phases of life. So what you see here then are measures that usually start between the ages of two and three, extend up through maybe age five, and are designed for use at the population level, meaning tracking groups of children. So you can see that there really is quite a lot of experience that we can build on. So one of the key elements of the agenda too is that it’s intended to apply to both high and low income countries. At this point in time most of the measures that we have are either for low and middle income countries or for high income countries, and there are very few that cross that, even though of course we know that there are lots of inequity within countries as well as between, so that will be another issue that we need to figure out how to deal with.

So then going back to measurement and its contribution to the SDGs, I think that whether we like it or not measurement will help to create a common language. And once we put something on the table in terms of how it should be defined that will be a contributor to how people understand the sustainable development agenda for early childhood.

So whatever that understanding is of school readiness, which is not a term that I necessarily like very much but one that shows up a lot in the public discourse for a definition of quality that protects children’s rights and supports their environments, these types of elements will be very important for the dialogue.

And then as well, even though we’ve made a lot of progress in the early childhood community in terms of helping people understand the importance of early development, it’s also the case that within the education arena and also sometimes among people that I talk to in health there’s a sense that we as an early childhood community are still a little warm and fuzzy, and that we’re not quite as rigorous as we need to be about our data. I don’t know that I agree with that, but it’s just a perception that’s out there and something that we’re going to need to address.

So in light of that I did want to give you a quick update on the indicators. So there will soon be an agreement on one set of indicators for global monitoring. There was a meeting last week in Bangkok where they made quite a bit of progress. There will be two proposed indicators for goal four target 4.2. Right now it’s the MICS ECDI, which is an index of early childhood that MICS collects through a household survey, and access to quote organized learning.

There will be a very strong push we’re anticipating to replace any outcomes measurement for early childhood with access to pre-primary education only as the global indicator for early childhood. So that’s what we’re anticipating within the next few months. We probably will need some very strong support from member states in order to advocate for anything other than access to formal pre-primary as being the indicator. We’ll see how the process unfolds.

At the same time we’re going to have sets of national thematic and regional indicators where there’s quite a bit more flexibility, and where hopefully there will be an opportunity to enter into a dialogue with member states about what’s important to measure and how they can set up measurement systems. And again, that we need to come up with something ideally over time that applies to both high and low income countries, in line with this focus on decreasing inequity. I think that one of the other key questions, just to keep putting it out there, is that we as a community need to decide how we want to use the high leverage that comes from globally comparable data against our understanding of the most accurate way to characterize young children’s development, and so that will be a question that we can continue to debate. So if we’re moving then from the global where we have this high degree of interest and political leverage right now and this opening, of what’s happening within context for children, and how do we bridge that gap. So I did my best to sort of summarize what I think would be ideal, and I’m really eager to hear people’s thoughts, and where we are a little bit more commonly.

So if we said that the ideal is child development and learning, the measurement reflecting culturally relevant normative development for all children, meaning that we have good scales that include all children, that we’re measuring context as well, that we’ve got family environment, schools, exposure to violence, a broad definition of context, and also that we have actionable data that derives specific changes for children, that that might be something characteristic of a system that we would eventually want to get to.

And where we are now or what we tend to see a little bit more frequently is project level measurement using a variety of tools, but not necessarily scaled, which governments may or may not necessarily want to take up these scales and continue investing in them. We see a tendency towards a very narrow definition of school readiness, less investment in measuring children’s environments than in outcomes, and usually at one point in time. And that very little measurement of access to services or children’s development early in life.

Even though we know, as we heard on Monday from the World Bank, great presentations yesterday too, that’s too narrow, and we miss a lot as a result. We also have unclear use of data to systematically improve. And I think that the tendency at the global level is to really think about population level data, which is fine, and it serves a very important purpose, but there’s a big question out there of what data actually leverage the highest amount of change.

So but then I’m putting out some principles around measurement and service of equity for us to discuss, and I think that the presentations later will also be really helpful in further articulating these principles. The first one would be ensuring that all children are counted and their skills are acknowledged. So that means that we would be creating measures for use across all settings so that we can capture children wherever they are, and for children from birth to age eight. So but then I’m putting out some principles around measurement and service of equity for us to discuss, and I think that the presentations later will also be really helpful in further articulating these principles. The first one would be ensuring that all children are counted and their skills are acknowledged. So that means that we would be creating measures for use across all settings so that we can capture children wherever they are, and for children from birth to age eight. Now there’s a question of whether then it needs to be normed for the local population. Norming scales is a big job, it takes a long time, and it’s technically challenging to do, but I think that that’s a question that I would love for us to discuss, is how much norming is necessary for us to come up with accurate scales. That we would be measuring environments as well as outcomes, and that the data would show a clear pathway from action to improvement.

So in this next slide there are a lot of words, but I’m going to sum it up really quickly. It’s basically trying to outline a role for the global I’m promoting local level measurement, so that if we begin with a certain set of principles that we all feel we can agree to around equity based measurement, then can we come up with some tools and methodologies that then are adapted at the local level.

So, but as we move to this larger scale measurement we’ve got a number of tricky questions to resolve. The first one is the screening and population based measures, of how we bring those two together, so that if we’ve got great tools for identifying children with disabilities, if we’ve got population based tools how do we put them together to come up with a measurement system. The second then around measuring holistically and then around using the data for improvement.

So now I’m going to talk specifically about different types of tools. In this little chart I’ve put together four different types of tools. There’s population based tools, the diagnostic or formative, the screening tools, and contextual. And so these are all different types of tools that might be taking place within countries at the same time, and then they all have different reasons why they’re used and how the data are collected. And I think that over time ideally we would be able to help countries put together systems that pull from different types of data. I think the odds of countries adopting one measure to the exclusion of all others is probably unlikely at this point.

And then in terms of the tensions of integrating data from the formative and diagnostic and population based is a question I would put to this group too. Can we do that? Is it possible for us to use some of the measures that have been developed for formative use or for diagnostic use and to use them to track trends and to describe inequity at a population level? And if so, how should we best do that?

We also have the pros and cons of using common measures across diverse settings, so that if we have global measures that have been developed in one place, what’s the sense of how those can best be used in other places? And then finally the different types of measurement might require different approaches. So quality measurement for example, it may not necessarily fit into the same type of structure that measurement of child development and learning would have. So that then in the projects that I’d been working on we have this question of how do we make measurement easier, that in low and middle income countries the number of people who are available to work on measurement within governments is usually pretty limited, and it’s not necessarily a high priority, nor do I think that it necessarily should be given how many demands there are on services.

So then the question becomes what do we do to make it easier. Is there something that we can do at the global level that both reflects the principles that we’ve got and also bootstraps and makes the work at the country level a little bit faster? So there are four different levels that we’ve outlined then. One is that we use the same measure everywhere, the same administration, the same items, with a small amount of adaptation. The second is that we could use a common core of item, one small set of items, maybe part of a larger set and a more culturally adapted set, but one set that we feel meets a standard for applicability across settings.

We could try to just have common constructs, with items that vary. So maybe we agree that there’s a certain set of things like social and emotional development, or the quality of teacher child interaction, but the definition of how that’s measured in different places may be very different. And then finally that we just do an item bank. And so we say to people here are lots and lots of different ways that you can measure all of these different elements of child development, which do you think is best? All of these would allow us to have some sort of message in the global dialogue. But the level at which we are then attempting to summarize our information would vary quite a bit. So then that would be a decision as a community, that hopefully we could come to some sort of understanding of where we think we’re going to get our highest political leverage.

I’m going to talk now about two projects. One, the measuring early learning and quality outcomes project, which is a collaboration with UNICEF, UNESCO, World Bank, and Brookings institution, with support from the Children’s Investment Fund Foundation and lots of partners and lots of people who are involved, who are here with us today. And then also the World Health Organization Terandua has asked me to present a little bit of information about her birth to three measurement project.

These are both examples of how we’re attempting to address these questions of measurement at sale. And I won’t say that we’ve got the corner on any answers, I think it’s more just a sense of how we’re trying to think through these issues. So first, to begin with the World Health Organization, of global indicators for child development of zero to three years. So I think with the World Health Organization project as well that the Bernard van Leer Foundation has also been very involved in it, so that as questions come up I’m also very happy to have colleagues there to answer questions. So the first question that they had has to do with potentially existing and validated cross-cultural and feasible indicators that they could adapt for wider use. There are also then a focus on the magnitude of the developmental fluctuation between children between countries, meaning how much variance do they see. Can we collectively define normal and concern and delay? Which developmental milestones observable before age three are reliable predictors of outcomes? And then this feasible process for developing and implementing.

So that then in pursuit of their goals they’re systematically assessing data from developing countries on children less than three on core items of global developmental competencies, evaluating the feasibility and applicability of all the valid items, and field testing.

So the process that they have done, it has been quite amazing to watch, as they went through all of the different measures and then said do these items look to be the same thing? If a child kicks a ball, a child kicks a ball four feet for example. It might be two different items, and then measured in different ways, and they ask the question of do these items work in the same way or not.

So this is an example then of how they’ve summarized the data, and this was an item that they said had good item performance. So what you can see then is that for each of those different colored lines, those are different measures. So the ASQ in Indonesia, the ASQ in India, Peru, the MDAT which is from Malawi, and then the Kenya Development Inventory. And by looking at the average ages at which children achieve those, the conclusion would be this is a developmental milestone, because at the decimal age at the bottom you can see that those lines all line up around year one. So this one here it’s walking, so yes of course you would think yes that would be something that would emerge on a relatively consistent timeline, so this would count as a good candidate for global measurement, and that then would be something that could be considered a milestone in a lot of different places.

So for a very different approach, for MELQ what we did is we began with expert groups on quality and child development and learning. We do have some of the same depth of data that exists for children’s development birth to three, but as we all know the nature of children’s development as they get a little older changes. So the environmental influences are greater, and you can even see that in the WHO data as children get a little bit older that those lines start to spread out a lot. So that tight curve that we see around walking, you just don’t see that really as much as kids get older, so it requires a different approach.

We used existing measures to come up with a new set of items, we drew heavily from the IDELA, Save the Children’s Instrument that Ivelina will talk about. We also use the ASQ, the Denver Developmental Screening Scale, anything that had been tested in more than one country. The Easy Asia Pacific Child Development Scales was another place where we drew items. We have two new tools then on Child Development Learning and quality. The purpose of this exercise though wasn’t so much to come up with a new tool as it was to come up with a method for defining a common core. So we were sort of banking on that second option that I showed you earlier, which was let’s say that we do have a good deal of national adaptation, but can we come up with a common core that we could use to compare across settings?

We’re also looking at some of the institutional supports that are needed for scaling, and we have a nationally representative study underway. So I’m just going to show you the framework that we used for developing the child development and learning items. So we had this core set of items, and again our goal of can we find core items in social-emotional, in domain general meaning executive function, and in domain specific early literacy and math.

And we now have data from seven or eight countries, and the very quick summary is that it looks as though there’s more consistency in the way that children respond to items in literacy and math than there is in social-emotional development, and executive function is somewhere in the middle.

So it really becomes a challenge politically then with that idea of a common core, because we could be in a position of communicating to governments that we have a common core for pre-academic skills or literacy and math, however we would want to call that, but not for social and emotional development. And I’m afraid that if we were to go that route we would be reinforcing the idea that you don’t need to measure social and emotional development, which couldn’t be farther from the truth. So I think that those are some of the tensions that come up when you actually start to look at the data, is that things get a little bit tricky.

So then just a few words on quality measurement. We also have this tool for Malco that was focused on quality, and we began with formal pre-primary environment. So not because there was any sense that that was the most important type of environment, simply because we needed to start somewhere.

And so we began with those environments, and we had questions around whether it was possible to use political leverage to measure quality. So it was not part of the MDGs, so there was only access to primary education. It was considered a huge mess in that many children enrolled in extremely low performing schools. And so now politically there’s an opening to measure quality in ways that people will pay attention to.

And then we had this question of whether it was possible to define items or constructs that are comparable across countries to at least serve as a starting point, and we hosted a meeting last year with Bernard van Leer and ISSA around this question of common constructs, to begin addressing that question. And then we had this question of whether quality measurement could be connected to child development and learning, and then also how it could be built into the monitoring and evaluation systems for an emphasis on improvement.

So I’m going to talk for just a second about quality monitoring, because I feel like it’s a really critical piece of quality measurement. So in most countries there’s some standard, there are standards written about what quality setting should look like. People may not know what they are, but they’re there. They’re usually written there.

So what we did, we looked at the SUBAIR ECD, which is a world bank project, and OECD on the state of monitoring and evaluation systems around the world. And what we found is that there’s very little consistent information about monitoring quality of services or implementation really anywhere. In some high income countries they do a good job, but in a lot of places there’s just not really much that seems to be available. There’s agreement on many of the key constructs of quality. So like the principles that ISSA put together and some of the other principles that are out there, I don’t think there’s a lot of disagreement about those, I think people have good sense of what the characteristics are of quality environments, but there’s very little support for actually improving. So we may agree on these constructs, but there’s not much in place that actually helps people reach them.

And finally there are very few tools that are available to help countries measure quality at scale, and of the tools that are available they’re really narrow in focus. So there might be one on formal pre-primary environments, but there’s nothing on home visiting, there’s nothing on services for children with disabilities, there’s nothing that really goes beyond a very narrow definition. So I think that the sense is that we have a real opportunity to try to figure out how to move forward in measurement of quality.

So when we’re thinking about measurement and quality and how they fit together with child development, that we put together a conceptual model then that outlines rules at different levels, beginning at the bottom level. If we said at the policy level that would be the policy structure, and that’s an extremely important piece of quality, meaning the types of regulations that are in place, the connection to the national standards, and the amount of funding that’s available that our group considered to be a key element of quality.

Then at the system level of professional development, which again seems to be largely absent from many of the countries that we looked at, at least from the information that we had. And the resources in the school leadership. And then to the classroom or setting level, and then finally child development in a different color, because I think that the message that we’re trying to send is that you can’t just measure child development and learning without understanding how it fits together in this larger system.

I think in terms of data use over time we would be able to identify a decision maker at each of these levels who could use the data. So if you’re thinking about helping somebody make better decisions with data then you have to define what types of pieces of information they would like and how we can help produce that information.

And thinking then through to how quality influences child development and learning. This table here, it has backwards arrows on purpose, because it begins with the policy side of the policies that need to be in place, then all the way to what the children need to achieve, so that if you had adequate funding for preschools then you would also need curriculum that has age appropriate expectations, you need counting materials, and then maybe you could expect children to learn how to count in the way that you’re anticipating.

So we have a little bit of experience to date. We thought as I said earlier there isn’t a whole lot of disagreement among our expert groups about the constructs, but I think Hiro had a very interesting experience in Colombia recently that he might be able to mention, that they didn’t necessarily think the constructs worked for them. That they had different names for the constructs of quality, and that what we had generated didn’t really resonate with them.

So then the items too may substantially vary from one place to the next, and that we’re finding that there’s not necessarily a lot of alignment between an idea of global ideal of quality and how it’s implemented within countries. So then the next question becomes defining quality ranges, item banks, what’s the solution? Is there a technical solution that allows us to take advantage of political leverage while still saying true to what countries are experiencing? And then the implications for equity in measurement. I think that one of the other experiences that we’ve had, certainly a principle that we’ve got is avoiding high stakes. And one way that you can avoid high stakes decision making on data is by sampling, so that you never have full information on any population of children, and it’s actually a lot less expensive too, so that you can then pick and choose and do representations.

I think that the question of high stakes testing especially around school readiness is a really critical question in many countries. We have the sense of even talking to some government officials that making the decision to have some children out of the primary system would actually be beneficial, that they would prefer that, because these children don’t necessarily have a strong basis. So that’s something that we’re going to continue to have to monitor.

And of course the definitions of healthy development, quality, and equity vary by place. And so that question of how we get the cultural norms, and the lens that we need in order to adequately interpret the data, continues to be something that we spend a lot of time thinking about. And then using the data to build effective professional development systems and MNE systems is really important. My sense is that we collectively don’t know enough about that, and that that’s something that we need to explore more fully.

So in terms of our next steps, what we see at the country level is a lot of demand, and a lot of willingness to engage with very few resources. Very few structures in place in many countries to support partnerships with researchers, and other experts who are coming from local universities, so that if we can help facilitate those strong connections between people in countries working in universities and the government that might be a way that we can support them.

And then at the regional and global capacity we’re seeing many investments in measurement happening in many organizations, but it’s not necessarily coordinated. We don’t necessarily I think have a sense of how to best maximize the investments, and we need to potentially invest more in building measurement systems.

And then in terms of the discussion questions that I had upon looking at the other presentations of the successes of the measurements to date, how we can build upon them, how we can take advantage of the global emphasis on ECD, and then how we can best work together to support the integrated action at country level, and how we can use our collective experience to move forward. Thanks.

Heymann - Comparable Data

All of our countries can benefit from truly comparable data. Now each of us may want to compare our experiences to different groups. So we may think that the country’s experiences who are most relevant to us are those in our region. They could be those who are in our income group. They could be those who share political systems or societal structures to ours. But we can always benefit from looking at other countries that we think are truly comparable in saying what are they doing that works? And as well as sharing what are we doing that works. Because of that having information across countries on basic issues. Do mothers have paid leave; do fathers have paid leave? Can breastfeeding breaks be taken? How affordable is early childcare; how is it made affordable? What nutrition programs are there? What programs are there to support parents and other caregivers in stimulating children? That allows us to look at who’s getting the most successful outcomes and what can we learn from those successes. Now, of course, each of our countries and in fact each of our states or provinces will also have some questions that are very specific to our local circumstances. So it’s important to add that kind of data and information that allows us to answer our local questions at the same time.

Heymann - Child Outcomes

I think we should be able to know that all children have a chance at healthy development; that they are all ready to start school. So much when you look at unequal outcomes in primary school; when you look at unequal outcomes in secondary school; that’s all been determined by the state children were at when they were five. We need children to be able to have an equal start at school which means they have to have had enough health, nutrition, development at that very beginning that they have an equal chance. That they’re really starting at the same starting line. So we should be measuring that and we can. Right now, we’re really just measuring the most basics across countries. Do children survive? Of course that’s crucial but that’s not enough to know if they’re going to have an equal chance to thrive regardless of their gender, regardless of where they have a disability, their race, ethnicity, their income, their social class. We need to begin to measure, are they all ready to thrive?

Marfo - Outcomes

So if I say that siloed research, especially applied work done in siloes is one of our biggest problems the other big problem is really the whole issue of how we actually address the issue of outcomes and how we measure what we do. And I think I’ve always used the example of even in the natural sciences where most of us would have thought that the natural sciences are so objective that you don’t need to worry about what is different from place to place. Like a metal is a metal is a metal. It doesn’t matter. You want to heat a metal to 600 degrees Fahrenheit, it’s going to increase by about three inches, you do it Ghana, you do it in Canada, you do it in Singapore, chances are the answer will be the same thing. Human development is not like that. Human development is really primed by social and cultural processes. And so the context in which people live their lives is very, very important. So, but as human society researchers we are confronted with one problem with two sides. One side is biological existence. And of course, that is what we share. Human beings across all cultures, we share a common biological heritage. So the trajectories of development are driven by biological processes which do not vary a whole lot going from one setting to the other. But there is also the contextual path of our development. The culture in which we live. The geographic locations and, in fact, the social and cultural realm there are some elements that are universal. I mean, most cultures have families. Family is another universal concept. So it doesn’t have to be biological to be universal. You can get ecological, environmental things that are actually universal as well. So here is the challenge. The challenge is as we try to understand the outcomes of human development, you know, how do we do it or are we trying to deliver an intervention? How do we do it? Do we think we have a program that works for everybody? And therefore, all we have to do is just implement it. And I hear that a lot in the circles where some of my colleagues like to tease me as someone whose office hasn’t changed or what happened to you Kofi; you’re always talking of context and I say context is very important.

So if my goal is to create a scientific database that allows us to understand what is common across our universe it’s a very laudable goal. But it will also mean that we will have to be ignoring a lot of things that are very important but are localized. So localized that you cannot measure it in the same way across context. That’s one of the biggest challenges for international comparisons right now, the way I see it. So every time somebody says, “well, but we cannot be just studying individual settings. We need to look at what cuts across.” And I say, “by all means, let’s do that. But just remember that by the time you isolate the things that you think you can compare across cultures, you’ve already dropped a lot of things that are very important but cannot be compared across context because they differ from one context to another. ” So how do you balance it? I think there’s a practical, commonsensical to balance it and even the international organizations like UNICEF, in some of the databases they are creating like the MICS are beginning to have what they call the more general kinds of questions and then questions that are designed in addition so that countries can determine whether they want to gather data on those questions that are very, very localized. When you do that you get the best of both worlds.

Marfo - Catching Data

Now, I bet you that these international organizations when they get the data are not going to analyse the local data. They don’t think it’s that important. I really don’t. I’m yet to see a major project that gathered international data, that then reported findings for individual countries. We are always going for the gold. We can catch both worlds. And the way you do it is to make sure that there are the things that are instruments that get at the commonalities. Now, I’m not convinced, however that we even know how to capture what we call the commonalities, because very often when we say that it means that we have an instrument that was developed in Canada or the United States that we are going to try to use across the board. And even when we think we are trying to make it more valid for other places, we think of adaptations and very often what we mean by adaptation is translating it from one language to another. Actually the linguistic translation does not necessarily give you the basis for comparison because the conceptual underpinnings of those items could be very, very different. So in one culture you think of the concept of conservation and people understand it. You go into another culture, they have no idea what it means. So even the process of recognizing the limits of comparisons it requires us to do more. But guess what? To do more often it is very expensive or demanding of our time and effort and we are a world that believes in getting things done as quickly as possible and we use tools, not because somebody has decided that these are the best tools to use. Very often we use those tools because they are the tools we have.

Borisova – IDELA

DR. BORISOVA: Good morning everyone. It’s really a pleasure to be on this panel on measurement. My name is Ivelina Borisova and I work with Save the Children. I’ve been doing a lot of work around our impact and evidence building globally. And whenever I am at meetings like this talking about measurement people usually do a double take. You’re at Save the Children, aren’t you implementing programs? In fact we are, we’re implementing ECD programs in 75 countries as we speak, but we’re also extremely grounded in evidence based approaches and solutions, and this is where our interest in measurement is driven by and I think grounded in as well.

So what I’ll talk about is our efforts over the last couple of years to develop an instrument, which is new in some ways and old in other ways, to really help us both track impact of programs, be accountable to donors, communities, governments, as well as be able to have some comparable data across different sites that we work in. And this effort has grown, starting with Save the Children and has now kind of grown into partnerships beyond our expectations I think in really positive ways. So a lot of what I’ll show is not just work by Save the Children but also work by other partners who have kind of joined the effort in helping us learn how this tool is working across sites.

So a brief overview of what Hiro mentioned. And again it’s a high order for 15 minutes. It might feel like I’m skimming through a lot of important information. Feel free to ask questions at the end. I’ll briefly speak about some characteristics of IDELA, the International Development in Early Learning Assessment. I would really like to spend more time on exploring the margins, because this meeting is very much about children at the margins, and what we find by looking at data for what margins actually look like and how we define them.

And then I’ll talk about tracking effectiveness of programs for policy change, and really implications, and this idea of how actionable is the data that we’re getting from these global assessments and what can we do with this data and what has already happened in some settings. And I’ll very briefly touch on future directions.

So let me start with the characteristics of IDELA. And I think it was very nice to have Hollie before me because she was speaking about the ASQ which is really focused on the younger age groups. IDELA is very much targeting pre-school, pre-early primary grades kind of age group. So three to six with extensions to seven in some contexts depending on the level of experience with early childhood or background of those kids. But it’s very much in the middle part of the continuum of early childhood development.

When we embarked on kind of this journey of developing a tool that would serve our purposes and agency, which is really a global purpose, we wanted a tool that we would be able to use globally, but one of our top priorities is that it would be feasible and possible to use in low resource low literacy environments, which is very much where save the children does the majority of its work. And this is where the gap that we were hoping to fill was, because when we looked at available tools there were really very few that seemed feasible and that actually made measurement possible in these sorts of environments. So a really underprivileged low resource community.

We obviously wanted to be informative to the communities where we work to programs and to policy makers, so looking at different stakeholders and again thinking about how actionable the data that we receive from an instrument can be for policy change. And of course we wanted to have some international comparability, so that we can say something about how our program is working in this setting versus in other settings.

So this is kind of the backdrop of the tool. I will not get into the story of how IDELA was developed, but I will mention just a couple of important points I think linking to Abbie and to Hollie’s presentations. I think what’s interesting about IDELA is that it balances very nicely kind of the bottom-up and the top-to-bottom approach to development. The top-to-bottom approach is kind of linked to, IDELA is based on existing tools and on synergies and overlaps that already existed in constructs, and in what other tools out there were already measuring. So in a sense we didn’t necessarily come up with new constructs, we really looked at where the connections were, where were the overlaps, and we tried to find which of those items and constructs were most feasible and most relevant across many settings.

The ground-up approach is really how IDELA was validated and really brought to life in terms of being a feasible and practical tool. Those items were then tested in 12 different countries in very diverse populations, looking specifically at the practicality and the feasibility of what it’s like to measure these particular items across these populations. And there’s a lot of careful considerations, both qualitatively and quantitatively to really make final decisions about which items really made sense to retain and to really constitute the tool that I’m describing to you today. So I really like these bridging these two components.

So what is IDELA? I’ve already talked to you about some of the key characteristics. It’s a direct child assessment, it is intended for global use in the sense that we have already used it in a lot of different countries, and we meant for this to be used in varied contexts across the world, across the globe. I think with the focus still in low and middle income countries, specifically the more under-privileged populations that I think other tools, we were not able to use them previously.

It has been successfully adapted to date in up to 25 countries. So it does work in terms of adaptations, and I’ll speak a little bit to that later. The intention of IDELA was really to monitor and track progress of a particular population. It could be at a community level, it could be at the original level within a country, it could be at the national level, but it’s really a population based tool in the sense that it’s not looking to diagnose a child or to screen a child, it doesn’t have a cut-off score, and it’s not normed in the way that the ASQ is meant to be used. So it’s really not a diagnostic or a screening tool in that way.

It also is intended to track impact of interventions, which is a key question for Save the Children, to compare effectiveness of programs which I think more and more practitioners in the field are looking to answer questions around which program is more effective in addition to the cost effective side of the issue. And really to improve policy practice and inspire more investments, both nationally as well as globally.

When we looked at kind of the constructs that were out there, we wanted to make sure that we retain a more holistic nature of the tool. So it looks at four developmental domains, motor development, emergent language and literacy, emergent math and socio-emotional development.

And it has some cross-cutting elements, things I call approaches to learning and kind of kids’ motivation and persistence to tasks, as well as a little bit of self-regulation in there. It’s hard to do all this with 24 items, and I think this is what we’re excited about, that it seems to be working so far. We are able to retain some of the domain specific kind of specificity and scores as well as have an overall understanding of the broader kind of school readiness and holistic development of kids.

It takes about 30 minutes per child, and includes 24 items across these domains. And it is meant to be somewhat fun, so tasks are really developed in an interactive way, in a sense that kids shouldn’t feel like this is necessarily a test, there are manipulatives, there are ways that kids interact with the materials and the questions, which of course is very important for this age group and making the tool actually realistic and practical in the field.

I wanted to come back to Abbie’s point about bridging the global with the local, the national level versus the global level assessment. And it’s very much the way we sort of conceptualize IDELA, thinking about these 24 core items that I kind of flashed very quickly at you in the earlier slide, and those sub-scales in the four domains, but having a lot of flexibility built into the actual instrument that could be contextualized at the national level of regional level, or even linguistic level depending on what kind of diversity you’re working with in a country.

So in addition to the core items there are optional sub-scales. Some countries are really working on health and hygiene practices in pre-school, and they could ask specific questions around that. There is an upward optional extension to a lot of the items or to the subscales themselves to avoid ceiling effects in some of the more urban, middle income, highly developed populations.

That doesn’t mean that they don’t use the items in kind of the core IDELA items, it basically is adding to those items a few extensions upwards to make sure that we can track progress as Hollie was saying earlier and make sure that we can say something meaningful about what kids are learning and how the programs are impacting them.

And then there is a lot that happens in the context of culturally specific items that countries, teams, or partners feel they want to add to the core IDELA items. And this happens across the board. I can think of maybe one country where this hasn’t happened. Across the 25 countries to date there’s always something very culturally or context specific that they feel they want to add to the items that are already in IDELA. I think that’s what makes it both flexible and at the same time comparable and provides some of these global linkages between countries and contexts where we’re able to use the assessment. A very quick flash of where IDELA has been used to date and where we’re planning to use it by the end of 2015. As you can see it’s in quite a few places, and it’s the partnership and the demand is growing around it, as Abbie mentioned, and Hiro has referred in his presentation this interest to really measure and understand what we’re gaining from engaging in early childhood services and for providing programs is really fantastic.

I can talk more in the Q&A about kind of adaptation across settings. We do follow a kind of pretty systematic process in every time we introduce the assessment in the new country or where a partner is introducing the assessment in the new country. Certainly the actual manipulatives or the kind of the tasks that we use throughout the assessment are adapted to the local cultural contexts, there’s a very strong translation process that happens as well. But we find that for the most part we’re quite successful in being able to bring a generic assessment into the particular setting and make that work across pretty diverse populations. So I think it can be done.

To the question about can it be done to adapt a tool to various settings, yes it can, and I think it can be done quite successfully. We do have obviously manuals that help standardize some of these processes and help partners guide them through the process of how to adapt a tool, but I think for the most part we’ve been quite successful at this.

So I want to move now into the question that I’m excited about, and I think it has been an area that Save the Children has put a lot of effort in, really to think about how do we define these margins and understand vulnerabilities and inequities through data, especially in our case data coming from these multiple countries at a pretty large scale.

And yesterday we talked a lot about kind of very specific groups and vulnerabilities - children with disabilities, children that are outside of the family care, migrants. The way that we tend to think about margins is really in a more broader way. What are the contextual factors or the conditions within which children live that really put them at the end of the kind of developmental trajectory because they’re not able to receive the sufficient support they need in order to develop optimally. So I’m going to talk about a few, kind of slice the data in a few different ways and talk about factors that I think this group is quite familiar with, but it’s really interesting to see them coming over and over from our data globally. So this is a slide from Ethiopia looking at skill variation, in this particular slide literacy, but this slide very much kind of represents what we see across other domains by socioeconomic status.

And what we see of course is that children in the lowest quintile from the poorest families are at quite a bit of disadvantage compared to their peers in the highest quintile of socioeconomic status. So this idea of poverty and extreme poverty that is really affecting the developmental outcomes of kids, we’re seeing this over and over and I think it’s an important story to keep bringing up to national governments as well as globally.

Here’s another factor or another margin. The location where kids live, the urban versus rural. Again without a doubt in every context where we have this kind of data and we have a population sample that has both urban and rural, we see that urban kids are at a higher advantage in terms of developmental outcomes compared to their rural counterparts.

Here’s maternal literacy, which again to this group this is no surprise that kids who have literate mothers are at an advantage in terms of developmental outcomes compared to kids with illiterate mothers. And again this is Mali but we see this in other countries as well.

Here’s gender. I will mention that gender for the most part doesn’t appear. Across the 25 countries where we’ve looked at these data there’s two countries where this is pronounced as kind of a trend in terms of inequity. Pakistan, and Afghanistan is the second country. But for the most part we actually don’t see these gaps. However in countries where this is an issue I hope you can start thinking about the policy implications of these kinds of data. These are five year olds about to enter grade one next year. These are the inequities that we see at the age of five, imagine what happens to that gap by grade two, three, and four.

And here’s one that we love exploring in Save the Children. It’s the home learning environment, and again I think it’s linked to Hollie’s slide before. The engagement of parents and the support kids get at home in terms of learning and development, this is really just a graph of the number of activities anybody does with the child over a period of a week, and you can see what a difference it makes, if a child is receiving a lot of support and a lot of engagement their score is obviously quite different from kids who have very stressed parents or parents that are not necessarily engaging in that (indiscernible). And what happens if we, I think the interesting thing about the margins is that margins can overlap, and all of these factors and contextual kind of characteristics that we’re talking about, they often converge in a lot of groups.

So I want to talk about access to ECD. I’m sure you’re not surprised to know that across all settings, this is Bhutan data, kids who are not receiving ECD services are kind of at a disadvantage in terms of developmental outcomes and school readiness. But what happens if you actually add and converge some of these factors?

What you see in the second slide is children who are not receiving ECD services who are in the low SES quintile, and who also are not receiving home support. And then you see that the gap between the two groups actually doubles. And we see this over and over in terms of converging risk factors and converging vulnerabilities. And I think when we think about margins I hope globally we can start considering how these factors actually come together to create even more powerful inequities. So I’m going to skip this and talk very quickly about program impact and policy implications, and come back to this. I’ll only show two slides on this. So here is a slide in Ethiopia again looking at the difference between children who are not receiving an ECD program, children in standard kind of ECD provision by the government, and the high quality ECD program. Going back to the point around quality, I think this kind of data is really informative both to communities and to program implementers like save the children, but also to the government, to think about what do they gain by additional investment in quality.

I think the next one I will show is the different program approaches. This is a slide from Bangladesh, and again what we can see here is the difference, you’re missing a bar in the very first column, but it’s really the difference between kids who are not receiving ECD support, kids who are receiving an intensive six week program, which I think our colleague from ACEV talked about a very similar program as an interim kind of solution to a one year program, and kids who are in a one year program. And it’s showing that actually some of these alternative approaches are a fantastic kind of opportunity of investment. So come back to me at the end and I’ll answer more questions around this. I’ll mention quickly that I think for us the multi-partner engagement at the national level has been tremendously important to be able to make that data usable and actionable. So engaging relevant ministries, local academic partners, and of course implementing partners in civil society. And I’ll just say that in terms of practice and policy and next steps we’re deeply engaged in further equity analysis globally, which I think is really important for this group and for the general global discussion on measurement, exploring effectiveness of different modes of delivery to ECD, which is really I think the conversation to access is moving. it’s very unlikely that one model is going to be relevant across all the settings, especially in terms of the cost of a more traditional preschool model, so exploring different ways to provide ECD support to kids, and perhaps more cost effective ways, but still with strong impact.

Looking to expand partnerships and also national level use of the tool, some countries are already embarking on this. And of course looking to disseminate a lot of this fantastic data that’s coming from the work of Save the Children and partners, and making the tools more easily accessible to a wider group of actors. I think I’ll stop there. Thank you.

Krishnamurthy - Guide for Monitoring Child Development

This is in the context of using the GMCD, the Guide for Monitoring Child Development, a tool that I’ve been working with since 2008. So before we get into talking about GMCD, these are the organizations that I’m working with on this tool, more about them in a bit.

So the first question was who has access to children below three years of age. It’s not intuitive in a lot of the countries that we work with, and particularly in India, that it would be pediatricians or physicians. The picture actually over here is taken right outside my office, and that child and family live in that concrete thing that you see here.

So who has access to this little boy, who’s going to monitor his child development? It turns out from a study that my colleague Ilgi Ertem who is the author of the GMCD also, where she looked at 33 different low and middle income countries and looked at who accesses young children, the answer was actually the healthcare system in some form or the other. So it may not be a physician, it may not be a nurse, but at the very least it’s a community health worker who has access to these children, primarily because of the need for immunization that these children, and acute health care.

What are the barriers then in accessing these children for the purposes of monitoring child development? The answers seem to be two. The first was as many of us pointed out yesterday care providers have very little training in early childhood development and identifying children with difficulties.

I was commenting to somebody yesterday that when I did my residency training 20 years ago I was completely clueless about early childhood development, or child development in general, and that’s why I took up this branch actually. Twenty years later, when I’m teaching residents, they’re in the same spot that I was in India. And that to me is really a tragedy, and hopefully discussions like this and fora like this will change that. The second piece was the fact that Abbie spoke about so eloquently this morning, that there aren’t any internationally standardized tools available. So then we started thinking about tools. And when you think about tools the first thought that comes to your mind is well then we need to go out and test children.

Any one of us here who is being the mortified parent of a child who didn’t do what you wanted them to do in a span of 20 minutes will bear witness to the fact that child testing can be unreliable. Children don’t demonstrate particularly functional skills. How do you eat with a spoon? How do you take off your jacket? They don’t demonstrate that in a test setting.

Also, just think about the last two factors that I’ve highlighted in this slide, which is does child testing really catalyze family participation? In a zero to three year old you’re really looking for families to be engaged and involved in child development, and child testing really doesn’t allow you to do that. You’re sitting nervously, any of you who’ve watched your children being tested, know that no matter what a star you know that your kid is, you’re nervous about the results of that test, and you’re not really thinking about how you can contribute to it.

Most importantly also it doesn’t identify risk factors in the family. it doesn’t tell you if the mother is depressed, it doesn’t tell you if there’s alcoholism in the family, it doesn’t tell you any of those things that we want to know.

Additional barriers in low and middle income countries, when you ask yes and no questions they usually respond with what they think you want to hear. The power equation is clear. Does your child speak in two word sentences? Yes, right? Also, importantly, the literacy levels are low, they can’t answer written questionnaires. And we all know about caregiver knowledge of child development in settings like these may be skewed because of the developmental delays in the children around them and because of the lack of their own knowledge. Stigmatization is high, you don’t really want to talk about your child with delays. So what the Guide for Monitoring Child Development really tries to do is these five things. Firstly, reflect the bio-ecological theory, which all of us have been talking about yesterday, the multiple factors that contribute to early childhood development. Secondly, the key thing in this is the monitoring and support component, which I’m going to talk more about.

Hollie touched upon that briefly when she spoke about screening versus monitoring. Screening is this, it’s a snapshot in time, it tells you how the child is doing at that particular moment. Monitoring is sort of a continuous video recording of how that child’s development is doing over time. And this we know is important. It identifies risk factors. It looks at functional outcomes. And I’m going to talk about the evidence base for the GMCD.

The GMCD has a history of 20 years. It was developed starting in 1995 by Ilgi Ertem in Turkey. This tool was used extensively in Turkey, the validation studies are done, published in Pediatrics in 2006, I think it’s there in our reading material. And the idea behind this tool was to provide an instrument that would enhance developmental monitoring, provide support to families, and manage children who were detected with developmental delays. But the ultimate goal of this tool is to catalyze care provider and caregiver relationships, catalyze that partnership.

So the basic principle of the GMCD is that it’s a clinical open-ended evaluation. And it’s an intervention, it’s not a checklist for screening. The key thing is the partnership, and I’ll show you what I mean. So when the GMCD begins you actually ask an open ended question.

Well you explain first what child development is. You talk about, we’ve been following your child’s community health worker, she says we’ve been talking about your child’s health, they’ve been talking about immunization, illnesses. Just as it’s important to monitor health we want to monitor how your child is learning, how she’s developing language. You know your child the best, and the first three years are really important. Would you give me a few minutes to talk about this?

The first question in the GMCD is the only one that’s a closed-ended question. It describes what child development is, and then asks if the parent has any concerns about this. The second question onwards we run through all the developmental domains of expressive language, receptive language, fine motor skills, gross motor skills, play and relationships to look at socio-emotional development and self-help skills. And the questions look like this: Tell me how your child communicates with you. Tell me how Asha communicates with you. How does she let you know when she wants something? What kind of sounds does she use? What kind of words does she use?

So you’re asking an open-ended question, and the mother gives you an answer, and I have some video clips later if anybody is interested in seeing community health workers administering this tool, where the mothers are answering in long sentences and paragraphs, they’re talking about their kids. And the tool takes just 10 to 15 minutes to complete, but this is the first time the mother has had an opportunity to talk about her child’s development and know that it’s a topic that she can talk to her care provider about. The questions go on until questions eight, nine, and ten. Eight, nine, and ten are the ones that look at psychosocial risk factors. So we begin with question eight, which says what do you do to support Asha’s development? How do you play with her? How do you get her to communicate? Usually by this time the parent has already given you examples as part of expressive language, as part of play, in terms of what she does with the child. And you could reframe this question in terms of I have heard so many wonderful things that you do to support Asha’s development.

Is there anything else you do, how you play with her, how do you get her to communicate? So these are the prompts that the health worker gives. And below the prompt is a suggestion for the health worker to support by praising every effort. If needed give suggestions, look at the card. And I will show you the card in a minute. Encourage the caregiver to try these out in your presence. Praise positive interactions. Question number nine and ten actually are related to maternal depression and other risk factors. I’m happy to share those with anyone who wants to look at them later. This is the support component, and it’s nothing but an improved version, an added on version of the WHO UNICEF CCD Card, the Care for Child Development Card, except this includes social-emotional milestone suggestions as well as fine motor.

A few words about the international GMCD. The International GMCD is what we’re just completing this year. For the last five years we’ve been working on an RO1 funded by NIH in three middle-income and one low-income country. We’ve been working with Turkey of course, but India, Argentina, and Pretoria in South Africa. It’s an ethnically diverse group of children with different cultures and ten different languages. You can only imagine the logistics of translating and back-translating all of those.

And enrollment of over 20,000 children in four countries. The first phase was a standardization, the second phase was establishment of validity, and the third phase, which we are just completing now and analyzing the data, is the implementation phase. We wanted to look at the challenges to implementation.

I do have to mention here, which I’ll mention again, that we used children without any known health risk factors. So in other words we used a prescriptive population of children. These were children who were not low birth weight, not malnourished, not anemic, and didn’t have chronic illnesses or known developmental disabilities. So these were our exclusion criteria, and these were the children we looked at. I’d love to share this data with anybody who’s interested, but suffice it to say we ended up excluding almost 50 percent of the sample, which shouldn’t come as a surprise to most of this audience.

These are the partnering universities. On the extreme left is Ankara University, on the extreme right Yale University, where the two PIs were from. And the research question we really wanted to answer is, is development in healthy children similar in different countries, and can these similarities be used to develop an open access international GMCD that does not require re-standardization.

What we really wanted from a policy perspective was to cut through the eternal cycle of re-validation again and again of every single tool in every single country to see if we could find milestones that would cut across at least four countries.

And here are some of the answers we got. This graph is similar to what Abbie had when she was showing her graph. The points actually are points that do not represent single children but the percentage of children who achieved the milestone at that age. And this particular one is an expressive language milestone, it uses index finger to point, and you can see the four countries are four different colors, and there was beautiful overlap between them. And we were fortunately looking at the zero to three age group, and we could certainly see that in the majority of the milestones we looked at actually.

Here’s play actually, this is simple imaginary play, like putting a doll to bed or feeding the baby. And you can see again the overlap between the four countries here. And here is where they diverse dramatically, not surprisingly where there were cultural considerations in how children use their adaptive skills, how they use their self-help skills. So items like wearing clothes, drinking from a cup, those were the ones that diverged majorly.

So the conclusions of the International GMCD Study was that most developmental milestones in the zero to 42 month age range were attained at similar ages by both genders in different countries by children without known health risks. We are just actually analyzing the data on our excluded sample, and the results are fascinating, and they’re certainly different. The GMCD has been, we hope, developed as a unique comprehensive method that we can offer internationally as an open access tool.

I’m commenting on practice. So I can speak personally about how we’ve used this in our early childhood development and disability programs since 2008. Not the international version because that is only now just ready, but the version that was the original Turkish version. It has been a delight to have a structured way of teaching a family centered approach to community health workers who have never had anyone use a strengths based or family based approach with them.

And if I show you the video clips you’ll see what I mean. They have really internalized what it means to use a strengths based approach and to support families in supporting their child’s development. And there is that cascade effect. The way we teach them and their supervisors is how they work with families, and that’s how in turn they work with their children. And so it’s really important to use that.

We’ve also done this training at several levels. Not just community health workers, because we had the strange situation of having community health workers identify children with delays and then have them refer the child to a pediatrician who then said I don’t really see anything wrong with this child.

So we’ve done training at the top as well with pediatricians, physicians in neighboring countries, in Bangladesh and Nepal. These are all the countries that the GMCD training has been done in. Bettina can speak more to you about what has happened in this region with partnership for the UNICEF. In summary over 25 countries is where the GMCD is currently being used.

From a policy perspective certainly in some countries in this region and particularly in Turkey it’s part of the Health Ministry’s recommendations for monitoring children. We are working on an app that looks at using this that will ease the implementation of the use of the tool. And the dissemination is really a major plan for the next few years which we begin at the International DDP Conference in Istanbul this year. Thank you.

Laxminarayan - Population Trajectories

Thank you so much and thanks to Zulfi and Kimber for inviting me. I was asked to speak on something that was more India-relevant, and of course the cost of inaction comes next. I figured I would focus on just this one thing and particular studies that are ongoing right now and try to summarize where things stand with respect to child development from the economics side.

As most of you know, there are 27 million kids born each year in India. If you look at the Granthem McGregor paper, 65 million of those 200 million kids who are disadvantaged by year five are in India. That is roughly one in three. Just for comparison, the number of kids that die under five each year in India is about 1.5 million. These are large numbers for sure.

Now, India has done quite well on child survival. It won’t reach MBGs for survival, but it certainly has made a significant amount of gain. It certainly lags on child development. This is from the Commission Investing in Health, the report that came out quite recently. This is 1997 on the left side and then this is 2032 projected on the right side. You can see a huge amount of the averted deaths are going to be in that 0 to 4 category. That is quite significant and certainly shifting the distribution.

So India’s infant mortality rate is about 47 per 1000 and about a third of the kids are low birth weight. India has high rates of stunting, about 42 percent of the under five kids in India are underweight, 48 percent are stunted and a lot of them are anemic. There are huge variations across states. States like Kerala do reasonably well on under 5 mortality, but then you have got states like Madhya Pradesh that continue to lag behind.

I want to spend a couple of minutes on this. Peter actually mentioned. In fact, I sent this to him a few years ago from the Economist. This was a very interesting report that was done by part of us. This was an idea of inclusive wealth. We measure GDP often in countries, which is income, but we don’t really stop and measure wealth. How much is the country really worth at the moment? If you want the entire inclusive wealth report that was out in 2012, it is downloadable from some website. This was an Economist picture that summarized this.

You can see that if you look at a country like Japan most of their wealth is in human capital. They have very little in natural capital, and the physical capital is being physical infrastructure, roads. The natural capital is minerals in the ground, environment and air quality to some extent. Human capital I will tell you in just a second, but it is really to do with education, attainment, productivity, wages, health, life expectancy and so forth.

Japan, obviously, is heavily concentrated on human capital. You see countries like Canada has more of the natural capital as does Norway and Australia. By the time you get down to a country like India, even if you take Saudi Arabia, the natural capital is going to outweigh the human capital and certainly the natural and physical outweighs the human.

India, for instance, is somewhere out there. India, as you can imagine, in the main report has a relatively smaller amount in human capital category. Measuring it in this way is useful. When I was listening to the discussion yesterday, the thing about moving from child wealth to child development is hugely complex. One is binary and the other one is so complicated that the idea of having constructed even consistent measures that are consistent across countries would be quite hard to do.

Now, this report views population mortality probability, employment, education attainment and employment compensation, wages, labor force participation, population and so forth. This is still not a full measure in a sense of anything that you would consider child development, but these are the indicators that you might think child development would eventually reflect upon. If you did well on child development then all of these should actually improve.

These are all in a sense measureable. They are obviously not proximal. They are distal and they are multifactorial. At least it gives a sense for what we are really aiming for. The goal at the end is really to improve human capital or the human potential. Those we really can’t measure even if we can’t necessarily measure everything that is going on in a more proximal sense.

This is from a book called Happiness by Layard, which some of you may have read. This is the distribution of self-reported happiness. This is per capita GDP. If you look at just income, you can see that happiness increases with income a fair bit and then flattens out after that. We are constructing something like this which is more health-related. I haven’t found one yet, but at least the preliminary estimates we have is that this it doesn’t flatten out. It actually does go up. Income and health are not necessarily perfectly correlated. You might not necessarily see this reverse L-shape if you did it that way.

Now going back to India, the percentage of children under three who is underweight or stunted, there have been improvements in stunting and underweight, but certainly not the pace that one would consider acceptable. This persists over the entire zero to 60 months, the first five years of life. It is a pretty persistent pattern. This is more recent data from the Paul and others from the Lancet.

If you look at a country like Vietnam for instance, they brought down their child malnutrition at an annual rate of about 3.5 percent between those 20 years. India’s rate of decline over the 1990s was a little less than percent. So other countries have done vastly better. In fact when we look at Nepal and Bangladesh which is just right next store, they have done much better on child nutrition than India has. It is clearly an area that is an important target.

What has India done in response? I will get to that in a future slide. There is a lot of money being spent on food and food security. There is now a plan to spend roughly 20 billion dollars a year on Right to Food Act, which is essentially about food security. It is essentially supplying carbohydrates, which is going to do wonders for obesity of course.

With child malnutrition, it turns out the correlation with income is not as strong as you might think. A third of kids in the richest income quintile are actually underweight. That suggests that child malnutrition is not as strongly related to food insecurity as you might think. There are probably other environmental factors at work as well. There has also not been much of a dent on hunger. The number of hungry people in India has been fairly consistent over time, even though agriculture productivity has been pretty robust over the last couple of decades.

Here is a summary picture. Crop yields have increased. This is the average crop yields of all food grains and cereals over the last 40 years. There is arguably quite a strong performance. There are both in food grains and of course cereals. Calorie intake has actually declined in India. It is hard to explain why. This has been a very consistent story for the rural areas and one of these conundrums. There is actually no discernible link between the crop yield and the calorie intake.

The first and the second pictures put this together. It is not as if we can solve this problem simply by producing more food. That is not where the solution lies. There are lots of conundrums about child under-nutrition in India. It stagnated even though cell phone rates have increased dramatically. I think there is a cell phone for every two Indians at this point.

The question is why do the poor even when given the opportunity choose to spend the money on cell phones or other luxury durables when their kids’ nutrition is really at stake? I use nutrition as a proxy for broader things that they could be doing for child development. It is hard to figure out why it is that this is really not making the progress that it should.

Maybe India doesn’t have enough economists to explain to people that there is long-term growth potential and maybe we need to clone Paul Gertler and sprinkle him across the country to make the excellent presentation that he just gave. This is perhaps something which needs to be addressed. Why is it that people don’t really appreciate that their households would gain from? Peter, you talked about Vishal Kumar people care about their kids not surviving, but thriving. They don’t seem to want to do anything to make their kids thrive even though they have the financial resources to do it. There is certainly a mismatch between what they want and what they actually end up doing.

It is not very possible that economists are wrong. Maybe our assessments of economic growth potential are completely off of the mark. Maybe under-nutrition is not as big of a deal, and maybe the people are smarter, and we are just dumb thinking that all of these huge wage gains that we get from reducing stunting, maybe they just don’t exist. Maybe people are responding rationally. That puts us in a tough spot. We want to believe that people behave rationally. We also want to believe that our numbers are right, and those two can’t be right at the same time.

So this is a huge loss of child development potential in India. Of course, this is not an Indian study. This is by Tom Vogel Journal of Development Economics. I think this is going to be said a million times at this meeting. The link between height and early earnings and years of education, I think there is fairly good consistent data from around the world which shows a positive correlation between height and cognitive scores, self-report health status, et cetera. This is just one slide that essentially summarizes this.

As Paul just mentioned, this early height is actually a good predictor of what might happen later. If you look at height for age at two years and height later on in life for adults, you can see that essentially if you are taller at that age, you have roughly three cm height advantage at adulthood. This is across countries. Height for age at two years is also a good predictor of attained schooling. So it is about a half of year greater mean change in schooling. I am going to show you some results from India, which shows roughly the same thing.

This is years of schooling. It doesn’t necessarily mean improvement in being able to read or do anything. Just the fact that you go to school for half a year or more and the school is terrible and the teacher doesn’t show up, doesn’t mean that you actually benefit from going to school. Certainly the kids do go to school more.

So height is a good proxy for this early life environment. There is fairly strong consistent relationship between this and birth weight with the education earnings and health status. I think we all agree on that by now. As someone mentioned yesterday quite appropriately, the estimates of early life health shocks may be biased because of mortality selection. We haven’t done a good job of really estimating that and see whether people who actually survive are somehow different. We are keeping kids alive more now, and that is that actually downward bias or upward bias for the folks that do survive.

Turning a little bit to the programmatic side, India does spend a lot of money on child development. The bottom line is that these pay off when implemented well. This is spending on key programs. It is about 10 billion dollars. It was direct food subsidy this year. It is going to go up to about 20 billion in the next couple of years.

Health spending overall is about six billion dollars a year. ICDS Program, Integrated Child Development Services, which I will talk about in a second is about three billion, Mid-Day Meal Scheme is 2 billion, and rural drinking water and sanitation. These don’t look like large numbers. I think Montgomery County, which is next door, has an annual budget of about 5 billion dollars, but for India these are actually pretty large numbers.

As much as we are talking about translating what we are saying about child development into our government program, someone in the Indian Bureaucracy must have had this right a long time ago. This started back in 1975. This is the idea that you had to integrate child development, and it had to include nutrition, early education, et cetera. ICDS is now almost 40 years. It will be 40 next year. That is pretty significant. It was universalized in 2005.

There are 1.2 billion Anganwadi centers, which is where every two villages have one of these centers. There are 92 million beneficiaries. There are 35.5 million preschool education beneficiaries. There is a 3.2 billion dollar budget as I mentioned. There is a center for every 800 people. Most villages actually do have a center.

Now, these are all of the things that ICDS does do. It provides a supplemental nutrition, both what the child may eat at the center after three to six years, and a take-home if they are younger and if they are pregnant and lactating mothers. Some of you may have been to these. The way they work is actually impressive. They seem to work quite well. That is where people show up for immunizations, health checkups, referrals, preschool education and health and nutrition education.

A lot of the things you might want as a child development program are all built into this and have existed there for a long time. The take-home ration is variable across states, but is usually some sort of a porridge mix, which has rice, wheat, lentils and soy all fortified and so forth. It is not to say that all of this stuff works perfectly because I don’t have to say that at all, but there have been any number of scams associated with procurement of these rations in various states. In some states it does actually work well.

So this is what these ICDS centers look like. There has been work on impact of ICDS programs that boys are five percent less likely to be underweight if there is an ICDS center. Jain did a study on them being taller than average. We have done some of this work. I see a red light. Am I supposed to pay attention to this? I thought we turned that off for the session.

So there is observation data on beneficiaries, but there is obviously selection bias, people who are in places with ICDS may have other things going on that are different from people who are in more remote areas and don’t have an ICDS center. There is also large variation in ICDS quality.

We use the NFHS data, which essentially has data on people whether an individual has use their whole life in a given place or whether they were exposed to the local ICDS center during the first three years of life. What we find, bottom line, is that having an ICDS center does have about 0.6 years of schooling. It does have a very small improvement in reading score. These are statistically significant, not for any of the others. So ICDS does seem to work when actually implemented.

To supplement that we actually looked at a different trial, which was sort of like the Guatemalan Trial, which is the Hyderabad Nutrition Trial, which was done from 1987 to 1990. Fifteen villages randomized to an intervention which was a food supplement of upma for pregnant women, mothers and children, and then looking at 14 control villages. The idea was to study the impact on birth weight of the infant. This is what the trial design looked like.

A lot of these folks were re-enrolled about 25 years later by Sanjeh Kinra who was looking at lower cardiovascular risk. What he found was that the adolescents who had been exposed to the program were about 14 mm taller and had lower cardiovascular risk. So this is now called APCAPS Study. It still continues with the Welcome Trust funding. The next stage will be looking at education outcomes and even wage outcomes for some of these folks who would have been in this study. There was remarkably little dropout.

This is the height advantage, and this was also the fat-free mass index advantage that occurred in kids who were enrolled in the trial. Here again we look at current enrollment and we see a positive effect. It is pretty small, but the treatment villages had kids that were more likely to be enrolled in programs than kids in the control village and a smaller statistical significance for attainment.

The bottom line is that ICDS and these kinds of early nutrition programs, when they work, or when they are implemented, seem to have an effect on school enrollment, but not really on attainment or test scores that we can tell. School attainment also depends on supply side factors, which is infrastructure and demand for schooling. All of these are important as well.

Just quickly we will go into Anganwadi centers of ICDS program. It turns out that there are not that many vacant positions. Most of these Anganwadi centers are actually staffed reasonably well across there. This is good news and surprisingly good news. However, most of their supervisor positions are not staffed. Essentially these folks are working around the country, and no one really knows what they are doing unless there is a periodic visit. When you have half of the supervisors not around, it is no secret that many of these positions are also auctioned and then you don’t have the program effects that you really want.

The last point I will make is that what really need is these sorts of multisectoral interventions. There is a lot of interest and work in India now on open defecation and reducing open defecation. Dean Spears has done some interesting work here which is on access to toilets and correlation with height for age. This is still early stuff, but still I don’t think anyone argues with the fact that reducing open defecation or the roughly 400 million plus Indians who don’t have access to toilets or don’t use them. That is a situation that needs to change.

This was an interesting last slide I will show. It comes from a more recent paper by Dean and other folks. Muslims actually face a lower child mortality rate in India than Hindus, even though Muslim parents are typically poorer and less educated on average. They find that there is a significant difference in sanitation and open defecation. That accounts for 18 percent of the child mortality gap between Hindus and Muslims.

When you are looking at child development you are not just looking at ICDS type programs. Even they work you are going to get about a one cm growth advantage a half year of schooling. I am not suggesting that people change religions, but really look at defecation or looking at women’s empowerment. These are broader, more difficult to deal with environmental factors and more distal factors might be the places to look for doing something here. There is probably a limit to what we can do through the direct interventions. This is looking at fraction of total area that was Muslim and the infant mortality rate. These look surprisingly good. It is still a working paper, but I thought I would put that out there.

Summary is all of the stuff I said. India lags on child development and has done well on child survival, although more needs to be done there. I think you will see more as immunization coverage goes up and so forth. Also, the neonatal mortality will start hopefully dropping there as well. High rates of stunting for nutritional status are a huge problem. There is a lot of money being spent, but there is variable quality of the program. There is certainly a lot more attention on the multisectoral interventions. ICDS performance is one aspect of it, but dealing with water sanitation and other issues might be a part of this puzzle.

I want to thank some of the folks who have supported this work including Saving Grains and the Disease Control Priorities Project, which we mentioned yesterday and our collaborators on both of these including Jere who will be talking more about this in the next session. Thank you very much.