Researchers and policymakers devote considerable effort to understanding gaps in academic achievement between low-income students and their better-off classmates. [1] And rightly so: the income-based achievement gap is a large and growing source of educational inequality in the United States. The test-score gap between high- and low-income students is 40 percent wider today than it was 25 years ago. [2]
One widely-used marker for poverty in schools is a student’s eligibility for free or reduced-price lunch. But while nearly half of students nationwide are eligible for subsidized meals, only a quarter of US children live in poverty. These two statistics make clear that eligibility for subsidized meals is a blunt measure of economic disadvantage. This rough measure may be perfectly appropriate for determining which children should receive school lunch subsidies, but it may be less useful for other purposes, such as measuring income gaps in achievement, determining the effectiveness of educational interventions targeted to low-income families, or steering resources toward the neediest children. Yet it is, for now, the only measure available to the many researchers and practitioners who work with administrative data to evaluate the effects of educational programs, measure gaps in student achievement, and steer resources toward the neediest children.
We use administrative data from Michigan to develop a more detailed measure of economic disadvantage. Our data contain information on the entire population of students in the Michigan public schools. We leverage the longitudinal nature of these data to document systematic variation in outcomes within the population of children who are eligible for subsidized meals. We do this by counting the number of years in which a given student qualified for subsidized meals, over multiple years of school enrollment.
In Michigan, roughly half of 8th graders are currently eligible for a subsidized meal; in math tests, they score about 0.69 standard deviations below those who are not eligible. By contrast, just 14 percent of 8th graders have been eligible for subsidized meals in every year since kindergarten. These persistently disadvantaged children score 0.94 standard deviations below those who were never eligible (and 0.23 standard deviations below those who were occasionally eligible). This gap is 40 percent larger than that measured using the conventional approach, which considers only current disadvantage.
Demographics differ starkly by these measures of economic disadvantage. In Michigan, 90 percent of those who were never disadvantaged are white, compared to 60 percent of those who were ever disadvantaged and 46 percent of the persistently disadvantaged. Students who had ever been disadvantaged by 8th grade were six times more likely to be black and four times more likely to be Hispanic, compared to those who were never disadvantaged. Students who were persistently disadvantaged by 8th grade were eight times more likely to be black and six times more likely to be Hispanic, compared to those who were never disadvantaged. The persistently disadvantaged are more concentrated in urban areas, while the transitorily disadvantaged are more concentrated in suburban areas.
The demographics available in state administrative data systems are limited. We turn to nationally-representative, survey data to shed further light on demographic differences between children who are persistently disadvantaged, transitorily disadvantaged and never disadvantaged. The Early Childhood Longitudinal Study, Kindergarten Class of 1998-1999 (ECLS-K) includes information on household income and subsidized-meal eligibility.
In the ECLS-K, about half of 8th graders in 2006-2007 were ever eligible for subsidized meals (similar to Michigan) and about 10 percent of 8th graders were eligible in each survey wave of the ECLS-K (again, similar to Michigan). [3] As in Michigan, persistently disadvantaged students in the ECLS-K are much more likely to be a racial or ethnic minority (73 percent compared to 46 percent among transitorily disadvantaged and 11 percent among the never disadvantaged). They were also much less likely to live with both parents at the start of the survey (51 percent compared to 65 percent among the transitorily disadvantaged and 91 percent among the never disadvantaged) and much less likely to have a parent with any college experience (29 percent compared to 56 percent among the transitorily disadvantaged and 85 percent among the never disadvantaged).
An indicator for eligibility for subsidized meals is often included as a control in a regression that includes other student information, such as race, ethnicity, sex, and school characteristics. For quantitative researchers, a key question is therefore whether these other observables “explain” the larger achievement deficit among persistently disadvantaged students. If other observable characteristics can explain the differences, then an analyst need only include these variables in the regression in order to eliminate biases that may otherwise be induced by unobserved heterogeneity within the population of currently disadvantaged students.
We find that other observable differences between the persistently disadvantaged and other students do not explain their larger test score deficit. When we control for race, ethnicity, and gender, as well as their interactions, the gap between the never disadvantaged and the persistently disadvantaged (0.76) is still nearly 40 percent larger than the gap based on standard measures of contemporaneous eligibility (0.55). Comparing children only within the same school (by controlling for school fixed effects) reduces gaps further, but the within-school gap between the never disadvantaged and the persistently disadvantaged remains 40 percent larger than the gap based on the standard measure of contemporaneous eligibility. [4]
In Figure 1, we plot the relationship between scores and the number of years spent in economic disadvantage and 8th grade scores. There is a negative, nearly linear relationship (this pattern holds after controlling for student demographics and school fixed effects, as described above). A natural interpretation is that this is an exposure effect, with each additional year of disadvantage further reducing scores. However, this linear relationship is nearly identical in 3rd grade, before children have been differentially exposed to five more years of economic disadvantage.
Figure 1. Each additional year of disadvantage is associated with a roughly constant increase in the achievement gap
What explains this pattern? The number of years that a child will spend eligible for subsidized meals appears to be a reasonable proxy for her current level of income. When in kindergarten, the children in ECLS-K who will be persistently eligible have an average family income of $18,000. For the transitorily eligible it is $31,000 and for the never eligible $71,000. That is, family income in a given year is negatively correlated with the number of years that a child will spend eligible for subsidized meals.
Our results imply that the number of years that a child spends eligible for subsidized meals can be used to proxy for household income. While still a crude proxy, this proposed measure captures greater variation in economic resources and educational outcomes than does the dichotomous variable currently used by researchers, which measures a child’s current eligibility for subsidized meals.
Our proposed measure can be used to estimate heterogeneous effects in program evaluations, to improve value-added calculations, and to better target resources. Two classrooms may have identical numbers of currently eligible children but different numbers of persistently eligible children. A value-added measure that does not account for these differences will be biased against teachers of the most disadvantaged children. Our measure of persistence can also be used in program evaluation, in order to estimate heterogeneity in causal effects or as a control to reduce omitted variables bias.
Our proposed measure can also be used to better target resources toward the most disadvantaged children. Many federal, state, and local programs distribute money based on the share of a school’s or district’s students eligible for subsidized meals. In Michigan, schools that have identical shares of students who are currently eligible for subsidized meals vary considerably in the share of students who are persistently eligible (Figure 2). By taking these differences into account, practitioners and policymakers can better target resources intended to support the most disadvantaged children and their schools.
Figure 2. School-level share of eighth graders currently disadvantaged versus share persistently disadvantaged
— Susan Dynarski and Katherine Michelmore
Susan Dynarski is a professor of public policy, education and economics at the University of Michigan. Katherine Michelmore is Assistant Professor, Public Administration and International Affairs at Syracuse University.
This post originally appeared as part of Evidence Speaks, a weekly series of reports and notes by a standing panel of researchers under the editorship of Russ Whitehurst.
The author(s) were not paid by any entity outside of Brookings to write this particular article and did not receive financial support from or serve in a leadership position with any entity whose political or financial interests could be affected by this article.
Notes:
1. This post summarizes a longer research paper by the authors: “The Gap Within the Gap: Using Longitudinal Data to Understand Income Differences in Educational Outcomes,” AERA Open, Vol 3, Issue 1, First published date: February-01-2017. We thank our partners at the Michigan Department of Education (MDE) and Michigan’s Center for Educational Performance and Information (CEPI) for providing the data used in these analyses. The Institute of Education Sciences, U.S. Department of Education, provided support through Grants R305E100008 and R305B110001. This research uses data structured and maintained by the Michigan Consortium for Educational Research (MCER). MCER data are modified for analysis using rules governed by MCER and are not identical to data collected and maintained by MDE and CEPI. Results, information, and opinions are the authors’ and do not reflect the views or positions of MDE or CEPI.
2. Reardon, S. F. 2011. The widening academic achievement gap between the rich and the poor: New evidence and possible explanations, in Greg J. Duncan and Richard J. Murnane (Eds.) Whither Opportunity?: Rising Inequality, Schools, and Children’s Life Chances (New York: Russell Sage Foundation).
3. The ECLS-K does not collect annual information on subsidized meal eligibility; we can observe whether a student is eligible in each of the five waves of data collection. We define the persistently disadvantaged as those who were eligible in each of the five waves. The transitorily disadvantaged were eligible in at least one wave but not all five waves.
4. Controlling for household income at the zip code level does very little to change the within-school results.
