Last week, Michigan’s legislature passed a bill requiring schools to hold back third-graders who fall a grade-level behind in reading. [i] If Governor Rick Snyder signs the bill, Michigan will become the 17th state to adopt such a policy. [ii]
Mandatory grade retention is clearly popular, at least among many state legislators. This is understandable given the importance of early reading skills for future outcomes and the shockingly low levels of basic reading proficiency in many communities.
But is it good policy?
The impact of mandatory retention on student academic outcomes
Critics of mandatory retention have long argued that it is harmful to children, offering little academic benefit while increasing the likelihood of eventual dropout. And, it is true, compared with peers who have progressed normally through early grades, students who repeat a grade during elementary school tend to have notably worse outcomes.
But it is also true that students who are held back tend to face greater challenges for many reasons—they are more likely to come from disadvantaged backgrounds and have lower initial academic achievement relative to other children. Even if two children appear similar in terms of all observable characteristics, it is nearly certain that the retained child differs in some important way from the child who was promoted, and it is likely that this difference will influence each child’s long-term outcomes. In the program evaluation literature, this is known as selection bias, and it makes it very hard to infer the impact of a particular program or policy from observational data.
In order to circumvent the issue of selection bias in the case of mandatory retention policies like the one proposed for Michigan, recent studies have utilized a statistical approach that compares children who fall just above and just below the cutoff used to determine retention. [iii] Given the stringency with which such policies are enforced, the difference of just a few test questions on one exam can determine if a student is retained or promoted, making the retention outcome close to random.
While these studies find that retention per se is not nearly as harmful as prior research suggests, they also do not find significant and lasting benefits. Several studies find that retention is associated with short-term improvements in standardized test scores, but these seem to fade within several years. And none of these “new generation” studies indicate any positive effects on high school completion. (To be fair, they do not find that retention reduces the likelihood of graduation, either.)
Consider the case of Florida. While it is true that the state saw impressive initial gains in fourth-grade reading scores after adopting the policy in 2002, a rigorous analysis found that by the time kids who repeated third grade reached middle school, they were no better off than their peers who just missed being retained. [iv] Moreover, Florida invested nearly $100 million per year in tutoring, summer school, and teacher professional development to support early reading at the same time it implemented mandatory retention. It is unclear whether a retention policy alone would have similarly positive effects.
Costs
Even if one concedes (as I do) that retention may have some short-term positive impacts on student achievement, it is important to examine the costs.
Mandatory retention is an extremely expensive way to help struggling readers. In other states that have tried this approach, up to 20 percent of third-graders have been held back, which means that they will end up spending an additional year in school. In Michigan, only 45.2 percent of third graders scored at or above the proficiency threshold in 2016. [v] This means that nearly 60,000 children in the state would have been subject to retention had the policy been in place last year. Given that Michigan spends over $10,000 per pupil each year, if even two-third of those students were retained, it would cost hundreds of millions of dollars. [vi]
Alternatives
If there were no other strategies to improve early reading performance, one could excuse the Michigan legislators for considering retention. But it turns out that there are a number of interventions that have been shown to help struggling readers. [vii] The randomized evaluation of the i3-funded scale up of Reading Recovery found quite large short-run impacts of the program at scale (0.3-0.4 standard deviation improvement from participation in the 12-20 week intervention). [viii] This intervention is fairly expensive at $4,400 to $7,300 per student, [ix] but other effective interventions are much cheaper. Last year, MDRC released results of a randomized evaluation of a volunteer-based tutoring program, Reading Partners, that increased reading achievement by 0.1 of a standard deviation for an out-of-pocket cost to the schools of only $320 per student. [x]
Conclusions
The Michigan bill is particularly puzzling because it actually mandates many of the alternative interventions identified as effective by practitioners and/or researchers, including regular diagnostic assessment of children in grades K-3, “evidence-based” small group or 1-on-1 supplemental instruction for students in K-3 failing to meet benchmarks, and intensive teacher professional development. Of course, there is no guarantee that districts and schools will have money to pay for such programs, particularly given the very large number of students who will qualify for them based on the standard established by the retention policy. [xi] Early reading proficiency is an important policy priority, and state legislators are right to focus attention in this area. But smart policy involves looking closely at costs as well as benefits of interventions.
—Brian A. Jacob
Brian Jacob is a Nonresident Senior Fellow at the Brookings Institution and the Walter H. Annenberg Professor of Education Policy, Professor of Economics, and Professor of Education at the University of Michigan.
This post originally appeared as part of Evidence Speaks, a weekly series of reports and notes by a standing panel of researchers under the editorship of Russ Whitehurst.
Notes:
[i] http://www.freep.com/story/news/education/2016/09/21/third-grade-reading/90800246/
[ii] http://www.thenewstribune.com/sports/article103180952.html
[iii] The following links describe analyses in Chicago, New York and Florida.
http://www.mitpressjournals.org/doi/abs/10.1162/003465304323023778#.V-Q5-fArLmE
https://www.aeaweb.org/articles?id=10.1257/app.1.3.33
http://www.sciencedirect.com/science/article/pii/S0304407607001194
http://www.nber.org/papers/w21509
[iv] http://www.nber.org/papers/w21509
[v] http://www.michigan.gov/documents/mde/Spring_2016_M-STEP_Statewide_Two-year_Bar_Charts_533305_7.pdf
[vi] If one were doing a rigorous cost-benefit analysis of retention, one would want to take into account a variety of other factors, including the extent of excess capacity (i.e., open seats) in schools currently, the additional costs (if any) of hiring new teachers and/or expanding building space, etc.
[vii] http://www.huffingtonpost.com/robert-e-slavin/twenty-four-proven-progra_b_12030772.html?
[viii] http://www.cpre.org/sites/default/files/rr_cover_snipit.png
[ix] http://cbcse.org/wordpress/wp-content/uploads/2013/05/2013-Hollands-Improving-early-literacy1.pdf
[x] http://www.mdrc.org/sites/default/files/Mobilizing_Volunteer_Tutors_FR.pdf
[xi] It is possible that the threat of grade retention will have an independent effect on student performance by mobilizing school staff and families to more effectively and efficiently utilize some of the early grade interventions described above. But this seems a tenuous rationale on which to base such a high-stakes policy.
