In this series of posts against the high-regulation approach to school choice, I have demonstrated that performance accountability is not typical of government programs and that heavy regulation drives away quality supply, hurting rather than protecting the students these regulations are meant to help. If high-regulation is not the norm and does not help children, supporters of this approach might still favor it if they think it has certain political advantages.
For those interested in private school choice, two political advantages are claimed: 1) High-regulation addresses some objections, winning votes among skeptics to improve the political prospects of passing and sustaining those programs; 2) High-regulation protects private school choice programs from the political damage caused by scandals and embarrassing outcomes.
Neither of these arguments is supported by experience. Conceding regulatory measures to skeptics and opponents has hardly changed a single vote. Backers of the Milwaukee voucher program thought they would get relief from legislative opposition if they accepted more burdensome regulation. No votes have changed as a result and the program remains as precarious as ever. Nor has regulation protected programs from scandal. Judging from the steady stream of news reports about teachers in traditional public schools sleeping with students, it appears that no amount of background checks or government oversight can eliminate rare but regular instances of misconduct. I’m not arguing against a reasonable and light regulatory framework, I’m just suggesting that higher levels of regulation provide little or no additional political protection. Determined opponents can always find scandals to exploit and cannot be appeased with anything short of preserving the traditional public system.
I’m actually more worried that key backers of school choice are starting to abandon private school choice and focus all of their energies on charters. High-regulation is the norm in charter programs. You don’t have to worry about charter schools refusing to participate in a heavily regulated program since they have no alternatives. And charters seem to be flourishing. Charter programs exist in more states with more schools serving more students than do private choice programs. Many important backers of school choice seem to believe that charters are also getting better results. As Neerav Kingsland of the Arnold Foundation tweeted yesterday: “why is it the over-regulated charter sector that has had the most breakthroughs with low income students?”
Unfortunately, Neerav is mistaken. Charters are not producing better results than private school choice. High-regulation comes with a cost to quality. Let’s consider rigorous evidence on how charter and private school choice affect educational attainment. For reasons I will discuss at greater length in the next post, I think attainment is a more meaningful indicator of long-term benefits than achievement test results. I’m aware of 4 rigorous studies of the effect of charter schools on attainment. The general pattern among them is that programs producing large gains in achievement test outcomes are producing little or no increase in educational attainment.
Angrist, et al examined Boston charter schools and found significant benefits for charter students on MCAS, SAT, and AP performance. On attainment they write:
Does charter attendance also increase high school graduation rates? Perhaps surprisingly given the gains in test score graduation requirements reported in Table 4, the estimates in Table 7 suggest not. In fact, charter attendance reduces the likelihood a student graduates on time by 12.5 percentage points, a statistically significant effect. This negative estimate falls to zero when the outcome is graduation within five years of 9th-grade entry. (p. 15)
Nor are results much better for attending college: “While the estimated effect of charter attendance on college attendance is positive, it is not large enough to generate a statistically significant finding.” (p. 16) Angrist, et al do find a significant shift of students from attending 2 year to 4 year colleges, but we don’t know yet if that shift represents a positive development until we see whether they complete their degrees. Shifting students to 4 year college for which they are ill-suited and from which they drop out does them no favor.
Dobbie and Fryer examine the results of a single charter school in Harlem, the Promise Academy. Like Angrist, et al, they find large achievement test gains but little benefit for attainment. Dobbie and Fryer find a higher high school graduation rate after 4 years of the start of 9th grade, but it disappears by 6 years. (p, 18) College attendance benefits are also fleeting: “Similar to the results for high school graduation,however, control students eventually catch up and make the treatment effects on college enrollment insignificant.” Dobbie and Fryer similarly find a shift toward 4 year colleges, but again this result is ambiguous. Four year college should help students obtain more schooling but they report “The number of total semesters enrolled in college between lottery winners and lottery losers is small and statistically insignificant.” (p. 19)
Tuttle, et al’s recent evaluation of KIPP charter schools also finds large achievement test gains for charter students but little or no attainment benefit. Tuttle and her team at Mathematica make two types of comparisons to assess the progress of KIPP high school students. In one they find: “For new entrants to KIPP high schools, we also examine the probability of graduating within four years of entry. We find that this group of KIPP high schools did not significantly affect four-year graduation rates among new entrants.” (p. 36) When they examine students who continued from KIPP middle schools into KIPP high schools, they find a small but statistically significant drop in the rate at which students drop out — about 2 percentage points. (p. 39)
Booker, et al examine charter schools in Chicago and Florida and find significant benefits in educational attainment as well as higher earnings later in the workforce — at least for Florida charter students. They write: “In Florida, the charter high school students show a consistent advantage in absolute terms of 8 to 11 percentage points from high school graduation through a second year of college enrollment.” (p. 22) On later earnings they find: “Charter high school attendance is associated with an increase in maximum annual earnings for students between ages 23 and 25 of $2,347—or about 12.7 percent higher earnings than for comparable students who attended a charter middle school but matriculated to a traditional high school.”
Before the high-regulation folks get too excited about the Booker, et al results as vindication of their approach, they should note that these charter schools did not produce impressive achievement test results. Booker, et al write:
The substantial positive impacts of charter high schools on attainment and earnings are especially striking, given that charter schools in the same jurisdictions have not been shown to have large positive impacts on students’ test scores (Sass, 2006; Zimmer et al., 2012)…. Positive impacts on long-term attainment outcomes and earnings are, of course, more consequential than outcomes on test scores in school. It is possible that charter schools’ full long-term impacts on their students have been underestimated by studies that examine only test scores. More broadly, the findings suggest that the research examining the efficacy of educational programs should examine a broader array of outcomes than just student achievement. (pp. 27-8)
In the high-regulation approach, these charter schools might well be identified as the “bad” schools for failing to improve test scores, and yet they are the ones that produce long-term success for their students. In the high-regulation approach a portfolio manager or harbor master might kick these schools out of the program or restrict their growth for failing to produce achievement gains.
Let’s briefly review the results from the three rigorous examinations of the effect of private school choice on educational attainment. Unlike the charter research, they all show significant benefits for attainment. Wolf, et al examined the federally funded DC voucher program. They found little benefit for voucher students on achievement tests but those students enjoyed a 21 percentage point increase in the rate at which they graduated high school. Cowen, et al examined the public funded voucher program in Milwaukee and found a 5 to 7 percentage point increase in the rate at which voucher students attended college. And Peterson and Chingos examined a privately funded voucher program in New York City and found that African-American voucher recipients experienced a 9 percentage point increase in attending college. There was no significant benefit for Hispanic students.
If the high-regulation folks wanted to ditch private school choice to go all-in on charters, they would be making a horrible mistake. The evidence suggests private school choice is producing stronger long-term results. In addition, among charter schools, the kinds of schools that high-regulation folks like the most are the ones producing weaker long-term outcomes. Focusing only on charters making the biggest achievement score gains would miss those charters with more modest achievement results but truly impressive attainment outcomes. Charter schools offer the illusion of getting the benefits from choice without too much of the messiness of markets. As it turns out, central planning among charter schools is no easier than central planning among traditional public schools.
In addition to losing quality if key choice backers were to support charters to the exclusion of private school choice, there are obvious political advantages to backing both types of choice. Private school choice has helped make the world safe for charters by taking more of the political heat. We wouldn’t have the same expanding charter sector were it not for the credible threat of even more private school choice. And the choice movement would be wise to spread its bets across a variety of approaches to expanding school choice. No one knows the ideal political strategy or regulatory scheme, so having a variety of different approaches allows us to learn about how these different methods for expanding choice are doing. We need choice among choice.
– Jay P. Greene