A recent Wall Street Journal analysis of the Milwaukee voucher program concludes that the share of voucher students in a private school is the key to success—that private schools with few students using vouchers do well, while schools with many students using vouchers do poorly.

If accurate, this finding could be of value. It would suggest that perhaps there is a “magic formula” that leads to higher performance for all students in schools participating in a voucher program. It also might suggest that the benefits of school choice are limited to students attending a small subset of schools that admit few voucher students.

One of us (Wolf) was quoted in the story, though out of context. The bottom line is that this analysis is more likely to confuse the debate over vouchers than clarify it.

It’s worth noting, first and foremost, that there already has been a great deal of scholarly work completed on school choice in Milwaukee, which is the oldest voucher program in the country.

The School Choice Demonstration Project at the University of Arkansas conducted the most comprehensive longitudinal analysis of the Milwaukee Parental Choice Program (MPCP) to date. Among the findings: higher graduation and lower arrest rates for students enrolled in the MPCP, plus students who left choice schools to return to public schools subsequently performed better in their public school. With a few notable exceptions, research from other voucher programs around the country and world has found similar positive impacts on performance and attainment.

More recent research from the Wisconsin Institute for Law & Liberty (WILL) showed that low-achieving private schools are losing market share to higher achieving schools in Milwaukee. In other words, school choice is working in Milwaukee to lift up better schools at the expense of schools that are struggling.

The possibility that there is a causal relationship between voucher enrollment and academic performance cannot be evaluated using the data in the WSJ story. As the article notes, schools in the MPCP cater to low-income, minority families; the program is limited to Milwaukee families within 300 percent of the federal poverty line, most of whom are African American or Hispanic. The table below documents the strong correlations between demographic characteristics and the percentage of students in the school enrolled via a voucher: The greater the share of students in a participating private school using a voucher, the greater the share that are low-income minorities.

Unfortunately, as the author also noted, it is still the case that these demographic factors also are highly predictive of academic performance. This means that simply comparing student achievement at schools serving more and fewer voucher students is apt to be misleading. We cannot say that a lower share of choice students is the cause of better performance because concurrent with lower shares of choice students are lower shares of students who are more likely to have a difficult time academically, including for reasons that are hard to observe.

The Wall Street Journal does highlight important disparities in funding among MPCP private schools. While some schools raise funds independently, many rely almost exclusively on the vouchers. This does result in limited resources for schools with a mission to educate disadvantaged students. This may have some effect on the academic outcomes observed at these schools, but how much of a role resources play (if any) is entirely unclear.

It’s also worth noting that there are many additional ways to measure student outcomes and success beyond test scores, which most parents rank low when it comes to choosing schools for their children.

For 25 years, the Milwaukee voucher program has been subjected to study after study. As the longest running voucher program in the nation, there is ample evidence to draw conclusions about its success. We should judge Milwaukee’s program and the schools that participate in it on the totality of that evidence, not on crude correlations.

— Will Flanders and Patrick J. Wolf

Will Flanders is the Research Director at the Wisconsin Institute for Law & Liberty. Dr. Patrick J. Wolf is Professor and 21st Century Chair in School Choice in the Department of Education Reform at the University of Arkansas College of Education and Health Professions.