Contact: Patrick Wolf, University of Arkansas, (479) 575-2084
STANFORD — The reading effects of the D.C. Opportunity Scholarship Program (OSP) show the largest achievement impact of any education policy program yet evaluated in a randomized control trial by the U.S. Department of Education and reveal an important trend toward increased reading gains for students the longer they remain in the program, according to the evaluation’s principal investigator, Patrick Wolf of the University of Arkansas.
As Congress considers the future of the OSP, Wolf presents his interpretation of the significance of the evaluation’s findings to the public for the first time in the new issue of Education Next.
Although the evaluation found no impact on student math performance, the estimated reading impact of using a scholarship to attend a private school for any length of time during the three-year evaluation period was a statistically significant gain of more than 5 scale points. That estimate, Wolf noted, provides the impact on all those who ever attended a private school through the voucher program, whether for one month, three years, or any length of time in between.
The method used to evaluate the D.C. voucher program — a randomized control trial (RCT) — is considered the “gold standard” of evaluation design. With an RCT, a group of students who all qualify for a voucher program participates in a lottery. The students who win the lottery become the “treatment” group. The students who lose the lottery become the “control” group. This same design is widely used in medicine to evaluate the efficacy of medical drugs prior to making them available to the public.
“Only a voucher offer and mere chance distinguish the treatment students from their control group counterparts,” Wolf explains. “Therefore, any significant difference in student outcomes for the treatment students can be attributed to the program.”
The OSP’s reading effects should be considered the low estimate of the possible impact of attending a private school, according to Wolf, because many of the scholarship students did not remain in private school throughout the entire three-year period. Scholarship students who were still attending private schools in the third year of the evaluation gained an average of more than 7 scale score points in reading from the program. Students in the evaluation’s control group would have to remain in school nearly 4 more months on average to catch up to the reading achievement level of the scholarship students.
The 2,308 students in the OSP study make it the largest school voucher evaluation in the U.S., making the achievement results even more compelling when compared to results from other, similar experimental evaluations of education policies undertaken by the federal government. The Institute of Education Sciences, the research arm of the U.S. Department of Education, has overseen 11 randomized control trials of specific education programs or policies, including the OSP evaluation. Only three of the 11 education interventions have demonstrated statistically significant achievement impact overall in either reading or math. As Wolf notes, of those three, the D.C. voucher program has the largest achievement impact yet reported.
The OSP serves a highly disadvantaged group of D.C. students. Overall, participating students were performing well below national norms in reading and math when they applied to the program. More than 90 percent of students are African American and 9 percent are Hispanic. Their family incomes averaged less than $20,000 in the year in which they applied for the scholarship.
“These descriptive data show how means tests and other provisions to target school voucher programs to disadvantaged students serve to minimize the threat of cream-skimming,” Wolf says. “The OSP reached a population of highly disadvantaged students because it was designed by policymakers to do so.”
Wolf points out that the evaluation findings also reinforce previous research that shows that parents who have been given the chance to select their child’s school have reported much higher levels of satisfaction. In the D.C. voucher evaluation, the proportion of parents who assigned a grade of A or B to their child’s school was 11 percent higher if they were offered a voucher than if they were not, 12 percent higher if their child actually used a scholarship, and 21 percent higher if their child was attending a private school in the third year. Parents whose children used an Opportunity Scholarship also expressed greater confidence in their children’s safety in school than parents in the control group.
Established in 2004, the Opportunity Scholarship Program is a means-tested program. Initial eligibility is limited to K-12 students in the District of Columbia with family incomes at or below 185 percent of the poverty line. Congress has appropriated $14 million annually to the program, enough to support about 1,700 students at the maximum voucher amount of $7,500. The voucher covers most or all of the costs of tuition, transportation and educational fees at any of the 66 D.C. private schools that have participated in the program. By 2008, a total of 5,331 eligible students had applied for the limited number of scholarships.
“Lost Opportunities” can be read online at www.educationnext.org/lost-opportunities/.
Visit the Education Next website at www.educationnext.org.
Patrick J. Wolf is professor of education reform at the University of Arkansas and principal investigator of the D.C. Opportunity Scholarship Program Impact Evaluation.
Education Next is a scholarly journal published by the Hoover Institution that is committed to looking at hard facts about school reform. Other sponsoring institutions are the Harvard Program on Education Policy and Governance and the Thomas B. Fordham Foundation.
