Evidence-based Studies

We are accustomed to spirited intellectual debate and critique, but not to the kind of misinformation in Eric Hanushek’s article (“The Confidence Men,” check the facts, Summer 2007). We’d like to set the record straight.

First, our adequacy reports were prepared for state education agencies, and legislative and gubernatorially created task forces, not plaintiffs.

Second, we have never added effect sizes to estimate the effectiveness of our model’s recommendations. We provide effect sizes to show strength of program impacts and to help establish funding priorities. We stated this in our response to Hanushek’s earlier critique of our Washington study, which is widely available, and told him so verbally. And none of the effect sizes should be linked to the National Assessment of Educational Progress (NAEP), but to the tests associated with each study, where the performance implications of a given effect size are much smaller. Nor have we ever stated that all of our recommendations must be implemented simultaneously. We routinely recommend gradual phase-in of funding for reforms.

Our model is based on existing research. While each report begins with that research, it is modified through an extensive series of meetings with state policymakers and education professionals. The result is a unique document that meets each state’s individual needs.

Hanushek implies that our recommendations will cost most states a bundle. That is not true; using national average prices and national student demographics, a new paper we have written finds that our recommendations can be funded at the national average expenditure per pupil. Hanushek overstates the cost of many of our contracts and fails to note that usually the work includes additional studies to assist policymakers in developing funding systems for high-performing schools.

In second-round studies in Washington and in Wisconsin, we found several schools that have dramatically improved student performance using approaches similar to those recommended in our evidence-based reports. Research-based practice has improved many other fields, including medicine, and offers the same promise in education. Yes, more education research is needed, but we should apply what we have learned to date to help improve our schools. The fact that half of the states that have implemented school funding systems relied on “costing out” studies based on the evidence-based method suggests the strength of our approach.

Our goal is to help each state improve learning for all children. Hanushek seems to believe nothing will work within the current system. We disagree.

Lawrence O. Picus
Allan Odden
Picus and Associates

Hanushek responds:

I join with Professors Picus and Odden in seeking effective school-finance policies. The difficulty is that education research varies widely in quality, reliability, and policy usefulness. Policymakers quite reasonably look for expert advice in judging various claims and suggested policies, but have not received it in the evidence-based studies. Picus and Odden’s own “validation studies” for Washington and Wyoming make the point. They find “several schools that have dramatically improved student performance using approaches similar to those recommended in our evidence-based reports.” But they do not show that all or even most districts using their approaches improve. Nor do they show that districts generally fail to improve without their approaches. Most important, they have no way of determining that introduction of the “similar approaches” was the cause of improvement, as opposed to being just present when other systematic or idiosyncratic factors promoted improvement in these several schools.

Medicine is an apt comparison, because that field generally applies strict scientific standards when judging evidence, as contrasted to education, where many inferior research designs are given equal standing. My critique simply shows that their “evidence” is not credible. Moreover, any method that completely ignores costs when judging policies lacks merit.

Los Angeles Update

Since these articles (“Power Struggle in Los Angeles,” forum, Summer 2007) were written, Los Angeles mayor Villaraigosa has publicly announced that he will not continue to press the courts to support his legislation. And the mayor’s slate of candidates for the Los Angeles Unified School District (LAUSD) school board has won, providing the mayor with a slim majority of allies (four of seven).

I have a unique perspective, having been both an assistant deputy mayor and the elected LAUSD president. What we are seeing is a renaissance of public education being led by the people. For example, in May the majority of the tenured teachers at Locke High School, one of LAUSD’s most troubled schools, voted to become an independent charter school. Significantly, they voted to opt out of LAUSD’s teachers’ union in favor of the more progressive Maestros Unidos, a California Teachers Association (CTA) affiliate. Across the district, parents, teachers, administrators, and students are demanding more independence from the central bureaucracy. New leaders are focusing on rigor, quality, and deepening the relationships that support excellence. They are taking responsibility.

The question is whether the mayor’s efforts will provide the depth of challenge to the entrenched system required to drive real student achievement. His comprehensive plan didn’t mention charter schools as part of the solution, although it was written by a former charter school leader and the stage was filled with charter school students at the press unveiling. The new team can choose to support this collective public action by providing facilities, cutting red tape, and making the academic outcomes transparent, but doing so will require the courage to honor the initiative of the people above the interests of the establishment.

Caprice Young
President and CEO
California Charter Schools Association

Invest in Indianapolis

David Skinner’s article (“Indianapolis Mayor Bart Peterson,” features, Summer 2007) aptly describes Indianapolis’s efforts to reform and improve public education. The legislature’s leadership and commitment for the past six years, combined with Mayor Peterson’s and Ball State University’s ability to authorize charters (and willingness to do so), along with reform-minded superintendents such as Eugene White working to improve the Indianapolis Public Schools (IPS), demonstrate to the country’s education reformers that Indianapolis is prime territory for innovation and investment.

Now it is time for the final piece to be put into place: investment. On the East and West Coasts, philanthropists and venture capitalists are investing huge sums to replicate charter schools and get charter management organizations (CMOs) and education management organizations (EMOs) up to scale. NewSchools Venture Fund, the Charter School Growth Fund, and other individual foundations are focusing a great deal of attention on large states. Rightly so. However, I would argue that it is time to focus on the nation’s 12th largest city. From this city, ripples can become waves across the Midwest.

It is worth noting that all the good that Indianapolis is producing is even more stunning because of the relatively low level of private funding committed to charter schools. The Lilly Endowment remains mum on charter schools. Many corporate headquarters have moved out of Indianapolis and Indiana altogether, and they have taken their philanthropic attention and dollars with them. Thus, in addition to the political leaders, the real leaders in Indianapolis and all of Indiana are the few private donors who have supported education reformers in developing their dream schools. Without these two pieces of the puzzle, the school innovators and the funders, the charter school law would still just be a law on the books.

With leaders all across the country increasingly recognizing the power of charter schools to transform public education, increased attention on Indianapolis by national funders, as well as an infusion of efforts from school reform groups such as Teach For America, New Leaders for New Schools, and CMOs and EMOs seeking to grow, could tip the scales in Indianapolis and begin to improve public education for all children in the city.

Kevin Teasley
President and founder
Greater Educational Opportunities Foundation

Adequacy Suits

In “Adequately Fatigued” (legal beat, Summer 2007), Joshua Dunn and Martha Derthick describe court decisions in Texas, Massachusetts, and New York that suggest the courts may be growing weary of educational adequacy cases. They significantly understate their case. Eleven other adequacy cases have also been decided in the last two years. Only in New Hampshire have plaintiffs enjoyed any significant success.

In the other ten cases, the results have been largely disappointing for plaintiffs. In Oklahoma, Indiana, Nebraska, Colorado, Oregon, and Kentucky, the courts ruled that the amount of educational funding is a political question for the legislature, not the courts, to decide. In Arizona, the courts dismissed the case, concluding that the state had no liability for achievement disparities it had not caused. A trial court upheld the adequacy of South Carolina’s K–12 education system, approving only a claim related to pre-K programs. In the latest trial in Wyoming, the court rejected the most significant of plaintiffs’ claims, granting them relatively minor relief. Most recently, an Alaska trial court ruled that plaintiffs had failed to prove inadequate school funding in that state, holding only that more state oversight over how some districts spent their money was needed. This was hardly the result plaintiff school districts could have wanted.

The track record of the last two years is a discouraging one for plaintiffs considering filing an adequacy lawsuit. Rather than the courts, it has been through the often maligned legislative process that advocates for increased funding for schools have had the most success. Most notably, legislatures in New York, North Dakota, and Wyoming significantly increased K–12 education appropriations in the last year, but only after plaintiffs’ court claims for significantly increased funding had either ended or been rejected.

Alfred A. Lindseth
Partner
Sutherland Asbill & Brennan

Pre-K for All

“Pre-K 101” (features, Summer 2007) sets up a false choice between schools and community child-care centers as the providers of pre-K. The reality is that we need both, and this debate shouldn’t be allowed to sidetrack one of the most dramatic improvements in the nation’s education system in the past decade. Six years ago, the Pew Charitable Trusts launched a national initiative aimed at helping states provide access to high-quality pre-kindergarten for every three- and four-year-old whose parents want it for them. Since then, the movement for pre-K has truly taken hold, with 29 governors recommending more than $800 million in new spending this past spring, and growing numbers of farsighted states making pre-kindergarten available to all three- and four-year-olds.

Underlying any decisions about how to structure early education has to be a discussion of what will be of greatest benefit to the children, and quality is the key. Private child-care providers can be an essential part of a comprehensive network of facilities providing high-quality pre-K. This year, Iowa, Michigan, Pennsylvania, and Texas have proposals on the table that embrace providing pre-K in diverse settings. Indeed, the majority of states that have embraced pre-K build on the success of community-based providers and give parents choices among school-based, church-based, for-profit, and nonprofit child-care centers. And there are good reasons for this: parents get choices, and states can serve far more children than schools alone can possibly accommodate.

Susan Urahn
Managing Director

Sara Watson
Senior Officer

State Policy Initiatives
The Pew Charitable Trusts

Teacher Dispositions

“Return of the Thought Police?” (research, Spring 2007) says there is a growing “approach to teacher education and certification based on ideology rather than teaching skills or mastery of content knowledge.” The author, Laurie Moses Hines, teaches cultural foundations of education at Kent State University, a National Council for Accreditation of Teacher Education (NCATE)-accredited institution. Were she to visit with faculty who are preparing accreditation program reports in their respective disciplines of mathematics, science, English, foreign language, and other areas, she would better understand the depth and breadth of content knowledge and its application for teaching that candidates must demonstrate. This is the focus of NCATE: teachers who know the content they plan to teach and how to teach it effectively so that students learn.

Regarding dispositions, NCATE  expects institutions to ensure that candidates “demonstrate dispositions that value fairness and learning by all students.” In addition to these commonsense expectations, institutions may develop other dispositions that fit their mission. NCATE refers institutions to licensing standards for professional educators adopted or adapted by most of the states. Institutions often identify dispositions that encourage pre-service educators to be caring teachers, lifelong learners, and reflective practitioners. Institutions are encouraged to measure dispositions by translating them into observable behaviors in school settings.

NCATE believes that the development of professional dispositions is an important component of pre-service education. NCATE does not expect or require institutions to inculcate candidates with any particular social or political ideology. We hope that the record has been set straight so that we all can continue the important job of preparing the next generation of highly qualified educators who can work successfully with all students.

Arthur E. Wise
President
National Council for Accreditation of Teacher Education

Educational Software

The statement by Todd Oppenheimer in “Selling Software” (features, Spring 2007) that Renaissance Learning’s Accelerated Reader has not been “held up to serious scrutiny” is simply inaccurate. There currently are several articles on Accelerated Reader that are either published in or in press with peer-reviewed journals. The peer-review process that determines whether an article will be published subjects manuscripts to the utmost scrutiny.

One of our studies was a randomized trial in a large urban district that found significant positive effects on reading achievement for students who used Accelerated Reader according to the publisher’s recommendations. This study was published more than a year ago in the peer-reviewed Journal of Education for Students Placed At Risk. Other studies have included large quasi-experiments that also reported generally positive outcomes for Accelerated Reader users, one of which is in press with another peer-reviewed publication (Research in the Schools). Beyond our studies, we are aware of several other refereed articles on Accelerated Reader dating back to the 1990s.

In addition, the author made a serious omission by not drawing distinctions between very different types of educational software. The bulk of the article was dedicated to integrated learning systems (ILS), in which the student sits at the computer and receives instruction through the technology rather than from a teacher and also may complete exercises and assessments.

Accelerated Reader is not an ILS. It’s a progress-monitoring system that encourages book reading and helps the teacher guide, monitor, and personalize student reading practice. Students read books and then use Accelerated Reader to take comprehension quizzes covering what they have read. Students have reading quantity and comprehension goals, and the software tracks their progress against those goals. The distinction between an ILS (to provide instruction) and a progress-monitoring assessment system is a critical one.

Steven M. Ross
Center for Research in Educational Policy
The University of Memphis

John A. Nunnery
Department of Educational Leadership and Counseling
Old Dominion University

In “Selling Software,” Todd Oppenheimer opines that none of the studies on Accelerated Reader have held up to serious scrutiny. However, there is consensus among three key federally funded agencies (What Works Clearinghouse, National Center on Student Progress Monitoring, and Florida Center for Reading Research), as well as several peer-reviewed journal articles that review research on education products, that Accelerated Reader has met high standards of scientific rigor with positive effects and no contrary evidence.

For 21 years Accelerated Reader has helped teachers hold students in grades K–12 accountable for their reading practice. What other educational product has withstood the test of time like that? For more than two decades, Accelerated Reader has made a consistent, reliable, and replicable contribution to the classroom.

Steven A. Schmidt
President and COO
Renaissance Learning, Inc.