Writing Under the Influence
A critique of “Asleep at the Wheel,” the Network for Public Education’s analysis of the federal Charter Schools Program
The federal Charter Schools Program has played a critical role in increasing the number of charter schools across the country. Authorized in 1994, the Charter Schools Program provides operators with short-term funding to cover school startup costs, because schools cannot access per-pupil funding until students enroll. As of 2019, a school can receive up to $1.5 million over a period of 3 to 5 years. The intent of the program, as specified by statute, is to “improve the United States education system and education opportunities for all people in the United States by supporting innovation in public education in public school settings that prepare students to compete and contribute to the global economy and a stronger Nation.”
Though the Charter Schools Program has enjoyed strong bipartisan support throughout its life, not everyone is a fan. The union-funded Network for Public Education, in a recent “analysis” called “Asleep at the Wheel,” alleges that “around $1 billion” in Charter Schools Program funds have been wasted due to charter schools’ receiving funding and then either never opening or quickly closing. Numerous outlets referred to this allegation as fact without further investigation, even though there is no basis for the claim. It also seems that the report’s release was coordinated with a Washington Post blog item, which was published right before U.S. House of Representatives Appropriations hearing on education spending with Secretary DeVos.
The report uses a hodgepodge of data sources, many not cited, in an attempt to show that some of the recipient charter schools haven’t opened or have closed during the ten years covered by publicly available data. The authors then try to calculate the amount of federal startup funding allocated to these schools, but here they rely on assumptions and sloppy, error-riddled analysis instead of data. They also address grant-management issues and recent federal audits of certain components of the program, but in both cases, they demonstrate a lack of understanding of how federal grants are dispersed and monitored.
The report illustrates how data can be misused to support a conclusion that is then advanced unwittingly by credulous media outlets.
Undermining the Charter Premise by Saying That Closure Equals Waste
While “Asleep at the Wheel” is deeply flawed on the data front, its attack on accountability—and the need to ensure that children aren’t defrauded by their school—is even more troubling. By arguing that closing charter schools amounts to wasting money, the Network for Public Education seeks to delegitimize the premise of not only charter schools but also strong accountability for academic achievement.
Charter schools should be held accountable for performance, which requires closing them when they don’t meet standards. Even with the best plans and under the ideal circumstances, opening a charter school is difficult. Charter Schools Program funding is intended to serve as seed capital to encourage innovation, and some experiments will fail. That is expected. School closure is just an indication that authorizers are doing their jobs and preventing wasted resources and the inadequate provision of services to students.
Meanwhile, numerous district-run public schools waste taxpayer dollars by providing students with a substandard education year after year. From 2010 to 2017, nearly $7 billion was spent on 1,250 schools, each of which was eligible to receive up to $2 million, in the federal School Improvement Grant program. Results were mixed at best, but none of those schools plans to return its funding, and none of the students at those schools can get back wasted years of learning. Yet union-supported advocacy groups don’t agitate for the closure of the School Improvement Grant schools, greater oversight over the program, or the elimination of recipient schools’ funding.
To suggest that school closure signifies a waste of resources gets the logic backwards. Closing failing schools prevents additional waste and allows students to find better options. Keeping failing schools open traps students in bad situations and wastes both money and opportunity.
Unclear Data and Flawed Analysis
The Charter Schools Program is one of the most complex federal education-grant programs, and analyzing it requires extensive cross-checking across data sources. Our research team at the National Alliance for Public Charter Schools has several full-time staff members working year-round to ensure that there are accurate, annual accounts of school openings, closures, and enrollment. Unfortunately, federal data is neither sufficiently timely nor accurate enough to serve as any kind of analysis’ sole data source; federal numbers have to be cross-checked with state and other data sources. As a result, there are several ways the Network for Public Education’s report fails to meet the standard of quality data analysis, suggesting that the authors were predisposed to “analyze” their data to achieve certain ends.
The main issue: There is no basis for the claim that $1 billion has been awarded to charter schools that are no longer open. The authors’ estimate is nowhere near realistic, because they don’t seem to understand either the underlying datasets or how federal programs work.
When looking at charter school data, researchers need to be mindful of the fact that it is actually very difficult to determine whether a school has closed. State and federal datasets often disagree on what constitutes a school, and records often aggregate or disaggregate school data depending on how information was reported. A single National Center for Education Statistics ID may encompass many schools or only parts of a school. These IDs can change for many reasons, such as a school expansion or an authorizer change. This is why it might look like a charter school has opened and closed on the same day, as “Asleep at the Wheel” claims some have. It’s incorrect to assume that a school is closed simply because its ID no longer shows up in a database, or even because it appears on a state closure list. Accounting for openings and closures is a tedious process, and getting it wrong can lead to inaccurate conclusions. A number of schools cited as closed in Network for Public Education’s report are, in fact, still open:
• University Yes Academy: The report states that University Yes Academy closed. In reality, school leadership decided to scale back the number of grades offered, primarily due to facility constraints. University Yes Academy now operates as a K–8 school instead of a K–12 school—but it’s still operating.
• New Paradigm Glazer Academy: The report states that New Paradigm Glazer Academy closed. In fact, the school merged with another school in the same charter network.
• Ace Charter Schools: In its analysis of California, the Network for Public Education counts two Ace Charter school closures. The schools were actually reorganized and given a new ID number—not closed.
In another example of sloppy reporting, the authors assert that the Tikun Olam Hebrew Language Charter School (which the authors mislabel as “Tikum Olam”) received program funding but never received a charter. This isn’t true. They’re right that Tikun Olam never received a charter—but the school also never received a grant. Tikun Olam received a conditional award, so when it didn’t obtain a charter, its award was rescinded.
These avoidable errors suggest that the authors didn’t spend adequate time verifying basic facts, instead preferring to make false assumptions that supported their claims. It appears the authors made no effort to determine how much money the supposedly closed schools had spent, or if those schools might have merged or changed authorizers instead of closing.
Furthermore, the $1 billion figure is based on a misunderstanding of Charter Schools Program data, not an analysis of publicly available data. The report mentions a publicly available dataset of subgrantees that contains very limited information, none of which is about school closures, and a Department of Education analysis of a non-public dataset. This Department of Education analysis (but not the dataset itself, because that data isn’t available to the public), indicates that 2,676 charter schools that were open in 2015 had received Charter Schools Program funding. It also found that 430 schools were closed as of 2015 and that there were 699 prospective schools. The Network for Public Education claims that $198 million in Charter Schools Program funding went to 430 schools that either closed or never opened between 2006 and 2015. Yet, there is no possible source for this dollar amount; the publicly available data on ed.gov does not indicate the amount of money a charter school actually received from the Charter Schools Program or whether the school is still open. The authors simply multiplied an “average” grant award by the number of closures found in the Charter Schools Program grants analysis.
The Network for Public Education authors do not explain how they get from the already unverifiable claim that $198 million was spent on closed schools to the more outrageous claim that $1 billion was spent on closed schools. The report authors can only reach that conclusion if they assume that one-third of charter schools closed by categorizing the 699 “prospective” schools (as of 2013-14) as “not open” and therefore “closed,” to give them a total of 1,129 closed schools out of 3,375 funded charter schools. The vast majority of current Charter Schools Program grantees will always be prospective schools, however, given that the purpose of the grant is to help new schools open. To describe prospective schools as “at risk” of “wasting” $322 million indicates how the report aims to twist and undermine the entire premise of the grant program.
Even if none of the 699 prospective schools ever opened, though, that would only bring the total amount to $521 million. After adjusting for an additional $700 million in Charter Schools Program funding from 2014 to present, the total would still only be $632 million—and only if the calculation were to use average grant awards, not the amount of funds the schools actually received (which would be significantly less in most cases where a school does not open). Even this adjustment to account for additional Charter Schools Program funding would not be accurate, however, since states have five years to subgrant funds; a significant portion of a state grants would not yet have been awarded.
Faulty math aside, the authors’ calculation is also based on the incorrect assumption that the Department of Education hands awardees a check for their full grant award. In many cases, the official award amount differs from the amount of money schools actually spend, as they don’t necessarily use their full grant awards. Awardees have to pay their costs upfront before getting reimbursed for allowable expenses, and for small single-site schools, this can be a challenge.
To provide another example of how the authors lack a basic understanding of how federal programs work, the report states that New Paradigm, a charter management organization, is operating at a loss. The reality is that public school districts, both traditional and charter, are typically reimbursed quarterly for federal grant outlays. The reimbursements occur after the money is spent, so while the budget reports an operating loss, the balance sheet shows a pending reimbursement. This does not mean the school is operating at a loss.
It’s also important to note that if a school closes, federally funded assets are liquidated or allocated to other schools, and unexpended funds are returned. While a school can’t “return” funds spent on training staff, other assets can be allocated to other schools in the district or sold, with the funds returned to the federal grant program. So, even when a funded school closes, at least some of the grant funding can usually be recovered.
Misunderstanding the Federal Grant-Making and Review Process
The authors accuse the Department of Education of running a “rigged” peer review process because reviewers can only review information included in an application, without independent verification, and because the department does not reveal the identity of reviewers. These requirements, however, apply to all discretionary grant programs, as indicated on page 52 of the department’s own discretionary-grant-program handbook. Nor are the requirements unique to the Department of Education; even agencies such as National Institutes of Health don’t disclose the identity of the reviewers. The presumed intent is to standardize the review process, as well as how reviewers are scoring applications.
The authors of “Asleep at the Wheel” seem to have no idea how this review process works, as they write that available information is “scant.” While federal peer-review panels are far from perfect (and one could argue that there are better ways to award large amounts of federal-program funds), information about them is hardly “scant.” The department’s handbook includes hundreds of pages of detail about how the peer-review process is designed to work, mentioning the checks and balances used to ensure fairness and including details about second-level reviews by staff. The report authors seem not to have bothered consulting this readily available resource.
Further, the authors of the Network for Public Education report seem to think that oversight ends with the peer review and grant award, even though the same monitoring process applies to all federal education grants, including the large sums of money distributed to school districts each year under Title I of the Elementary and Secondary Education Act. All grant recipients undergo very detailed, and some would argue burdensome, monitoring reviews and visits, as well as off-site monitoring. This oversight applies to all aspects of an application, regardless of whether that aspect was included in the scoring process. Grantees are also required to submit interim and final grant reports. The authors don’t reference any of these grant-monitoring protocols, however, and they don’t cite a single monitoring report as part of their “research” into oversight. Instead, they assert that “none of the thousands of charters receiving grants from CSP are ever compelled to provide an account” of how federal grant funds were spent. While the relevant monitoring reports may be difficult to find, they do exist, and it is irresponsible to criticize federal oversight without reviewing them.
Mischaracterizing Recent Inspector General Reports
In general, there are two types of U.S. Department of Education Office of the Inspector General reports: audits and investigations. Audits are part of the oversight of all federal programs, while investigations target waste, fraud, and abuse. Audit findings do identify risk, but, in and of themselves, they aren’t findings of waste or fraud.
Since 2012, there have been two Office of the Inspector General audits of the Charter Schools Program. The Network for Public Education notes that, in its 2016 National Audit of Charter and Educational Management Organizations, the Office of the Inspector General found that “22 out of 33 CMOs” in six states had conflict-of-interest findings. The report authors fail to add, however, that the Office of the Inspector General “judgmentally selected” the schools for review based on news reports about issues arising at the schools. The Office of the Inspector General didn’t pick a random sample of charter management organizations, and, therefore, in the words of the Inspector General’s own report, “results of the audit cannot be projected across the universe of charter schools with CMOs.” “Asleep at the Wheel” thus mischaracterizes the facts.
The authors also fail to note that the Department of Education developed and implemented a corrective-action plan in response to the 2016 report’s recommendations. By March 2019, almost all of the actions in that plan had been completed. The only exception was the release of a Dear Colleague Letter providing State Education Agencies with a suggested framework for monitoring, risk-assessment, and mitigation procedures related to charter schools’ contractual relationships with charter management organizations or education management organizations.
A second audit was released in 2018 after a three-year delay due to the death of the auditor. The National Audit of Charter School Closures reviewed 89 “closures,” 24 of which were not actually closures but conversions or transfers to other operators, in three states over two years (the 2011-12 and 2012-13 school years). The purpose was to determine whether the Department of Education has effective oversight over federal programs with respect to charter-school closures. The 2018 report deals very little with the actual operation of charter schools; it focuses on what happens after a charter school closes. Almost all of the findings and recommendations are directed at the Department of Education and relevant State Education Agencies, not at charter school operators. The audit found that, for certain closures, the State Education Agency did not take sufficient action to ensure that public funds were recovered or that student information was protected. It also found that the Department of Education had not provided enough guidance and monitoring to ensure that State Education Agencies had adequate school-closure procedures in place. Though more guidance would likely be helpful, the audit also noted several improvements at both the federal and state levels during the six-year period of the audit’s finalization.
While the Office of the Inspector General has not reviewed the closures of district-run schools that have received Title I funding or School Improvement Grants, it’s likely that the office would have similar findings, were it to do so. The closure of any public school requires close oversight by districts, authorizers, and State Education Agencies, and all schools, district and charter alike, need clear state and federal guidance.
Any government program, federal or otherwise, always has room for improvement, and the reauthorized Charter Schools Program includes significant new requirements and investments that went into effect in 2017. Seven percent of state grants must be spent on strengthening authorizers and oversight, for example, and the statute now details requirements for subgrant applications. These changes will enhance the quality of an already beneficial program.
There is also no question that we need better data about the Charter Schools Program—and better federal education data in general. While amateurish reports like “Asleep at the Wheel” don’t shed any light on the efficacy of federal funding, even more-seasoned researchers have difficulty producing timely and useful reports with the federal education data available to them. Improving the quality of education data is a constant battle, and one well worth fighting.
“Asleep at the Wheel” instead uses flawed data and analysis to draw erroneous conclusions about charter-school closures and “wasted” funding. Since the authorization of the Charter Schools Program in 1994, $316 billion has been spent on Title I, which provides aid for disadvantaged students. Over the past 25 years, the cost of the Charter Schools Program has been a little more than 1 percent of the cost of Title I. Yet, investments in the program have changed the lives of countless students and led to innovations taken for granted in education today, such as extended school days and blended learning. Educational innovators have leveraged this funding to transform the education landscape: 3.2 million students are in public schools of choice, and students of color, particularly in urban areas, are seeing tremendous learning gains. Not every school will succeed, and with strong authorizing, charter schools that fail to serve kids will continue to close. That is a good thing for students—and for anyone who cares about quality and accountability in schools.
Christy Wolfe is the senior policy adviser for the National Alliance for Public Charter Schools. Nathan Barrett, senior director for research and evaluation at the National Alliance for Public Charter Schools, contributed to this analysis.