Followers of Fordham’s work know that we are obsessed with charter school quality, both nationally and in our home state of Ohio. We are also a charter school authorizer, responsible for overseeing a portfolio of eleven schools in the Buckeye State—a job we take very seriously.

So when we learned that our colleagues at Ed Trust Midwest were giving charter quality—and especially authorizer quality—a hard look in our neighboring state of Michigan, we took notice.

Its new report, Accountability for All: The Need for Real Charter School Authorizer Accountability in Michigan, is an important contribution. It rightfully focuses on authorizers as the lynchpin of charter quality; they are, after all, the entities that screen and approve new charter schools and then hold them accountable for results (or—as is sometimes the case—do not).

And the group’s ranking of Michigan’s charter school authorizers—based on the test scores of the schools they oversee—is a good conversation starter. (Among big authorizers [thirty-plus schools], four get Bs, one gets a C, and one gets a D.)

Still, I have some quibbles. First, I can’t quite tell if Ed Trust Midwest calculated schools’ growth scores appropriately. The methodology says that schools’ growth was compared to “the average student growth of the state and the local school district where most of its students live.” That doesn’t sound as precise as CREDO’s methodology, which compares each student’s growth to a “virtual twin”—a more robust way of controlling for demographic differences.

That matters because, according to CREDO’s most recent report, Michigan charter students are, by and large, outpacing their public school peers. Ed Trust Midwest might ask CREDO to revisit its 2013 Michigan-specific study and break out charters’ growth scores by authorizer. It did such an analysis in Ohio, and the results were instructive. It might show, for example, that authorizers like Central Michigan University (a C authorizer by Ed Trust’s lights) are simply approving more schools in high-poverty areas than are other entities. More accurate growth measures would be a fairer way to disentangle performance from demographics.

But that’s not the entirety of my concerns. As with any accountability system, we shouldn’t rely on test scores alone. Here too Michigan might learn from its neighbor to the south. The Ohio Department of Education has spent the better part of two years building a very robust “Quality Sponsor Practices Review”—an evaluation tool for charter school authorizers that combines quantitative data like student achievement results with inspectorate-type information as well. Now Governor John Kasich wants to use the QSPR to put Ohio’s worst charter authorizers out of business and incentivize charter schools to sign up with the stronger ones.

Charter school quality, authorizer quality, and authorizer accountability are all great topics of conversation for policymakers in Michigan. The Ed Trust Midwest report should be seen as the beginning—but not the end—of those discussions.

—Mike Petrilli

This post originally appeared on the Fordham Institute’s Flypaper blog