John Arnold is something of a Rosetta Stone*. I tried to interview him for Education Next in 2022, after he dissolved his K–12 team. He politely declined.
But Tyler Cowen just interviewed him on his podcast! Let’s examine excerpts from their conversation.
Tyler Cowen: There’s a common impression—both for start-ups and for philanthropy—that doing much with K–12 education or preschool just hasn’t mattered that much or hasn’t succeeded that much. Do you agree or disagree?
John Arnold: I agree. I think the ed reform movement has been, as a whole, a significant disappointment. I think there have been isolated pockets of excellence. It’s been very difficult to learn how to scale that. I think that’s largely true of many social programs or many programs that are delivered by people to people, that you can find a single site that works extraordinarily well because they have a fantastic leader, and that leader might be able to open up a few more sites. But then, when you start to scale it to 50 sites, and start to go across the nation, it all mean-reverts back to what the whole system is providing.
We’ve all seen this sentiment before. I agree with Arnold’s scale critique. But most of my friends in “reform camp” can’t stop from forging ahead with “scale.” In a weird way, reformers themselves “revert to the mean”—that is, they (we) will stipulate Arnold’s point about scale over a beer by night, but then by day they (we) are on to proposing new scale efforts with a hand-wavey attempt at quality. Arnold tired of that.
Arnold: We’re very focused on the evaluations that we fund, of trying to do long-term effects of programs and policies. Oftentimes, if you’re just looking at what the six-month effect is, you find an effect and a cost-benefit ratio that’s not relevant to what actual policymaking should be.
“Big Reform” has really refused to grapple much with the “Bad Long-Term Evaluations Problem.” For me, the Fryer-Dobbie 2016 paper on Texas charter school alumni earnings was a biggie. We weren’t really lifting kids out of poverty. And that was the whole point of ed reform.
Arnold: That said, I do think there’s a lot of value in changing the incentives of the system and having the rewards and natural growth cycles and evolutionary components that are so strong and have been beneficial in the business world also go to the K–12 world. What I mean is that the service providers that are good and are delivering good results — both for parents and for kids — should grow faster than ones that aren’t. Right now, it’s kind of the same.
I think there’s a real question about whether government can be both the regulator and the service provider of the same program, which is what essentially independent school districts are today. I think the natural role of government is to be the regulator, and that they should be regulating third-party nonprofit providers. That model is what we support and has been rolling out across the nation. I think that the evidence is clear that it’s a better system, but it’s not delivering transformational results.
Here, Arnold seems to express polite disappointment with his City Fund? “Better” but not “transformational.” But he remains a major supporter and board member. Not sure what to make of this.
Notably, Arnold does not mention Education Savings Accounts; presumably he’s not a believer.
Cowen: What’s something you’ve learned about markets or society that you didn’t believe 10 years ago?
Arnold: I think on society, it’s very hard to get people off of their baseline for the long term. There’re a lot of interventions that can happen that’ll change somebody’s short-term prospects, whether that’s a job training program or an addiction treatment program or an education program. You can bump them off that baseline for a little bit, and over time, there’s reversion to the mean.
This! This is the most undiscussed point in modern education reform. Arnold is where, say, Freddie deBoer is, too: kids aren’t going to change much. I’ve had many similar conversations with fatigued smaller donors (nine-figure wealth, not 10- and 11-).
We’re a dispirited rebel alliance of do-gooders. Because political headwinds are so strong and undeniable, few of us want to pick a second battle with this “optional” existential question. Do kids really change?
It’s not just that things won’t scale. It’s that human nature seems different from what we’d hoped. Convenings like ASU+GSV and New Schools Venture Fund and The National Charter Schools Conference stay far away from this question, as best I can tell. (I haven’t gone in a while, but I get dispatches from attendees.) It’s also easy for reformers to rhetorically dismiss these critiques, and in very polarizing language.
EdNext in your inbox
Sign up for the EdNext Weekly newsletter, and stay up to date with the Daily Digest, delivered straight to your inbox.
I think Arnold is right about how hard “kid change” is; right that the short-term wins on test scores (no matter how hard won) haven’t consistently led to long-term gains; and right that the gains from strategies like high-dosage tutoring and City Fund models haven’t held up at scale.
But! I’m cheerfully optimistic.
After being part of several reform waves—sometimes in the driver’s seat but often in the caboose for efforts that generated short-term randomized-control-trial wins but experienced long-term fade out—I’ve moved back towards pursuing artisanal change with kids. Help 20 students here, 100 students there. Kids are tough puzzles!
Typically, you need twin engines to propel change: (1) relationship-building skill and authenticity, and (2) genuine human problem solving—people who can take dozens of details that separate this kid or this situation from 100 similar ones and build just the right recipe. You need to turn both keys to launch the missile, to “reset” the kid.
Maybe it’s a failing ADHD kid who, even after medication, therapy, and a school-side 504 plan, still can’t or won’t do any homework from 6:00 p.m. to midnight and is flunking out.
Maybe it’s a KIPP alum who persisted all through high school and college, yet, even after “career coaching” from a nonprofit and the university career office, he still can’t land a decent entry-level job.
You know one good thing about the coming AI displacement of white-collar workers? The price of labor from really talented problem-solvers is about to drop. Artisanal efforts that fully embrace the tendency of kids to “revert to the mean,” while still changing their arcs, can multiply. And if we build it (the 1,000 flowers blooming), new philanthropy will come. To get there, though, we have to engage with the good-faith donor critique (mostly unspoken but always under the surface) of folks like John Arnold.
* * *
* What makes Arnold a Rosetta Stone? The other donors can’t talk openly (because dead) or won’t. John Walton, Eli Broad, and Don Fisher are no longer with us. Gates and his army have an image to protect. So it’s maybe just once a decade he’ll openly express that he feels immunization work has gone well but the K–12 work hasn’t. I’m told the Buffett alienation closed him off more, but he’s voting with his feet. Zuckerburg has an empire to protect from politicians, and those winds shift.
Mike Goldstein is working on Teen Flourishing.
