In fall 2002, schools entered the No Child Left Behind era. The previous January, President George W. Bush had signed the Act into law. While it’s tough to remember amid all our polarization and after NCLB’s travails, Congress adopted the law with massive, bipartisan majorities. But that political success gave rise to a more complicated reality.

The final bill ran hundreds of pages and included new provisions on testing, reporting, school improvement, teacher quality, and more. The heart of the bill, though, was the requirement that states test all students regularly in reading, math, and science; annually report student proficiency by school and district; and adopt federally mandated interventions where proficiency (for any demographic group) fell short of state-designated performance targets.

Oh, and states had to adopt an education plan that would ensure 100 percent of students were “proficient” by 2014. (Though states first had to define what “proficiency” meant.) As Checker Finn and I observed many years ago when explaining NCLB’s stumbles, “While nobody doubts that the number of ‘proficient’ students in America can and should increase dramatically from today’s woeful level, no educator believes that universal proficiency in 2014 is attainable. Only politicians promise such things.”

Ultimately, NCLB attempted to do three things at once, and these three proved to be an awkward fit. One was to provide an overdue X-ray showing how schools and groups of students were doing in reading and math. A second was to provide a rudimentary accountability system that could help spur change in schools or systems where inertia ruled. A third was to set a “moonshot” target for school improvement.

Individually, each of these approaches had merit. The way the law linked them together, though, was unfortunate. An X-ray can provide invaluable transparency. But when the X-ray results were used to decide whether schools were failing, they were corrupted—and turned into a metric to be gamed. Performance-based accountability can be a very healthy thing, but tying it to an arbitrary (and unserious) moonshot target promoted cynicism rather than commitment.

What happened? Well, NCLB kinda, sorta worked as it was designed—and that was the problem. States and schools did the things they were supposed to. They adopted standards and set proficiency targets. They focused more intently on math and reading. Test scores increased in the first half-decade after the law was passed, and schools paid more attention to kids who had previously been overlooked and left behind.

Proponents had hoped that NCLB would raise both the “ceiling” as well as the “floor”—that schools would embrace excellence even as they did more to promote basic mastery of reading and math. That proved to be a hollow hope: The accountability incentives led state policymakers to adopt lax standards and school leaders to seek ways to game the test results. While short-run test scores went up, so did time spent on testing and test prep.

Meanwhile, the time devoted to non-tested subjects like social studies, civics, the arts, recess, and gifted education went down. Indeed, even as scores were going up, it wasn’t clear whether students were really learning more—or just learning more of the stuff on the reading and math tests and less of everything else.

Back when NCLB was crafted, there was broad agreement on the need for more transparency into how schools were doing and that schools should be more accountable for every student. That consensus splintered over time, as a fixation on test prep, benchmark testing, and test-based teacher evaluation (along with cheating scandals and game-playing about “proficiency”) fueled the impression that the tail was wagging the dog. And then, within a decade, most of the nation’s schools were judged to be failing under the law. The problem was that parents at many of these schools didn’t believe it. They decided the problem was with NCLB, not their schools.

Today, after years of populist backlash against testing and “big data,” support for testing and transparency is still far more fragile than it once was. That’s a problem, especially as we seek to understand and address the disruptions of pandemic. And it’s a reminder that the legacy of a policy, good or bad, can long outlive the political moment that shapes it.

There’s a broader lesson here. Policy is a powerful thing. When federal officials press schools and educators to do something, there’s a good chance they’ll do it—but perhaps not as intended. Laws may “work,” the trick is to ensure success is more than a kinda, sorta proposition.

Frederick Hess is director of education policy studies at the American Enterprise Institute and an executive editor of Education Next.

This post originally appeared on Rick Hess Straight Up.