So, the announcement of the round one Race to the Top finalists is upon us.  In the run-up, a pernicious parlor game in edu-policy circles has been “name the RTT finalists.”  It’s played in D.C. and Denver, New Orleans and New York… really, anywhere you get more than two edu-wonks together.  Thankfully, it’s about to come to a close.  Unfortunately, it’ll be followed by “name the RTT winners.” Sigh…

If the exercise were just tedious, I’d let it go.  But the whole game, and the mindset it reflects, is actually far more harmful than that.  It turns school reform into an Us Weekly–TMZ–In Touch echo chamber, ratcheting the pressure to sound wise by regurgitating the conventional wisdom of the moment (as if insufficient enthusiasm for hot fads was a problem in education).  Most of the predictions are based on anecdotes, enthusiasm for this or that program, vague impressions, which state chiefs someone knows and respects, and new stories.  Few who play this parlor game have deep knowledge of more than a few of the 41 applicants, and I don’t know anyone who’s read even a majority of the applications—much less done so carefully.

So, how is this problematic?  In several ways.

First, the echo chamber feeds on itself, creating a sense of legitimacy apart from the substance of the competition.  This means that credibility will rest on whether the results validate the conventional wisdom (did the expected states win?) rather than on a careful look at whether states delivered on the criteria. In other words, if Florida isn’t a finalist, many insiders will presume something was amiss—even though most of those same insiders have never read Florida’s application, much less scrutinized it alongside its competition.  This is like judging the fairness of football polls based on whether the teams thought to be the best at the start of the season wind up in the national championship game.

Second, the echo chamber is distorting reform efforts in the states.  States outside the buzz have been pressed to mimic the Kardashian states, whether or not the strategies make sense or the imitators know what they’re doing.  Meanwhile, the Kardashian sister states have been goosed to go “faster, faster” by our earnest Secretary of Education and enthusiastic would-be reformers, without a lot of attention to potential missteps.  For instance, I’m a huge critic of teacher tenure and the step-and-lane salary schedule, but I’m immensely bothered by how eager states are to lay the weight of reform on systems that will be primarily driven by value-added reading and math scores.

Third, the echo chamber conflates strong personalities with smart programs.  The presence of a terrific state chief or prominent superintendent is often taken as a proxy for a state’s “reform-mindedness,” but this kind of personality-driven judgment can leave observers blind to the problems with overhyped reforms.  Simply trying to do what this or that popular chief does is a lot like what those Hollywood studios do when they try to imitate the latest blockbuster.  “Avatar made $700 million domestically!  So let’s get a slate of 3-D science fiction flicks into production,” says the studio head.  Somehow, it never quite works as intended.

Fourth, because nobody has read the apps or really understands how to score them, the media horse-race stories have kept quoting pundits recycling the conventional wisdom.  Nobody has really read or examined the apps, but everyone knows who is expected to win—because they know who everyone else has said is going to win.  And an interesting wrinkle is that those impartial reviewers who are trying to make sense of these massively uneven phonebook compendiums are simultaneously reading the same recycled “hot” lists as the rest of us.   Hard to know how much impact it will have, but it’s not the ideal situation.

Let me be clear. I don’t think this was inevitable.  I believe it’s largely a product of the way RTT was hyped and designed. If the Department of Education had been more disciplined about establishing clear and comprehensible criteria for the apps, had opted to embrace fewer than 19 (!) criteria, had not insisted that states punch every box on the app, had enforced limits on application length and structure, had done more to make the apps transparent to and searchable by third parties, or had constructed the reviewing process with less haste and more thought, this could be more about the steak and less about the sizzle.

For most of this, it’s too late.  But, especially as we start looking forward to round two of RTT, the Department could devise a far more robust and reassuring process.   What would it take?  To start, here are three suggested steps.

One, buffer the process from ED’s political leadership.  Today, there’s no formal separation of the process from the Department’s political appointees.  While Joanne Weiss has asserted that the political appointees played no role in selecting reviewers, the formal documents both give them a major role in selecting reviewers and almost unfettered freedom to name winners and allocate dollars.

Two, make clear to observers what constitutes expertise in judging.  We know what kind of expertise is sought by IES reviewers or Olympic judges; given the newness and ambiguity of this competition, we need more (not less) clarity on this score.

Finally, in building a novel competitive grant program it is essential to be clear as to what the dimensions of the playing field are, how high the goal posts should be, or what constitutes a stellar double-axle.  What the Department is trying to do is different and harder than launching a new research agency, where the criteria for judging are framed by long traditions of rules and professional norms.  Observers don’t know how reviewers are to weigh buy-in versus boldness, ambitious promises versus realistic goals, or taut applications (that respected the ED guidelines) versus sprawling apps (that heeded ED’s warning that only materials in the apps themselves will be read).  By failing to be explicit on this score, the Department has ensured that the buzz has defaulted to the latest word on whether Rhode Island is hot and what kind of hemline Delaware is sporting this spring.

[Please see the blog entries  here and here and here and here and here and here for more discussion of the Race to the Top finalists.]