The Mismeasure of Learning

Who would have imagined in 1990 that only ten years later a reviewer would be asked to consider a crowd of books attacking the “established” practice of standards-based education? At the time, the idea of setting public expectations for what schools ought to accomplish rather than regulating the practices of schools and teachers seemed a goal worth fighting for, but not one that was likely to be achieved very quickly. The standards movement was part of an emerging politically centrist coalition: Republicans and Democrats espoused the idea in roughly equal numbers. “Education governors,” many from southern states in need of an achievement boost, joined forces with leading business organizations to promote higher expectations and a better-educated workforce. Civil-rights advocates were initially skeptical, but many saw the potential power of a reform movement that would not brook separate and lower expectations for poor children, immigrants, or racial minorities. Educators, by contrast, were largely absent from the early coalition for standards. Most assumed that this new idea, like so many previous fads, would soon pass, and they went about teaching in traditional ways.

But the idea did not go away. To a remarkable degree, standards-based education has become national policy. All states except one have created statewide standards. Most have developed statewide tests. Some use these tests to create “high stakes” for students (preventing them from advancing to the next grade or graduating) or for educators (taking over underperforming schools, requiring the schools to accept external assistance, or simply shaming them by identifying them as poor schools). Federal funding for education is strapped to requirements for standards and assessments. Citizen groups, business alliances, and nonprofit agencies of various kinds are organizing to support standards-based teaching and learning in the schools in their regions. Agencies that offer services to the schools, such as government agencies and various nonprofit and for-profit companies, promote their programs as ways to improve students’ performance on the tests.

When a policy change this sweeping happens so quickly (a decade is a very short time for a change of this scope, especially in America’s highly decentralized educational environment), it should come as no surprise when critics appear on multiple fronts. The execution of the standards-based educational vision has been uneven in quality, providing plenty of grist for the critics’ mills. Few states have followed through with the curriculum and professional-support programs promised by the early standards movement. In practice, raising standards sometimes looks more like punishing teachers and students than serious educational work. The quality of standards and tests is uneven; the tests are often not aligned with the standards they claim to measure. Finally, longstanding inequities in the distribution of educational resources have not been eliminated, so it is easy to argue that the standards movement is unfair to some children.

Who Should Set Standards?

One divisive issue in the standards controversy is fundamentally political. The question is, Who should make decisions that touch people’s lives? Some want this authority to remain as local as possible, vested in caring, face-to-face communities, not in a distant national or state government or even a downtown school board or mayor’s office. Others believe that representative bodies should set expectations and then “manage by results” instead of attempting to control the processes and procedures of schooling. In Will Standards Save Public Education? Deborah Meier argues that only those who actually know a particular group of students-the teachers in a school, those students’ parents, perhaps a school-level governing board or an advisory group-have the right to set expectations and standards for them. She explicitly rejects the argument that a representative group of citizens, acting from a distance, has a legitimate right to decide what students ought to learn.

Meier’s is a radical local control argument, local down to the individual school. Local control is a popular phrase in American political discourse. It is, by and large, invoked not by the liberal end of the political spectrum (which is Meier’s natural home in other matters) but by conservatives, even the radical right. A few years ago, standards were being attacked by Christian conservatives fearful of the outcomes-based-education movement, which appeared to be succeeding in making a particular set of liberal expectations about attitudes and values the official approach. As a quick way of blocking standards they didn’t like, members of the radical right appealed to the notion of local control, set against the specter of a federally controlled curriculum. At the same time, they promoted various versions of school-by-school choice for parents. Charter schools and home schooling, along with voucher programs, have developed as the political expression of this drive for choice and variety.

Meier, committed as she is to public education, does not want to opt out. She has spent her career building alternative schools entirely within large, urban public systems. In effect, however, she wants the public system to operate as if it were clusters of charters, free to set their own standards within only the broadest definitions of public requirements. Meier seems to be well aware of the odd political company she is keeping in her fight against officially imposed standards. She admits that localism will permit “wrong” (in her view) decisions to be made. But, she says,

I think the benefits of greater autonomy and more local control outweigh its dangers when it comes to schools. My long-term fundamental faith in local democracy, and a recognition that there are sufficient incentives in place-[such as the requirements of employers and universities]-that keep school standards from straying too far afield makes me rest easier than my critics.

Susan Ohanian goes beyond Meier by wholly rejecting external standards, claiming that only a child’s individual teacher can know what is right for that child. Ohanian’s book has little of Meier’s thoughtfulness and no discussion of whether the polity-however local-needs to develop shared commitments. Her title, One Size Fits Few: The Folly of Educational Standards, tells most of what the book has to say. She is concerned especially with individual children who do not fit standard categories. Claiming to speak for her fellow teachers, she portrays them as “pursued” by the boards setting standards and claims that a love of children is the primary (only?) criterion of being a good teacher. This is a breathless polemic, filled with horror stories, from accounts of meetings of state standards committees in Illinois and California to tales of children unfairly prevented from graduating, with passing accusations about the businesses and politicians who will somehow benefit from all of this standardisto activity. It is hard to discern Ohanian’s central argument-other than “Leave us alone; we know what is best for children.” Nonetheless, the book contains some ideas worth listening to, some warnings that responsible promoters of standards-based education systems need to heed. Among these is the likelihood that, in the race to meet standards, schooling will be reduced to following schedules and scripts; that time spent with literature and rich language will be driven out by vocabulary workbooks; that learning science will become merely the memorization of a string of definitions of technical terms; in short, that American anti-intellectualism (on which I have more to say) will triumph in a haze of bean counting.

The Standards Themselves

This brings us to a second great fault line in the theory and practice of standards in education: conflicting ideas about the proper substance of American schooling. Suppose we agree that centralized standard setting is appropriate in America’s representative democracy. Then we need to face the many differences of opinion concerning what schools ought to be teaching, to whom, and even how. Here we run up against another longstanding divide in American life. Visitors from other countries, from Alexis de Tocqueville onward, soon notice the apparent anti-intellectualism of American society. As a people, we don’t seem to value highly the complexities of knowledge, nuances of language, and details of interpretation. We like quick arguments and quick decisions, not extended reasoning. We are a practical people with a taste for getting things done, not talking about them. Ideas for their own sake do not tempt many of us.

This anti-intellectualism is widely reflected in our schools, even in our colleges and universities. The American school has long focused on a basic-skills curriculum, often repeated into high school. Only a minority of top students are exposed to a program that is intellectually challenging. For many, the standards movement represents an opportunity to weave into the fabric of our schools a more intellectually demanding program. We now have scientists, mathematicians, historians, and professors of literature joined with business representatives and educators in debating what students in secondary and elementary schools should be studying. America has not seen this much attention to the content of schooling in nearly a century.

These debates highlight longstanding disagreements over the meaning of “higher intellectual demand.” Traditionalists believe that there is a core body of knowledge that all students ought to learn: mathematical and scientific concepts, historical facts and interpretations, books that are part of our shared American heritage. Traditionalists call for building the curriculum around such concepts and shared texts. Progressives are less interested in the specific knowledge or the texts to be studied; they are more interested in students’ ability to use that knowledge and to find new information when necessary. Progressives often argue that there is too much information to imagine that anyone can master all of it, and they are-by and large-less concerned that everyone share a common core of knowledge.

Each side readily cites the abuses of the other. Traditionalists can point to empty, process-oriented teaching, where there is little accountability for getting the facts right as long as students are actively engaged. There is plenty of fuel for their attacks, although none of the intellectual leaders of the progressive camp condones such ways of teaching. Meanwhile, progressives can point to classrooms in which students read badly written textbooks or spend too much time memorizing isolated bits of knowledge. Again, such classrooms exist in abundance, but they are not what thoughtful promoters of traditional forms of education have in mind. In large measure, they want to recreate the excitement they remember from their advanced-placement courses and university education.

Alfie Kohn speaks for the progressive point of view, and he halfway succeeds in making a convincing case. In The Schools Our Children Deserve: Moving Beyond Traditional Classrooms and “Tougher Standards,” he argues in favor of schools in which students are intellectually engaged and encouraged to grapple with rigorous problems: schools, in other words, in which correct answers matter, but so does reaching those answers through a complex process that may involve making errors and misunderstanding concepts along the way. Images of such schools are set against descriptions of schools in which intellectual engagement is driven out by rote memorization and motivation by grades and test scores. These are dubbed traditional schools. Kohn knows the research literature, especially on motivation to learn, and generally gets the arguments right: too much working for points and grades reduces the potential pleasure of mastering knowledge and ideas. It also creates conditions in which students view challenging problems or tough courses not as opportunities to learn but as risky endeavors that may harm their chances of graduating to the achievement ladder’s next rung. Kohn also shows that many of today’s standardized tests stress disconnected facts and skills over the more intellectually demanding materials of a genuinely progressive educational diet.

However, Kohn doesn’t grapple with the important role that knowledge plays in the development of intellectual capacity. Research on learning over the past three decades makes it abundantly clear that:

• Knowledge matters. What one already knows is the foundation for new learning as well as expert performance. It pays to build a foundation of knowledge in any area of human competence that is valued. The idea that people can learn generalized skills for learning, reasoning, and gathering information and then get the facts later doesn’t survive scientific scrutiny. At the same time,

• Active processing of information is the only reliable way to acquire knowledge. Attempts to ingest knowledge by repeating lists of facts or drilling on routine procedures don’t work well, not even for the basics of multiplication tables or beginning reading, not to mention historical or scientific concepts. Acquiring information involves active intellectual work, so the idea of teaching the basics first by memorizing and drilling and later moving to “higher order” processes is equally doomed.

These twin, established facts about learning have guided my framing of the concept of knowledge-based constructivism or, in language friendlier to teachers and the general public, academic rigor in the thinking curriculum. The latter is one of nine principles of learning formulated by the Institute for Learning that I direct at the University of Pittsburgh to provide assistance to school systems in building organizational and instructional practices that will enable their students to meet higher achievement standards. As every district and school we have worked with has discovered, recognizing simultaneously the need for a solid core of knowledge and the need for active processing of information is a difficult and delicate task. But our work is providing common ground for traditionalists and progressives, because we are showing them how to hold themselves and their students accountable for both rigorous knowledge and intelligent management of that knowledge.

Tests

The heart of an accountability system lies not in the words of standards documents but in the tests and other assessments that are used to determine whether the standards have been met. In theory, tests in a standards-based system are supposed to be “aligned” with the standards; that is, they are supposed to examine the knowledge and skills that the standards specify. In practice, however, tests and standards are usually poorly matched. Evaluations of the tests and standards of several states conducted during the past two years by Achieve-an organization of state governors and business leaders working to promote high achievement-have revealed none so far in which the tests in use are well aligned with the state’s articulated expectations in math and literacy. The tests often overemphasize skills and knowledge set at lower levels than the standards underlying them. This is partly because many states rely on various incarnations of traditional American standardized tests. Some states build their own tests; some contract with testing companies; and some just adopt commercial tests. Whatever their origin, the tests tend to look alike because they have the appearance Americans expect-they are multiple choice in form, skill-oriented in content-and because they are what we know best how to create. They are also cheaper to administer than essay exams and other forms of performance assessment.

In Standardized Minds: The High Price of America’s Testing Culture and What We Can Do to Change It, Peter Sacks builds a case against such practices. He describes an accountability movement driven by standardized tests. He offers lively, personalized examples of states and school districts (and employers) using test scores to decide whom to promote, whom to graduate, and whom to hire, while often ignoring other evidence of an individual’s competence. These are familiar arguments against testing, cases of individuals who do not “test well” or who fall just below a cut-off score for qualification. But Sacks’s much more important argument concerns the fundamental invalidity of standardized tests for guiding educational decisionmaking. He shows how tests “dumb down” the curriculum by channeling teachers’ efforts and students’ time into activities that mimic the tests (for example, filling in the bubbles on practice tests). He argues, as I did a decade ago while working to launch the standards movement, that these tests help to perpetuate the “mile-wide, inch-deep” curriculum, rather than promoting teaching that pushes for solid knowledge of important topics. Sacks also challenges the supposed objectivity of standardized tests and shows how repeated administration of very similar tests produces test-score increases that may have little to do with real changes in achievement.

Nicholas Lemann, in The Big Test: The Secret History of the American Meritocracy, builds an even more challenging case against the dominance of standardized tests in education. In a compelling account, replete with biographical as well as institutional detail, Lemann traces the history of the founding of the Educational Testing Service in Princeton, New Jersey, and the eventual hegemony of its machine-scored college-entrance exams. The arguments that are now familiar-for example, that tests meant to measure inherited aptitude for higher learning are responsive to “test prep” drills-are developed here in a personalized style that makes it hard to put the book down.

The originality of Lemann’s argument lies less in what he says about the tests themselves than in his account of the vision of an engineered society that the founders and supporters of ETS (James Bryant Conant, Henry Chauncy, and Clark Kerr, to name some of the best known) had in mind. The founders’ vision of a well-oiled meritocracy, with individuals slotted into educational institutions appropriate to their abilities and streamed into jobs where they could best contribute to a growing economy, apparently fit the temper of the country as it emerged from World War II and entered the cold war. Today-more than a generation after the student upheavals, the civil-rights movement, and the establishment (and then challenge) of affirmative-action programs-it seems out of date, too planned, and insensitive to America’s diversity. We still believe in merit over inherited position, but we mistrust technocrats and institutional decisionmakers who attempt to control individual lives.

The SAT was first used to open the doors of elite institutions to certain middle-class and poor children who wouldn’t have gotten in under the system that favored the old boy network. Lemann shows how the engineered meritocracy later clashed with the country’s expanding recognition of its diversity and the desire of its most selective institutions to represent that diversity in their student bodies. An ultimate irony may well be in the making. To maintain diversity in higher education in the face of legal challenges to the use of different test cutoff scores for different racial and ethnic groups, major public university systems are experimenting with setting aside SATs in favor of high-school grades or other evidence of student accomplishment.

Reviving Standards

Lemann doesn’t like bubble tests any more than do Sacks or the other critics of standards and testing discussed here. His most important proposal entails renouncing the idea that the primary function of schools and colleges is to sort and select, asking that they focus instead on their role as institutions for educating everyone. What would matter then is not which college one attended but rather the fact that one had the opportunity for higher education and learned something important while there. Complex systems for sorting students into college would then be far less important than the quality of what was taught to everyone, both in school and in college. Tests designed to compare students’ aptitude for future learning rather than to assess what they had learned in a curriculum could be allowed to fade away, much as Chancellor Richard Atkinson has recently proposed for the University of California system.

Extended to elementary and secondary schools, such a move might rescue standards. Despite all their differences, the books reviewed here, taken as a group, make it evident that tests have “hijacked” the standards movement. In the early proposals for a standards-based system, the idea was to align all the elements of the education system-assessments, textbooks, teacher training, incentives-to publicly debated expectations of what students should be learning. In practice, however, tests, not standards, have become the centerpiece of accountability. In many parts of the country, educators spend more time analyzing tests and figuring out how to prepare students for them, often by directly teaching sample items from tests, than they do studying and understanding the standards. Where the tests are well aligned to high-quality standards and where they contain enough tasks requiring deep analysis and writing by students, matching teaching to the tests may work reasonably well, at least as a first step in reform. Where the tests are poorly aligned and consist primarily of short-answer, shallow questions-as is the case in far too many states-teaching to the test is educationally dangerous. It will lower real achievement in the name of raising scores.

For all the passion over standards, arguments between traditionalists and progressives about what a rigorous curriculum ought to look like matter less today than most people think. This is because no amount of getting standards “right” will make much difference when states and districts are calling for teachers to spend their time raising scores on tests that do not match the standards anyway. In the past decade, ample evidence has accumulated indicating that it is possible to create reliable assessments that come closer to measuring the kinds of achievement now called for in the most thoughtful state standards. These usually involve a mixture of multiple-choice and open-ended “performance” tasks. Systems for public grading of samples of students’ regular class work would carry the concept of standards-based assessment still further. Such assessments cost more than bubble tests, but they push teaching in the directions intended by the standards. Only a few states are using such assessments today. More need to-or the backlash against testing will almost inevitably kill the standards as well.

-Lauren B. Resnick is a professor of psychology and director of the Learning Research and Development Center at the University of Pittsburgh.