From Gatekeeper To Gateway: Transforming Testing in America

Executive Summary

America must revamp the way it develops and utilizes human talent, and to do that educational and employment testing must be restructured.

America can no longer rely on an abundant, largely unskilled labor supply. Instead, the nation is facing a shrinking entry-level workforce increasingly composed of linguistic, racial, and ethnic minorities, whose talents are often underdeveloped and under- utilized.

Yet in a global economy that is becoming more competitive and interdependent, we need more than ever the talents of all our people. Developing that resource is the province of our educational institutions.

From the outset, American education has had the dual goal of creating a skilled workforce and a knowledgeable citizenry. This report deals with the role of testing in pursuit of those goals.

We recognize that in the past testing has helped to reduce unfairness in allocating opportunities and directing resources to the economically disadvantaged, and has been useful for making decisions. However, the growing overreliance on testing over the past several decades deprives the nation of all the talent it needs and sometimes conflicts with the nation's ideals of fairness and equal opportunity.

This report summarizes our findings on the problems of testing and offers recommendations for its restructuring.

Current testing, predominantly multiple choice in format, is overrelied upon, lacks adequate public accountability, sometimes leads to unfairness in the allocation of opportunities, and too often undermines vital social policies. Tests may mislead as indicators of performance.

Test scores are at best an estimate of someone's knowledge or ability, and can be affected by numerous outside factors. Inevitably, some who could perform successfully will "fail" tests and thus risk being misclassified and erroneously denied opportunity.

Testing can result in unfairness. All tests are to some extent culturally dependent; nor has society yet been able to extend educational opportunities to all -- hence the score gap between minority and majority groups. Differences in performance on other indicators such as grades and ratings are generally much smaller than test score differences. Thus when test results alone are used in selection, misclassification falls disproportionately on minority groups.

There is too much educational testing. Mandatory testing consumes some 20 million school days and the equivalent of $700 to $900 million in direct and indirect expenditures annually -- an enormous cost and use of classroom time that could be spent on skill development.

Testing practices can undermine social policies. We cannot test our way out of our educational problems; the opposite is true. As teaching turns into test preparation, test results cease to reflect what examinees really know or can do. Thus our fixation on test results deflects attention from fundamental educational problems and so hinders reform.

Tests are subject to insufficient public accountability. Rarely are many important tests and test uses adequately scrutinized; standards for their development and use lack adequate enforcement mechanism; and truth-in-testing laws exist in only two states. Thus the industry whose products regulate access to opportunities is itself unregulated and unaccountable.

To help promote greater development of the talents of all our people, alternative forms of assessment must be developed and more critically judged and used, so that testing and assessment open gates of opportunity rather than close them off.

This Commission proposes that testing policy and practice be restructured to help people develop their talents and become productive citizens, and to help institutions become more productive, accountable, and just. To that end, we offer eight recommendations.

1. Testing policies and practices must be reoriented to promote the development of all human talent. We must reevaluate how we judge the quality of tests, the names we give them, the ways we report results, and the ways we use them. No testing program should be tolerated that classifies people as unable to learn; potentially negative classification in school or the workplace should be accompanied by learning opportunities.
   
   
2. Testing programs should be redirected from overreliance on multiple-choice tests toward alternative forms of assessment. Important decisions about people and institutions should, where feasible, be based on multiple sources of information, especially direct evidence of actual performance in school and on the job. Thus candidates should supply answers, perform acts, demonstrate skills, create products, and supply portfolios. Previous accomplishments should also be considered.
   
   
3. Test scores should be used only when they differentiate on the basis of characteristics relevant to the opportunities being allocated. For tests to be fair and useful, this differentiation must relate directly to the classifications and decisions to be made. With that aim, evidence should be accumulated to show how well test scores reflect real-life educational or job performance.
   
   
4. The more test scores disproportionately deny opportunities to minorities, the greater the need to show that the tests measure characteristics relevant to the opportunities being allocated. It is essential to evaluate critically the fairness and accuracy of all test-based classifications in terms of the opportunities being allocated, with full awareness of the implications for social groups already disadvantaged. Ensuring equality of educational and employment opportunities is so vital that immediate, but transitional, strategies should be adopted until appropriate forms of assessment can be developed.
   
   
5. Test scores are imperfect measures and should not be used alone to make important decisions about individuals, groups, or institutions; in the allocation of opportunities, individuals' past performance and relevant experience must be considered. Test scores should not be used by themselves to determine kindergarten entry, grade promotion, graduation, or employment opportunities. Furthermore, decision makers' judgments should enter directly into important decisions about people.
   
   
6. More efficient and effective assessment strategies are needed to hold institutions accountable. Assessment of the effectiveness of institutions -- e.g., schools and training programs -- should differ from assessment of individuals in order to help them. Large school districts in particular could use sampling techniques to gauge school performance. This would help prevent the distortions caused by using one testing program for both instructional and accountability purposes.
   
   
7. The enterprise of testing must be subjected to greater public accountability. Test quality and use should be subject to some form of independent public scrutiny. Tests would be more accurately labeled, the results constructively reported, and evidence as to what they do and do not measure made more accessible. Scrutiny must include the perspective of groups that have been most adversely affected by testing.
   
   
8. Research and development programs must be expanded to create assessments that promote the development of the talents of all our peoples. Beyond more accurate assessment, we need ways to communicate the uncertainty of all assessment results. In addition, we need to learn how to use multiple sources of information intelligently and sensitively in making decisions. Finally, we need forms of assessment that will prevent unfair classifications.

In conclusion, the Commission recognizes that testing is useful and inevitable: we must know how our institutions are doing, what our children are learning, and who will make the most of opportunities not available to all.

But to direct testing along a more constructive course, we would draw richer, more direct evidence of knowledge and skill from information sources beyond multiple-choice tests.

The design of assessments would differ with their purpose -- e.g., to inform instruction or to evaluate school performance. And test use would be monitored continuously.

The role of assessment information should be a supportive one; low test scores should never brand anyone as a failure or permanently restrict opportunities.

The shift we envision will be difficult to accomplish. New attitudes and policies cannot guarantee human development. But with resources and national resolve, we can bring testing policies and practices into line with our most important goals and most deeply held convictions.

Excerpted with Permission from the National Commission on Testing and Public Policy.