High Stakes Testing and High School Completion
Marguerite Clarke, Walter Haney, and George Madaus
National Board on Educational Testing and Public Policy
Carolyn A. and Peter S. Lynch School of Education
Volume 1, Number 3 January 2000
All schools test their students on spelling and fractions, the content of yesterday's lecture, what they've learned this semester. Educational tests range from pop quizzes to so-called high stakes tests: those used as a basis for entrance to kindergarten, for promotion to the next grade, for graduation, to determine teachers' and schools' effectiveness, and more.
It's a bull market for high stakes testing that far surpasses the rush of the early 1970s to test minimum competency. We now call them assessments rather than tests; but the issues surrounding their uses and effects are the same. Assessments "worth teaching to" will bring about better teaching and learning; students will be better motivated; dropout rates will drop and graduation rates increase or so we are told.(note 1)
But the evidence does not necessarily support these conjectures. Occasionally, it even suggests the opposite. In this report, the National Board examines how high stakes assessments affect dropout and high school completion rates important indicators of the health of any educational system. In particular, we look at five suggestive lines of evidence on this relationship, drawn in part from studies done at Boston College or by National Board researchers.
Our conclusion is that high stakes testing programs are linked to decreased rates of high school completion. The evidence is mainly correlational. However, it is suggestive enough to warrant further research to clarify the role of high stakes testing in decisions to drop out of school. We have made suggestions for the forms this research might take.
States with High and Low Dropout Rates
Our first intriguing piece of evidence comes from the minimum competency testing (MCT) era. A study of the ten states with the highest and the ten states with the lowest 1986 dropout rates shows a strong link between attrition or dropout rates and the use of high stakes MCT programs.(note 2)
There was no MCT in half of the ten states with the lowest dropout rates. The other five had MCT programs that involved rather low stakes: four used the tests to decide about remediation; only one used them for accountability. None used them for critical decisions about graduation or grade promotion. Furthermore, in three of the latter five states, local, not state, education agencies set the standards.
The states with the highest dropout rates had MCT programs with standards set at least in part by the state. Nine of the ten used the tests in decisions about high school graduation; four used them in decisions about promotion. In sum, these ten states used minimum competency tests with higher stakes and less flexible standards than the states with the lowest dropout rates.
These data do not necessarily mean that high stakes MCT programs increase dropout rates. The states with the highest dropout rates differed in obvious ways from those with the lowest. The latter were largely western and midwestern, with proportionately fewer minority and poor students. Perhaps high dropout rates are symptoms of the educational system's failure that spurred legislation of MCT programs in the first place, or perhaps MCT does contribute in some way to the dropout problem. In any case, crude as these comparisons may be, they underline the need to explore further whether and how high stakes testing and dropping out are related.
High Stakes Tests, Low Socio-economic Status, and Dropout Rates
A second piece of evidence, again correlational, comes from a study of the relationship between MCT in eighth grade and early high school dropout patterns.(note 3) Data from the 1988 and 1990 National Educational Longitudinal Surveys were examined to determine whether students who had to pass one or more minimum competency tests in eighth grade were more likely to have dropped out of school by tenth grade than students who did not. Results show that in schools with proportionately more students of low socio-economic status that used high stakes minimum competency tests, early dropout rates between the eighth and tenth grades were 4 to 6 percentage points higher than in schools that were similar but for the high stakes test requirement.
There may be many reasons for this apparent link. For example, the higher dropout rates may be symptoms of other problems in the school such as low student motivation or lack of support programs for students in danger of dropping out due to various life pressures. However, the data still suggest that more attention should be paid to identifying the possible differential impact of high stakes testing policies on students from different socio-economic groups.
High School Graduation Testing and Dropouts in Florida
A third piece of evidence suggests a more complex relationship between high stakes testing and dropout rates.(note 4) In this study, based on records for grades 10, 11, and 12 in Florida, researchers sought to control for other factors associated with dropping out of high school, such as gender, grade point average, English language proficiency, and whether students were enrolled in dropout prevention programs. Results show that students who performed poorly on the Florida high school graduation test were more likely to leave school, but that this relationship was affected by students' grades. For students with lower grades, there was no apparent relationship between failing the graduation test and the probability of dropping out. Only for students with moderately good grades (in the range of 1.5 to 2.5 on a 4-point scale) was failure on the test associated with a significant increase in likelihood of dropping out of school. Moreover, this study found that after controlling for grades, failing on the high stakes test did not increase the likelihood of minority students' dropping out of high school any more than it did that of non-minority students.
High School Graduation Testing and High School Completion in Texas
A fourth line of evidence concerns the evolution of high stakes testing in Texas and patterns of high school completion in that state over the last twenty years. Texas has had a statewide high school graduation test since the mid 1980s first the Texas Educational Assessment of Minimum Skills (TEAMS), and then the Texas Assessment of Academic Skills (TAAS). The TAAS measures the Texas statewide curriculum in reading, writing and mathematics at various grades between 3 through 8; at grade 10 students take the exit level tests. Since 1991, a high school diploma requires satisfactory performance on the TAAS exit tests. Research findings suggest that because of this requirement some 40,000 of Texas's 1993 sophomores dropped out of school.(note 5) The dropout rates for black, Hispanic, and white students were about 25 percent, 23 percent, and 13 percent respectively. In addition, it was found that the average black and Hispanic student was three times more likely to drop out, even controlling for socio-economic status, academic track, language program participation, and school quality.
While these data concern just one cohort of Texas high school students, they fit well with Texas enrollment patterns over a 20-year period that one of us (Walt Haney) has observed in connection with research for a lawsuit challenging the graduation test in Texas (see Figure 1).(note 6)
Ratio of Texas High School Graduates to Grade 9 Enrollment 3 Years Earlier, 197879 to 199798, by Ethnic Group
Haney found that the ratio of high school graduates in a particular year to the number of ninth graders three years earlier declined only slightly between the late 1970s and 1991. For example, between 1978 and 1985-86, the ratio of high school graduates to grade 9 students three years earlier was in the range of 0.72 to 0.78 for white students and between 0.57 and 0.64 for black and Hispanic. Between 198586 (the year the TEAMS graduation test was first administered) and 1989-90 these ratios fell very slightly for all ethnic groups. In other words, high school graduation testing in Texas in 1985 did not dramatically impede the progress of students from grade 9 to high school graduation. However, as Figure 1 shows, in 1990-91 (the year the new, more difficult TAAS high school graduation test was implemented), the ratios for all three groups evidence the most precipitous drops in the entire 20 years and were about 50% greater for black and Hispanic than for white students. From full implementation of the TAAS as a requirement for high school graduation in 1992-93 until 1997-98, the ratio of high school graduates to grade 9 students three years earlier has been just at or below 0.50 for black and Hispanic students, while it has been just about 0.70 for white students.
We discuss the Texas data in more detail below, but these patterns suggest that high school graduation testing in Texas affected the rates of high school completion differently in the 1990s than in the 1980s and that this impact was more severe for minority students. The fact that the less difficult TEAMS test did not dramatically affect completion rates seems at odds with the first piece of evidence presented in this paper. That evidence suggested that even tests of minimum competency are associated with higher dropout rates when used for high stakes decision-making. However, that study did not focus specifically on the effects of such tests when used as a graduation requirement. It would be interesting to explore the effect that the difficulty level of a graduation test has on high school completion rates, both overall and for different student groups.
High Stakes Tests, Grade Retention, and Dropout Rates
A fifth line of evidence comes from research on the relationship between grade retention, being overage for grade, and dropout rates. Research on the effects of grade retention has generally concluded that, at least beyond the early elementary grades, its harms outweigh its purported benefits.(note 7) In particular, being overage for grade as a result of being held back eats away at students' sense of academic worth. The impact is especially severe for black students. Overage students are twice as likely as on-grade students to be retained in grade again.(note 8) In addition, many of them ultimately become disengaged and drop out.(note 9) In fact, being overage for grade predicts dropping out better than do below-average test scores.(note 10)
These findings recall events fifty years ago in Ireland. A primary-school leaving certificate examination was given to all sixth graders between 1943 and 1967. To reduce the potential failure rate, teachers simply did not promote weaker pupils, mainly at two points in the system grades 3 to 4 and grades 5 to 6. A pupil that was held back at one or both points was old enough to leave school before reaching the sixth grade, reducing the number of overage students that ever sat for the primary examination.(note 11)
The Irish example is worth keeping in mind when looking at grade enrollment patterns in Texas. Figure 2 shows that the ratio of Texas ninth graders to eighth graders one year earlier has been increasing steadily since the early 1980s for black and Hispanic students, while remaining relatively constant for white students. By the late 1990s, there are close to 30% more black and Hispanic students in grade 9 than in grade 8 the year before. While these progression ratios show a fairly consistent linear trend over the last two decades , they become more suggestive when combined with the information in Figure 3. Figure 3 shows grade progression ratios for grades 1 through 12 for black, white, and Hispanic students between 1996 and 1998. The dramatic upswing in the grade 9 to grade 8 ratio for black and Hispanic students and the dramatic downswing in the grade 10 to grade 9 ratio for the same groups of students suggest that many of these students are being held back in grade 9 the grade preceding the first administration of the TAAS graduation test. Given what we know about the effects of grade retention and of being overage for grade, it would be worth watching closely how such retention policies informal or otherwise affect rates of high school completion, and how they interact with graduation test requirements.
Grade 8 to 9 Progression Ratio, 197677 to 199798, by Ethnic Group
Grade Progression Ratio 199697 to 199798, by Ethnic Group
The Texas data also shed interesting light on the seemingly anomalous Florida study mentioned earlier. This was the study that suggested, contrary to most other literature, that failing that state's high school graduation test did not increase the likelihood of minority students' dropping out of high school. But recall that this study examined the records of only grade 10, 11, and 12 students. It turns out that Florida, like Texas, has unusually high rates of retention in grade 9.(note 12) It also turns out that Florida, like Texas, has one of the lowest rates of high school completion only about 80% among the states.(note 13) However, the low rates of high school completion in these states are surely not due simply to their high stakes graduation tests; states in the South have historically had lower rates of high school completion than other regions of the country.
The strands of evidence reviewed here indicate that high stakes graduation testing, together with grade retention practices that may be affected, both directly and indirectly, is associated with decreased rates of high school completion. The evidence is divided on whether there is a differential effect by race.
While many policy makers and the public have expressed general concern about whether high stakes testing will increase dropout rates, research needs to focus on actually documenting patterns of high school completion and the manner in which high stakes testing, together with other polices and practices, affects high school graduation. Schools, districts, and states also need to begin devoting as much attention to patterns of grade progression and graduation as they currently devote to reporting test scores. To give one example, we think that analysis of student enrollment by grade, race and year, of the sort undertaken in Texas, might usefully be done for other states.
Other useful research would involve tracking attrition patterns for different student groups and interviewing students who drop out in order to identify where they are leaving the system and why. Knowing where students are most at risk of dropping out can then aid in the targeting of needed resources and programs. It would also be helpful to track attrition between October and April of senior year to discover when, and also why, students leave at this stage of their education. The effect of high stakes tests on schools' policies toward dropouts should also be examined in order to identify strategies that encourage weaker students to stay in school and to do well on the test.
Finally, dropout rates for schools, districts and states need to be calculated consistently in order to facilitate comparisons and cross-state research efforts to establish whether high stakes testing causes dropout behavior. A recent National Center for Education Statistics report made a significant step in this direction by proposing methods for adjusting state dropout data to make it more comparable, also enabling the calculation of comparable high school completion rates.(note 14) We trust that further research will build on this good example and help all of us move from suggestive correlational studies towards more definitive conclusions.
1 See Madaus, G. F. (1993), A national testing system: Manna from above? An historical/technological perspective, Educational Assessment, 1(1), 9-26.
2 See Kreitzer, A. E., Madaus, G. F., & Haney, W. M. (1989), Competency testing and dropouts, in L. Weis, E. Farrar & H. G. Petrie (Eds.), Dropouts from School: Issues, Dilemmas, and Solutions, (pp. 129-152), Albany, NY: State University of New York Press.
3 See Reardon, S. F. (1996, April), Eighth Grade Minimum Competency Testing and Early High School Dropout Patterns, paper presented at the Annual Meeting of the American Educational Research Association, New York.
4 See Griffin, B., & Heidorn, M. (1996), An examination of the relationship between MCT performance and dropping out of high school, Educational Evaluation and Policy Analysis, 18, 243-251.
5 See Fassold, M. A. (1996), Adverse Racial Impact of the Texas Assessment of Academic Skills, San Antonio TX: Mexican American Legal Defense and Education Fund.
6 For more information on this case, see Haney, W. (1999), Supplementary Report on Texas Assessment of Academic Skills Exit Test (TAAS-X), unpublished report, Center for the Study of Testing, Evaluation and Educational Policy, Boston College.
7 For an overview of the literature on retention, see Shepard, L. A., & Smith, M. L. (1990), Synthesis of research on grade retention, Educational Leadership, 48(8), 84-88. See also National Research Council (1999), High Stakes: Testing for Tracking, Promotion, and Graduation, Washington, DC: National Academy Press; Shepard, L. A., & Smith, M. L. (Eds.) (1989), Flunking Grades: Research and Policies on Retention, London, England: Falmer Press; and Wheelock, A. (Ed.) (1990), Locked In/Locked Out: Tracking and Placement Practices in Boston Public Schools, Boston, MA: Massachusetts Advocacy Center.
8 See Texas Education Agency (1996), Comprehensive Biennial Report on Texas Public Schools: A Report to the 75th Texas Legislature, Austin: Texas Education Agency.
9 See Wehlage, G. G., & Rutter, R. A. (1986), Dropping out: How much do schools contribute to the problem? Teachers College Record, 87(3), 374-92; also, Alexander, K. L., Entwisle, D. R., & Danber, S. L. (1994), On the Success of Failure: A Reassessment of the Effects of Retention in the Primary Grades, New York: Cambridge University Press. Alexander and his colleagues studied 790 Baltimore youths from entrance to 1st grade in 1982. They found that nearly 66 percent of those who repeated a grade and 95 percent of the double repeaters eventually dropped out of school.
10 Texas Education Agency (1996), Comprehensive Bienniel Report on Texas Public Schools: A Report to the 75th Texas Legislature. Austin: Texas Education Agency.
11 See Madaus, G. F., & Greaney, V. (1985), The Irish experience in competency testing: Implications for American education, American Journal of Education, 93, 268-294.
12 See Table 6-1 in National Research Council (1999), High Stakes: Testing for Tracking, Promotion, and Graduation, Washington, DC: National Academy Press.
13 See National Center for Education Statistics (1997), Dropout Rates in the United States, 1996 (Report No. 98-250), Washington, DC: Author.
14 See National Center for Education Statistics (2000), A Recommended Approach to Providing High School Dropout and Completion Rates at the State Level, Report No. 2000-305, Washington, DC: Author.
About the Authors
testing in the news