Boston College - centered tower banner

BC researcher advises testing students in same format as work style

(6-1-2000) Students taught to write using computers are at a disadvantage when they take paper-and-pencil tests like the Massachusetts Comprehensive Assessment System, according to a recent report by the Lynch School of Education's National Board on Educational Testing and Public Policy.

The report, based on a series of experiments conducted in five Wellesley public schools, indicated that students accustomed to writing on computers would perform four to eight points higher on the MCAS language arts test if they could write responses to all open-ended questions on a computer. The NBETPP and the Wellesley (Mass.) Public Schools prepared the report, which confirms findings from two similar studies conducted by Boston College researchers.

"MCAS has potential to help improve the quality of public education," said LSOE Senior Research Associate Michael Russell, who authored the report with Wellesley Public Schools Director of Libraries and Educational Technologies Tom Plati. "The test's current paper-and-pencil format, however, does not allow students who are accustomed to working on computers to produce their best work.

"These studies indicate that when students take written tests, they should have access to the same technology they use on a daily basis," Russell said. "If students are accustomed to writing with paper and pencil, then they should take tests using paper and pencil. But if students are used to writing with computers, then they must have access to computers during testing if we want to collect accurate samples of what students can do."

As part of the study, Wellesley students in grades four, eight and 10 were given the composition item from the 1999 MCAS language arts tests and randomly assigned to write their responses on paper or on computer. In grade four, a third group of students also composed responses on an AlphaSmart, a mini-word processor capable of displaying four lines of text. Some eighth-graders used eMates, a similar type of mini-word processor. Only paper or computers were used in the 10th grade.

Before using the MCAS scoring guidelines to score students' responses, all essays were transcribed to computer text so that raters did not know the mode in which they were written. Results showed that students who composed on computer or mini-word processors scored considerably higher than students who wrote on paper. Out of a total of 20 points, essays composed on computer scored about two points higher than essays written on paper.

Based on the 1999 MCAS results, the report states that allowing students the option to write both the MCAS composition item and the four shorter open-ended items on computer would move about 19 percent of Wellesley's fourth graders from the "needs improvement" category into the "proficient" category. In the case of one elementary school, the number of students scoring in the "proficient" category would jump from 35 percent to 60 percent. Overall, the number of Wellesley students in each grade level performing in the advanced category would double.

According to Plati, the use of computers to teach writing is now commonplace in many classrooms within Wellesley and across the state. The expected impact of MCAS scores on schools as well as students, he said, compelled researchers and educational leaders to examine how the test's paper-and-pencil format might affect the performance of students who are taught how to write using a computer.

"Research has indicated that writing quality improves significantly when technology is used as a tool," Plati said. "Our teachers feel very strongly that using computers and portable word processors is the best way to teach the writing process. When computers are used regularly as writing tools, teachers can work closely with students as they revise second, third, fourth and fifth drafts, helping students to understand that writing is a process, not a one-time event. The computer eliminates the drudgery of rewriting text by hand and allows students to focus on revising."

The Wellesley results confirm findings from Russell's two previous studies, which showed that traditional testing methods can shortchange today's computer-savvy students. The first study, conducted in 1995, reported that middle school students who used computers on a daily basis performed much better on short-answer and essay tests administered on computers rather than paper. Moreover, when the tests were administered on paper to a random sample of students, only 30 percent performed at a proficient or passing level.

The second, more comprehensive study, conducted by Russell in two Worcester middle schools in 1998, showed that students who could keyboard at least 20 words per minute performed much better on computer than on paper. Using MCAS and National Assessment of Educational Progress short answer items, the second study indicated that the difference in students' performance on computer versus on paper exceeded the amount that an average student's standardized test score changes over the course of an entire year.

News and Information From Boston College | InfoEagle Home Page | Boston College Home Page

Copyright 2000 - The Trustees of Boston College
This page provided by the Office of Public Affairs