Computer or No. 2 Pencil? LSOE Study Says the Difference Can Affect Test Scores

Computer or No. 2 Pencil? LSOE Study Says the Difference Can Affect Test Scores

By Patricia Delaney
Director of Media Relations

The newly begun new school year is likely to see the controversial Massachusetts Comprehensive Assessment System, or MCAS, in the spotlight once again - and a recent Lynch School of Education study on test taking may add a new dimension to the debate.


Michael Russell

A report released this summer by the LSOE's National Board on Educational Testing and Public Policy claims students taught to write using computers are at a disadvantage when they take paper-and-pencil tests like the MCAS.

The study, based on experiments conducted in five Wellesley (Mass.) public schools, indicated that students accustomed to writing on computers would perform four to eight points higher on the MCAS language arts test if they could write responses to all open-ended questions on a computer. The NBETPP and the Wellesley Public Schools prepared the report, which confirms findings from two similar studies conducted by Boston College researchers.

"MCAS has potential to help improve the quality of public education," said LSOE Senior Research Associate Michael Russell, who authored the report with Wellesley Public Schools Director of Libraries and Educational Technologies Tom Plati. "The test's current paper-and-pencil format, however, does not allow students who are accustomed to working on computers to produce their best work.

"These studies indicate that when students take written tests, they should have access to the same technology they use on a daily basis," Russell said. "If students are accustomed to writing with paper and pencil, then they should take tests using paper and pencil. But if students are used to writing with computers, then they must have access to computers during testing if we want to collect accurate samples of what students can do."

Results of the BC-Wellesley study attracted considerable attention from local and national media, including the Associated Press, Boston Sunday Globe, Boston Sunday Herald, Chronicle of Higher Education and the WBUR-FM "Here and Now" news magazine show.

As part of the study, Wellesley students in grades four, eight and 10 were given the composition item from the 1999 MCAS language arts tests and randomly assigned to write their responses on paper or on computer. All essays were transcribed to computer text so that MCAS raters did not know the mode in which they were written. Results showed that students who composed on computer or mini-word processors scored considerably higher than students who wrote on paper. Out of a total of 20 points, essays composed on computer scored about two points higher than essays written on paper.

Based on the 1999 MCAS results, the report states that allowing students the option to write both the MCAS composition item and the four shorter open-ended items on computer would move about 19 percent of Wellesley's fourth graders from the "needs improvement" category into the "proficient" category. In the case of one elementary school, the number of students scoring in the "proficient" category would jump from 35 percent to 60 percent. Overall, the number of Wellesley students in each grade level performing in the advanced category would double.

"Our teachers feel very strongly that using computers and portable word processors is the best way to teach the writing process," Plati said.
"When computers are used regularly as writing tools, teachers can work closely with students as they revise second, third, fourth and fifth drafts, helping students to understand that writing is a process, not a one-time event. The computer eliminates the drudgery of rewriting text by hand and allows students to focus on revising."

 

Return to Sept. 8 menu

Return to Chronicle home page