The Robots Are Coming … To Grade

by | May 2, 2012 | Uncategorized

Gort from the original The Day the Earth Stood Still

It’s term paper time. For the next week, my life will be non-stop grading. I’ll spend hours searching, sometimes in vain, for thesis statements and for that rare and beautiful thing called argument. Okay — I’m being cynical. Some of the papers are actually a pleasure to read, but others are … less engaging. In those times, I stare out my window and think there has to be a better way to grade.

There is: Robots!

Gort from the original The Day the Earth Stood Still

Gort's deadly eye-beam, from the original The Day the Earth Stood Still

Robo-graders, actually. Not quite as fun as robots; fewer deadly eye-beams and more servers in the cloud. The New York Times reports that Robo-Graders are now being used for various standardized writing tests. The program works by analyzing sentence structure, length, word choice, and other features, allowing it to grade in seconds what would take a human weeks.

One small problem though: the Robo-Grader has no idea if what you’re saying is actually true. Les Perelman, a researcher who studies automatic grading, has found that long essays employing certain sentence structures — but nonsensical arguments and content, even references to Jedi Knights in a question about college tuition —  achieve top scores. Short, thoughtful essays score less well. Also no extra credit for poetic or creative use of the language. It doesn’t compute.

Apparently, perfectly structured and formatted nonsense is the wave of the future, at least as far as standardized testing is concerned. While I would love to hand over my grading duties to the robots, they seem to exact too high a price. Truth in writing is worth looking for and rewarding when found, no matter how long it takes. Truth anywhere is worth looking for, even and especially when, as in grading essays, finding it is a long and difficult process with more than a few false starts.

Now if you’ll excuse me … I should be grading.


Matthew Dunch, SJ   /   All posts by Matthew