Photo: M. Scott Brauer for The Chronicle. Les Perelman (left), with the help of students at MIT and Harvard, created the Babel Generator, a software program that generates meaningless essays to test the mettle of machine graders.
Les Perelman is an old high school chum and presently retired as director of undergraduate writing at MIT. He is a long-time critic of the use and misuse of standardized testing. Though retired from MIT, he continues this work.
In this article from the Chronicle of Higher Education, they discuss his babel generator and what it means for computer-based assessments.
Les Perelman, a former director of undergraduate writing at the Massachusetts Institute of Technology, sits in his wife’s office and reads aloud from his latest essay.
“Privateness has not been and undoubtedly never will be lauded, precarious, and decent,” he reads. “Humankind will always subjugate privateness.”
Not exactly E.B. White. Then again, Mr. Perelman wrote the essay in less than one second, using the Basic Automatic B.S. Essay Language Generator, or Babel, a new piece of weaponry in his continuing war on automated essay-grading software.
The Babel generator, which Mr. Perelman built with a team of students from MIT and Harvard University, can generate essays from scratch using as many as three keywords.
For this essay, Mr. Perelman has entered only one keyword: “privacy.” With the click of a button, the program produced a string of bloated sentences that, though grammatically correct and structurally sound, have no coherent meaning. Not to humans, anyway. But Mr. Perelman is not trying to impress humans. He is trying to fool machines.
Critics of automated essay scoring are a small but lively band, and Mr. Perelman is perhaps the most theatrical. He has claimed to be able to guess, from across a room, the scores awarded to SAT essays, judging solely on the basis of length. (It’s a skill he happily demonstrated to a New York Times reporter in 2005.) In presentations, he likes to show how the Gettysburg Address would have scored poorly on the SAT writing test. (That test is graded by human readers, but Mr. Perelman says the rubric is so rigid, and time so short, that they may as well be robots.)
Fred Klonsky 