Grading Software Offers Professors a Break By John Markoff, New York Times
ESL Voices Lesson Plan for this post with Answer Key.
Robotic grading is not really new. There are already automated grading systems for multiple-choice and true-false tests used in schools everywhere in the U.S. Teachers spending hours correcting written essays is also nothing new. What is different is having essays corrected by an automated software program. The idea of using artificial intelligence in place of teachers to grade essays has not been accepted by everyone, especially those in the teaching profession. Will this type of program benefit teachers and students? The following article offers the pros and cons of robotic essay grading.
“Imagine taking a college exam, and, instead of handing in a blue book and getting a grade from a professor a few weeks later, clicking the “send” button when you are done and receiving a grade back instantly, your essay scored by a software program. And then, instead of being done with that exam, imagine that the system would immediately let you rewrite the test to try to improve your grade.
EdX, the nonprofit enterprise founded by Harvard and the Massachusetts Institute of Technology to offer courses on the Internet, has just introduced such a system and will make its automated software available free on the Web to any institution that wants to use it. The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks. The new service will bring the educational consortium into a growing conflict over the role of automation in education.
Although automated grading systems for multiple-choice and true-false tests are now widespread, the use of artificial intelligence technology to grade essay answers has not yet received widespread endorsement by educators and has many critics.
Anant Agarwal, an electrical engineer who is president of EdX, predicted that the instant-grading software would be a useful pedagogical tool, enabling students to take tests and write essays over and over and improve the quality of their answers…
But skeptics say the automated system is no match for live teachers. One longtime critic, Les Perelman…a retired director of writing and a current researcher at M.I.T. has drawn national attention several times for putting together nonsense essays that have fooled software grading programs into giving high marks. He has also been highly critical of studies that purport to show that the software compares well to human graders.
He is among a group of educators who last month began circulating a petition opposing automated assessment software. The group, which calls itself Professionals Against Machine Scoring of Student Essays in High-Stakes Assessment, has collected nearly 2,000 signatures, including some from luminaries like Noam Chomsky…
The group’s statement reads in part: “Computers cannot ‘read. Let’s face the realities of automatic essay scoring… They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others.
But EdX expects its software to be adopted widely by schools and universities. EdX offers free online classes from Harvard, M.I.T. and the University of California, Berkeley; this fall, it will add classes from Wellesley, Georgetown and the University of Texas. In all, 12 universities participate in EdX, which offers certificates for course completion and has said that it plans to continue to expand next year, including adding international schools.
Mark D. Shermis, a professor at the University of Akron in Ohio, supervised the Hewlett Foundation’s contest on automated essay scoring and wrote a paper about the experiment. In his view, the technology — though imperfect — has a place in educational settings. With increasingly large classes, it is impossible for most teachers to give students meaningful feedback on writing assignments.” Read more…
Level: Intermediate -Advanced
Language Skills: Reading, writing, speaking and listening. Vocabulary and grammar activities are included.
Time: Approximately 2 hours.
Materials: Student handouts (from this lesson) access to news article, and video.
Objective: Students will read the article with a focus on improving reading comprehension and learning new vocabulary. At the end of the lesson students will express their personal views on the topic of automated grading software through discussions, debates, and writing activities.
Analyzing headings and photos
Directions: Place students in groups and ask each group to read the title of the post and of the article. Then, have them examine the photos. Based on these sources, ask students to create a list of words and ideas that they think might be related to this article.
II. While Reading
Directions: have students choose the synonyms (the words closest in meaning) for the words in bold. Students will find this great Vocabulary Cluster Map By Learnnc.org a useful tool.
1. Imagine taking a college exam and receiving a grade back instantly.
a. at once
2. EdX is a nonprofit enterprise. Synonym ___________.
a. space ship
3. The use of artificial intelligence technology to grade essay answers has not yet received widespread endorsement. Synonym ___________.
4. He said the technology would offer distinct advantages over the traditional classroom system. Synonym ___________.
c. long established
5. EdX predicted that the instant-grading software would be a useful pedagogical tool. Synonym ___________.
6. But skeptics say the automated system is no match for live teachers.
7. The challenge is that this requires human graders. Synonym___.
8. With increasingly large classes, it is impossible for most teachers to give students meaningful feedback. Synonym ___________.
9. Critics of the technology have tended to come from the nation’s best universities. Synonym ___________.
10. They often come from very prestigious institutions. Synonym___.
True / False
Directions: The following statements were taken from the article. If a statement is true, students write (T) if a statement is false they write (F) and provide the correct answer from the article.
- EdX the nonprofit enterprise was founded by Stanford and Massachusetts Institute of Technology.
- The software uses artificial intelligence to grade student essays.
- Automated grading systems for multiple-choice and true-false tests are new in the education system.
- The use of artificial intelligence technology to grade essay answers has received widespread endorsement.
- Agarwal Agarwal is a professor and volunteer at EdX.
- According to Agarwal, the technology would offer distinct advantages over the traditional classroom system, where students often wait days or weeks for grades.
- Les Perelman is a retired director of writing and a current researcher at M.I.T. who has put together nonsense essays that have fooled software grading programs into giving high marks.
- Perelman’s greatest objection to the research was that they did not have any robots writing essays.
- There is now a range of companies offering commercial programs to grade written test answers.
- According to Mark D. Shermis with increasingly large classes, it is impossible for most teachers to give students meaningful feedback on writing assignments.
Structure and Usage
Directions: The following groups of sentences are from the article (some may be abbreviated versions). One sentence in each group contains a grammatical error. Students are to identify the sentence (1, 2, or 3) from each group that contains the grammatical error and make the correction.
- The software uses artificial intelligence to grade student essays.
- The new service will brings the educational consortium into a growing conflict.
- The use of artificial intelligence technology to grade essay answers has not yet received widespread endorsement.
- The EdX assessment tool requires human teachers, to first grade 100 essay questions.
- The software will assign a grade depending on the scoring system created by the teacher.
- It will also provided general feedback.
- This is machine learning and there is a long way to go.
- We found that the quality of the grading is similar to the variation you find from instructors.
- EdX is not the first to used automated assessment technology.
III. Post Reading
Reading Comprehension Check
Directions: Have students use the WH-question format to discuss or to write the main points from the article.
Who or What is the article about?
Where does the action/event take place?
When does the action/event take place?
Why did the action/event occur?
How did the action/event occur?
Directions: An option is to have students use this advanced organizer from Enchanted Learning to assist them with discussing or writing about the main points from the article.
Directions: Place students in groups and have them answer the following questions. After, have the groups share their thoughts as a class. To reinforce the ideas, students can write an essay on one of the topics.
- Anant Agarwal, stated that “the instant-grading software would be a useful pedagogical tool, enabling students to take tests and write essays over and over and improve the quality of their answers.” Do you think that a machine can “improve” a student’s quality of writing? Explain why or why not.
- Restate the following comment into your own words. “The EdX assessment tool requires human teachers, or graders, to first grade 100 essays or essay questions. The system then uses a variety of machine-learning techniques to train itself to be able to grade any number of essays or answers automatically and almost instantaneously.”
- “Computers cannot ‘read.’ They cannot measure the essentials of effective written communication: accuracy, reasoning, or meaningful organization.” Provide reasons to support or to refute this statement.
- What are your thoughts on having a computer grade your essay? Do you think that you gain more by having your teacher grade your writing? Provide reasons for your answers.
Directions: Divide students into two teams for this debate. Both teams will use the article as their source of information.
Team A will list five reasons for automated essay checking.
Team B will list five reasons against automated essay checking.
For organization, have students use this great Pros and Cons Scale organizer from Freeology .
IV. Listening Activity
Video Clip: Japanese School Tests Robot Teacher
Students at the Kudan Elementary School in downtown Tokyo were told a special teacher would help them with their science class. They had a big surprise!
True /False/NA statements
Directions: Review the statements with students before the watching the video. As students listen to the video if a statement is true they mark it T if the statement is false they mark it F and provide the correct answer. If there is no information available for a statement then students write NA.
According to the video:
- The robot teacher needed 2 men to carry her up to the podium.
- She was built by a student from Tokyo University of Science.
- The robot teacher has been used at other Japanese schools.
- She’s not meant to take jobs away from teachers.
- The main reason for building the robot was to use new technology to teach children about technology.
- The robot may be able to help in schools where there is an abundance of teachers.
- There are 2 more robots like this one.
- Scientists want to develop the robot to be controlled remotely to teach these classes.
- The children liked the robot.
- The professor didn’t think that the robot was ready to teach a class.
Directions:Place students in groups and have them discuss the following questions.
- Based on the article, your discusions, debate, and this video, discuss the pros and cons of providing robot teachers in classrooms.
- Would you like to experience working with a robot teacher? Provide reasons why or why not.
- With your group members, make up questions that you would like to ask the professor (who created the robot), the students, or the teacher.
That Dastardly Computer Gave My Essay a D! New York Times Opinion Page (4/11/13)
Please come back again.