This Fall 2016, the University of Waterloo will have 25 courses with 
a class size of between 500 and 1000 students and 10 courses of between 1000 and 2000 students.
The amount of paper handling to administer the potential 33,000 final exam papers from these large courses will be monumental. (For fun, guestimate the volume of paper this amounts to.)
The Mathematics Faculty has been successfully experimenting for a year with a online grading system called Crowdmark, a company founded by Professor James Colliander of the Mathematics Department of the University of Toronto.
Professor Colliander was faced with a similar problem: grading 5000 Canadian Open Mathematics Competition (COMC) papers each year with 100 volunteers. As with final exams, each paper is typically graded by a number of markers so keeping track of which questions are graded on which papers and when the papers are free to be passed to another marker is a time consuming and error prone business.
Crowdmark (CM) attempts to eliminate some of the time and trouble spent managing the grading process. We are not talking about a quiz system with automatic grading. Crowdmark is hand-marking done online. Skilled people still grade, and tests and assignments are still created for printing on paper so there is nothing new in this part of an instructor’s routine.
So, what is it that makes the marking process more efficient when done online?
- Markers are able to grade the same paper at the same time. No more locating and waiting for a paper that someone else is grading. Or waiting for a batch of papers to arrive at your location to begin your stage of grading. Grading can be done concurrently at multiple locations and times.
- Grades can be automatically summed, collected, summarized, distributed and recorded in a Learning Management System without needing to check for arithmetic or transcription errors.
- No time needs to be spent returning piles of exam papers.
There is a time and money cost to using online grading. The physical papers have to be scanned into digital format (PDF file) before grading can start. High speed scanners (500 pages per minute) can process 1000 10-page exams in 20-30 minutes once delivered to the scanning machine.
Here I’ll briefly discuss how instructors and students use CM.
Steps for an instructor:
- upload one test or exam pdf file into CM (leave 2 inches blank on the top of each page for CM ID info and set 1 question per page)
- CM duplicates the test pdf for each student and adds a paper and page ID to each page
- download from CM the pdf file of student tests and print it
- after the test scan all written test papers into a pdf file and upload the file into CM
- CM arranges the pdf file pages into a grid pattern: each row holds a student’s test pages
- each marker clicks on a page in the grid to read, comment, and grade it
- when grading is complete page grades are summed for each test paper by CM
- match each test paper cover page student ID with a student name in your CM course (assigned seating at UW can eliminate this step)
- you choose whether CM sends each student their grade and a CM link to their graded test paper or to keep the grades and graded papers private and just download the grades for inclusion into a course grade
Steps for a student:
- write the test paper by hand as usual
- may receive an email from CM with a link to a CM page showing their test results
The links at the end of this post provide further details about Crowdmark. In addition, 2 live sessions demonstrating Crowdmark are coming up at the end of August and the beginning of September. The first is an introduction to Crowdmark on Wednesday August 31 and the second follows up a week later on Wednesday September 7 (1:30-3 PM) with details about a University of Waterloo system named Odyssey that works with Crowdmark. Odyssey organizes test papers, students and exam room seating providing relief from some time-consuming management overhead.
Crowdmark is not a free service, but the University of Waterloo has a licence so there is no charge to individuals (instructors or students) at the university.
If you are interested in learning more about online grading for your course please get in touch with me.
Paul Kates
Mathematics Faculty CTE Liaison
pkates@uwaterloo.ca, x37047, MC 6473
Intro to Online Marking using Crowdmark: Wednesday, August 31, 2016 – 10:30 AM to 11:30 AM EDT
Crowdmark home page, help pages and youtube channel.
UW Odyssey Examination Management
