Crowd-Grades

To compute and view the grades, click on the Grades link.  The crowd-grade of a student depends on the submission grade and review grade received by the student.  This creates an incentive for students to both do their homework well, and provide accurate grades and helpful feedback to others. 

• • Submission grade: CrowdGrader computes consensus grades for each submission, with the help of a reputation system.  The reputation system compares all grades given by the students, and:

    • eliminates outliers,

    • learns the positive or negative bias of each student, and corrects for it,

    • gives more weight to the input of students who are more precise in their grades. 

  • Review grade:  For each student, CrowdGrader computes a review grade that depends in equal measure on two factors:

    • How precise are the grades given by the student. CrowdGrader compares the grades written by the student with the consensus grades: the closer they are, the higher is the review grade of the student.

    • How helpful are the reviews written by the student. Students can rate the reviews they receive, from 1 star (bogus, completely unhelpful) to 5 stars (very helpful review).  CrowdGrader eliminates ratings that are given in retaliation, and considers the remaining reliable ratings as a component of the review grade.

The Crowd-Grade received by each student is computed by merging submission and review grades according to the proportions specified in the assignment (a typical proportion is 75% submission grade and 25% review grade).