Consensus-based grading

NEW FOR NOVEMBER 2016

Consensus-Based Grading

In November 2016, CrowdGrader introduced a new method for grading, called consensus-based grading.  The main novelty is that students no longer see the grades they are assigned by each individual reviewer, but rather, they see only the consensus computed from those grades.  This increases student satisfaction, as students no longer stress due to possible inaccuracies of individual graders.  

• • If rubrics are not used, then the consensus grade is computed as the median of the grades provided by the individual reviewers. 

  • If rubrics are used, then CrowdGrader computes the median of each rubric entry, separately.  This computes "consensus rubric grades" that are then shown to the student. 

If rubrics are used, consensus-based grading is more precise than the previous algorithm, since the errors are eliminated on each point of the rubric before the total grade is computed.  

We have also eliminated the feedback phase.  We are considering bringing back the feedback phase with a simple thumbs-up or down approach, in which the feedback to a review would be entirely based on the textual content of the review, as the individual grades associated with each review are no longer shown to the students.  This would provide an incentive to provide accurate reviews, while avoiding any retaliation for low grades (as the grades of each individual review are no longer shown to students).