Reviews
The review process
Students can see which reviews they need to do from the main assignment list they see once they log in.
Reviews are assigned one by one. To perform reviews, students click on a button Request review, and are assigned a submission to review. The students can read or download the submission, and they are presented with the rubric for reviewing it. Students need to complete a review before they can ask for the next submission to review.
Reviews are anonymous. CrowdGrader does not show the identity of the submission's author to the students reviewing it, and it renames the files uploaded by the students to preserve anonymity.
For each assignment, students need to complete at least the number of reviews specified by their instructor, or their review grade will suffer.
If students by mistake click and request to do one too many reviews, they can just leave the extra review incomplete: their review grade will not suffer.
Students can choose to do extra reviews, if they wish to see more work done by their fellow students.
Resistance to collusion: CrowdGrader is resistant to collusion, since students cannot choose whose submissions they review.
Performing a review
Students can write feedback for the submission they are reviewing using the CrowdGrader editor. If the submission consists in a CrowdGrader document, the students can use the review tools to directly annotate the document:
The red import arrow lets students import the document submitted, so that it can be annotated.
The red pencil lets students (and instructors) mark the document adding their comments in red.
The red strikeout tool lets students (and instructors) strike out portions of the document using red color.
Declining a review
If students cannot understand the submission, they can decline the review, thus avoiding giving arbitrary or very low grades to submissions that they cannot properly evaluate. To avoid that students decline may submissions as a way to pick and choose what to review, the instructor can set the maximum number of reviews a student can decine; by default, this is set to 2.
Discussing the submission with its author(s)
If students do not understand some details of the submission they need to review, they can discuss it with its authors in an anonymous forum. In this way, small mis-understandings can be cleared up without affecting the grades.
Instructors can review too!
Instructors can easily see the list of submissions, and the grades they have received so far. They can also see the amount of disagreement on each submission, and they can sort the reviews in order of greatest-disagreement-first. Instructors can also choose to grade some of the submissions themselves: their grade will take precedence over the grades provided by students when computing the final grade.
