FAQs

Frequently Asked Questions

General

Can CrowdGrader be used for essays and other writing assignments?

Yes, and there are many instructors using it in such a way.

Although CrowdGrader was initially used by its authors for computer science assignments, CrowdGrader is used in all kind of assignments.

What is the maximum class size CrowdGrader can handle?

There is no specific limit.  CrowdGrader has been successfully used for classes with up to about 800 students.  It should be able to handle easily classes with many thousands students.  If you are interested in using CrowdGrader for classes with tens of thousands of students, contact us.

CrowdGrader can be used also for very small classes, but of course, if you have only two people in the class, you can ask that they do only one review each, and they will know whom they are reviewing!

Help!  Where is my assignment?

If you don't find your assignment, you might be logged in using a different account than the one you used for the assignment.  When you join an assignment, CrowdGrader sends you an email confirming your joining.  That email might help you remember which account you used to join.

Can CrowdGrader detect plagiarism or copying?

Yes, CrowdGrader now includes a powerful system for comparing submissions, that works with many submission types, including docx documents, and zip and tar archives.

Submission

What type of documents can students upload?

Students can upload any type of document.  Students can also use the wiki-like editor embedded in CrowdGrader to write their submission; the editor supports also math using a LaTeX-like syntax.

Is there a size limit for CrowdGrader submissions?

CrowdGrader does not enforce a size limit.  Your browser might limit the size of the files it is able to upload.  However, please be considerate of the students who will have to review your submission.  They may be working from home, or have a so-so internet connection, and they might not enjoy having to wait long time while downloading your submission.

How can I submit a late assignment?

CrowdGrader does not accept late assignments from students.  Each instructor has her or his own policy for late homework, and we have chosen to leave this in control of the instructor.  If you wish to submit or modify your submission after the deadline, send your submission to the instructor; she or he will be able to add it on your behalf.

How can I add a late submission on behalf of a student?

If you need to add a late submission on behalf of a student, you can do so from the Submission page, accessible from the main assignment page.

I forgot to confirm my membership in a group.  Now what do I do?

Let your instructor know.  She or he will be able to add you to the group (after checking with the other group members that you are indeed part of that group).

Review

Why can't review feedback be left after the grades have been computed?

The grades assigned by CrowdGrader take into account the review feedback received by the students.  If students could provide feedback after the grades are computed, it would not be clear which feedback is used to compute the grades, and which came too late. 

Moreover, when CrowdGrader computes the grades, it also labels some feedback as tit-for-tat.  An example of tit-for-tat is a student giving a low feedback (e.g., -2) to a reviewer who assigned a low grade to the student's solution.  If feedbacks could be left after the grades are computed, CrowdGrader would not be able to analyze all feedback for tit-for-tat behavior.

I cannot open / make sense of a submission.  What should I do?

If you cannot open or otherwise make sense of a submission, simply decline the review, adding an explanation of why you were not able to complete it.

The submission is empty/missing.  Should I enter a grade, or decline it?

Enter a grade.  You should decline submissions only when you are unable to assess their quality, but missing submissions... have no quality!

I asked students to do N reviews, yet I see some submissions with fewer than N reviews.  How comes?

If you ask each student to do N reviews, every submission will generally receive N reviews, but occasionally may receive N-1 or N+1 reviews.  There are several reasons why a submission may receive fewer (one less) than N reviews: 

• • Some students may complete less than N reviews. 

  • To ensure review uniformity, CrowdGrader needs to decide whether outstanding reviews are likely to be completed, or are likely to be never done.  Assigned reviews that are unlikely to be ever done are assigned to other students.   For instance, if a review has been outstanding for a couple of days, CrowdGrader judges it unlikely that it will be completed, and assigns the submission to be reviewed by other students to compensate.  CrowdGrader tries to be smart in guessing, but occasionally gets it wrong, so that a submission will get one too many reviews, and another will get one too few.

  • CrowdGrader cannot assign any submission to be reviewed by any student: it must obey some constraints.  For instance, CrowdGrader cannot assign a submission to be reviewed by one of its authors.  These constraints, together with the dynamic review assignment used by CrowdGrader, sometimes cause a plus/minus one in the number of reviews received by submissions.

Grades

All the reviewers gave me 10; why is my crowd-grade less than 10?

CrowdGrader learns not only the precision of each reviewer, but also their positive or negative bias.  If the people who reviewed your submission and gave it the top grade (10, in this example) were overly generous in grading other submissions, CrowdGrader comes to believe that their 10's are not always worth a 10, but a bit less, to compensate for their generosity.  So it might be that even though you got all 10's, the computed crowd-grade is a bit lower, for instance, 9.8. 

Instructors can easily compensate for this by recurving the grades, manually specifying a few final grades, and computing the others by interpolation. 

Can I use a spreadsheet to compute the final grades?

Absolutely.  You can download the crowd-grades along with submission and review grades, compute the final grades e.g. using a spreadsheet, and then upload them along with additional feedback.