Frequently Asked Questions
Can CrowdGrader be used for essays and other writing assignments?
Yes, and there are many instructors using it in such a way.
CrowdGrader was initially used by its authors for computer science
assignments, CrowdGrader is used in all kind of assignments.
What is the maximum class size CrowdGrader can handle?
There is no specific limit. CrowdGrader
has been successfully used for classes with up to about 800 students.
It should be able to handle easily classes with many thousands
students. If you are interested in using CrowdGrader for classes with tens of thousands of students, contact us.
CrowdGrader can be used also
for very small classes, but of course, if you have only two people in
the class, you can ask that they do only one review each, and they will
know whom they are reviewing!
Help! Where is my assignment?
If you don't find your assignment, you might be logged in using a different account than the one you used for the assignment. When you join an assignment, CrowdGrader sends you an email confirming your joining. That email might help you remember which account you used to join.
Can CrowdGrader detect plagiarism or copying?
What type of documents can students upload?
Students can upload any type of document. Students can also use the wiki-like editor embedded in CrowdGrader to write their submission; the editor supports also math
using a LaTeX-like syntax.
Is there a size limit for CrowdGrader submissions?
CrowdGrader does not enforce a size limit. Your browser might limit the size of the files it is able to upload. However,
please be considerate of the students who will have to review your
submission. They may be working from home, or have a so-so internet
connection, and they might not enjoy having to wait long time while
downloading your submission.
How can I submit a late assignment?
CrowdGrader does not accept late assignments from students. Each
instructor has her or his own policy for late homework, and we have
chosen to leave this in control of the instructor. If you wish
to submit or modify your submission after the deadline, send your
submission to the instructor; she or he will be able to add it on your
How can I add a late submission on behalf of a student?
If you need to add a late submission on behalf of a student, you can do
so from the Submission page, accessible from the main assignment page.
I forgot to confirm my membership in a group. Now what do I do?
Let your instructor know. She or he will be able to add you to the group (after checking with the other group members that you are indeed part of that group).
Why can't review feedback be left after the grades have been computed?
The grades assigned by CrowdGrader take into account the review feedback received by the students. If students could provide feedback
after the grades are computed, it would not be clear which feedback
is used to compute the grades, and which came too late.
when CrowdGrader computes the grades, it also labels some feedback as
tit-for-tat. An example of tit-for-tat is a student giving a low
feedback (e.g., -2) to a reviewer who assigned a low grade to the
student's solution. If feedbacks could be left after the grades are
computed, CrowdGrader would not be able to analyze all feedback for
I cannot open / make sense of a submission. What should I do?
If you cannot open or otherwise make sense of a submission, simply decline the review, adding an explanation of why you were not able to complete it.
The submission is empty/missing. Should I enter a grade, or decline it?
Enter a grade. You should decline submissions only when you are unable to assess their quality, but missing submissions... have no quality!
I asked students to do N reviews, yet I see some submissions with fewer than N reviews. How comes?
If you ask each student to do N reviews, every submission will generally receive N reviews, but occasionally may receive N-1 or N+1 reviews. There are several reasons why a submission may receive fewer (one less) than N reviews:
- Some students may complete less than N reviews.
- To ensure review uniformity, CrowdGrader needs to decide whether outstanding reviews are likely to be completed, or are likely to be never done. Assigned reviews that are unlikely to be ever done are assigned to other students. For instance, if a review has been outstanding for a couple of days, CrowdGrader judges it unlikely that it will be completed, and assigns the submission to be reviewed by other students to compensate. CrowdGrader tries to be smart in guessing, but occasionally gets it wrong, so that a submission will get one too many reviews, and another will get one too few.
- CrowdGrader cannot assign any submission to be reviewed by any student: it must obey some constraints. For instance, CrowdGrader cannot assign a submission to be reviewed by one of its authors. These constraints, together with the dynamic review assignment used by CrowdGrader, sometimes cause a plus/minus one in the number of reviews received by submissions.
All the reviewers gave me 10; why is my crowd-grade less than 10?
CrowdGrader learns not only the precision of each reviewer, but also their positive or negative bias. If the people who reviewed your submission and gave it the top grade (10, in this example) were overly generous in grading other submissions, CrowdGrader comes to believe that their 10's are not always worth a 10, but a bit less, to compensate for their generosity. So it might be that even though you got all 10's, the computed crowd-grade is a bit lower, for instance, 9.8.
Can I use a spreadsheet to compute the final grades?