Set up the review assignment procedure
On this page
In the submission process, it is possible to ask submitters to flag their submission as either ready to be reviewed, or as work in progress. This option can be enabled by activating field-ID 239, "Status of this submission" on the submission form. If this field is enabled, the allocation of reviews will only consider submissions flagged as completed.Allocate reviews manually once all the submissions have been collected
This option is suitable if you wait until the end of the submission process to start assigning the reviews.
- Step 1: Prepare the email templates
- Invitations and reminders can be edited and later sent in bulk to the reviewers at Emails / Bulk emails / Reviewers.
- Be sure to deactivate the sending of automatic emails at Emails / Automatic emails / Activate and edit the emails / Send an automatic email to a reviewer as soon as a review is assigned to that reviewer
- Step 2: Set reviewers' permissions to access the reviews
- You can control reviewers' ability to view and edit their assignments at Configuration / Access rights / Access to functions & deadlines.
- While reviews are being allocated, you might want to disable access, in case you decide to redistribute some of the assignments.
- Once the allocation is completed, you can enable access and send an invitation to start the reviews. As an option, you can set an availability period that matches your deadlines.
- Step 3: Check the constraints
- You can check that the automatic detection of conflicts is set at Configuration / Reviews / Options for the review process.
- If the reviewers are assigned speciality topics, you can check or assign them at Submissions & reviews / Review assignment / Assign topics to reviewers.
- If relevant, you can check that accepted and/or rejected submissions are excluded from the list at Configuration / Reviews / Options for the review process.
- Step 4: Assign the reviews
- To assign the reviews manually, go to Submissions & reviews / Review assignment / Assign reviews manually.
- In section 1, you can select the submissions and reviewers to work on (Fig. 1). If reviewers are given speciality topics, filtering by a topic will select the submissions and reviewers associated with that topic and leave out all others. This will help ensure that reviewers are only assigned reviews corresponding to their speciality.
- In section 2, you can allocate reviews for the selected submissions and reviewers.
- In section 3, you can get an overview of the number of assignments for each reviewer.
Allocate reviews automatically once all the submissions have been collected
This option is suitable if you wait until the end of the submission process to start assigning the reviews and want to perform a quick assignment that takes your constraints into account.
- Step 1: Prepare the email templates
- Invitations and reminders can be edited and later sent in bulk to the reviewers at Emails / Bulk emails / Reviewers.
- Be sure to deactivate the sending of automatic emails at Emails / Automatic emails / Activate and edit the emails / Send an automatic email to a reviewer as soon as a review is assigned to that reviewer
- Step 2: Set reviewers' permissions to access the reviews
- You can control reviewers' ability to view and edit their assignments at Configuration / Access rights / Access to functions & deadlines.
- While reviews are being allocated, you might want to disable access, in case you decide to redistribute some of the assignments.
- Once the allocation is completed, you can enable access and send an invitation to start the reviews. As an option, you can set an availability period that matches your deadlines.
- Step 3: Check the constraints
- You can check that the number of reviewers per submission is correctly set at Configuration / Reviews / Options for the review process.
- You can check that the automatic detection of conflicts is set at Configuration / Reviews / Options for the review process.
- If the reviewers are assigned speciality topics, you can check or assign them at Submissions & reviews / Review assignment / Assign topics to reviewers.
- If relevant, you can check that accepted and/or rejected submissions are excluded from the list at Configuration / Reviews / Options for the review process.
- Step 4: Assign the reviews
- To assign the reviews automatically, go to Submissions & reviews / Review assignment / Assign reviews automatically. This page contains two buttons: one used to assign the reviews; the other used to clear assigned reviews. The following rules apply:
- When you click on the review assignment button, the software distributes the reviews at random. In doing so, it takes, as much as technically feasible, the constraints into account: the number of reviewers per submission; the speciality topics; detected conflicts; biddings, if relevant.
- The automatic assignment can be used incrementally. If you do not wait until the end of the submission process to allocate reviews, you can visit this page at regular intervals, say once a week, and assign reviews. The software will take existing assignments into account and apply the rules described above.
- Using the clearing button will result in existing assignments being removed, but only if these assignments have not been started. Reviews for which a status of
CompletedorUnder evaluationis found cannot be removed.
Allocate reviews manually after a submission is created
This option is suitable if you do not wait until the end of the submission process to start assigning the reviews. With this option, automatic notifications can be emailed to the reviewers each time a submission is assigned to them.
- Step 1: Prepare the email templates
- Activate the sending of automatic emails at Emails / Automatic emails / Activate and edit the emails / Send an automatic email to a reviewer as soon as a review is assigned to that reviewer
- The template for this email is pre-filled but can be edited. Please be sure to include the
{list_of_assigned_submissions}placeholder. It will be replaced, independently for each affected reviewer, by the list of their newly assigned reviews.
- Step 2: Set reviewers' permissions to access the reviews
- Make sure that reviewers can access and edit their assignments at Configuration / Access rights / Access to functions & deadlines.
- Step 3: Check the constraints
- You can check that the automatic detection of conflicts is set at Configuration / Reviews / Options for the review process.
- If the reviewers are assigned speciality topics, you can check or assign them at Submissions & reviews / Review assignment / Assign topics to reviewers.
- If relevant, you can check that accepted and/or rejected submissions are excluded from the list at Configuration / Reviews / Options for the review process.
- Step 4: Assign the reviews
- To assign the reviews manually, go to Submissions & reviews / Review assignment / Assign reviews manually.
- In section 1, you can select the submissions and reviewers to work on (Fig. 1). If reviewers are given speciality topics, filtering by a topic will select the submissions and reviewers associated with that topic and leave out all others. This will help ensure that reviewers are only assigned reviews corresponding to their speciality.
- In section 2, you can allocate reviews for the selected submissions and reviewers.
- In section 3, you can get an overview of the number of assignments for each reviewer.
Allocate reviews automatically as soon as a submission is created
This option is suitable if you do not wait until the end of the submission process to start assigning the reviews and want to perform a quick assignment that takes your constraints into account. With this option, automatic notifications can be emailed to the reviewers each time a submission is assigned to them.
- Step 1: Activate the option
- Go to Configuration / Reviews / Options for the review process and activate automatic assignments at 6. Automatic assignments.
- Step 2: Prepare the email templates
- Activate the sending of automatic emails at Emails / Automatic emails / Activate and edit the emails / Send an automatic email to a reviewer as soon as a review is assigned to that reviewer
- The template for this email is pre-filled but can be edited. Please be sure to include the
{list_of_assigned_submissions}placeholder. It will be replaced, independently for each affected reviewer, by the list of their newly assigned reviews.
- Step 3: Set reviewers' permissions to access the reviews
- Make sure that reviewers can access and edit their assignments at Configuration / Access rights / Access to functions & deadlines.
- Step 4: Check the constraints
- You can check that the automatic detection of conflicts is set at Configuration / Reviews / Options for the review process.
- If the reviewers are assigned speciality topics, you can check or assign them at Submissions & reviews / Review assignment / Assign topics to reviewers.
- If relevant, you can check that accepted and/or rejected submissions are excluded from the list at Configuration / Reviews / Options for the review process.
- Step 5: Tracking the reviews
- Assignments will be made automatically and displayed at Submissions & reviews / Review assignment / Assign reviews manually where they can be tracked and manually adjusted (e.g. you can set extra reviews).
- In section 1, you can select the submissions and reviewers to view (Fig. 1).
- In section 2, you can view or modify existing reviews for the selected submissions and reviewers.
- In section 3, you can get an overview of the number of assignments for each reviewer.