Adapting Peer Review to Journal Scope and Scale

Organizing a robust peer review process for an academic journal, especially a new one, involves several key steps, and certain aspects do change depending on the journal's domain and size.

Initial Assessment and Foundation (Especially for New Journals)

A new editor should begin by getting to grips with the mechanics of the journal, including the submission system and timelines, ideally in conjunction with the previous editor.
It is highly recommended to assess current practices using a tool like the COPE Journal Audit, which is designed to help editors identify areas needing attention or revision to adhere to COPE's Core Practices on publication ethics.
This assessment should be done in consultation with those familiar with the journal's abilities and constraints, such as the publisher and journal manager. Be aware that changing journal practices can take considerable time.
The COPE Journal Audit tool explicitly considers factors like the size of the journal staff, the resources available, and the discipline covered when evaluating how journals handle ethical issues.

Define and Document Clear Peer Review Policies and Procedures

Adopt a peer-review process that is appropriate for your journal's field of work and the resources/systems available.
All peer review processes must be transparently described and well managed. State on the journal website what content is peer reviewed, the model used (e.g., single-anonymous, double-anonymous, open), and how the process is managed. The model influences confidentiality and ownership of the review. Note that double-blind review is not always possible and there's no evidence it's inherently better than other models.
Publish clear guidelines for reviewers, including what is acceptable in a report regarding tone, language, and content. Instructions should clarify policies on tone, language quality, and content, emphasizing that comments should be constructive, courteous, and clear.
Establish and publish policies covering diverse aspects of peer review:
- How many reviewers review each manuscript.
- How peer reviewers are selected and trained.
- Policies on author-recommended and excluded reviewers. Authors may list preferred and non-preferred reviewers, and policies should state whether these suggestions are considered and if reasons for exclusion are required.
- Procedures for handling conflicts of interest for reviewers. Reviewers must declare potential conflicts and notify the journal if one arises or is discovered.
- How decisions about manuscripts are made and who is involved.
- Procedures for handling appeals and disputes that may arise in peer review. Decisions on appeals should be made by senior editors.
- Policies regarding the editing of peer reviews. The policy should be clearly stated, and ideally, edits are collaborative with the reviewer. Significant edits should be communicated to the reviewer. Edits should address tone/language but not change the reviewer's opinion. An alternative is providing guidance to the author on responding to a difficult review.
- Implement processes for preventing and detecting manipulation of the peer review process.
- Define policies and procedures regarding ethical oversight as part of the review criteria. This includes obtaining informed consent and ethical approvals for research involving humans or animals. Journals must ensure policies align with regulations and norms of their discipline. Reviewers should be asked to address these ethical aspects.
- Develop policies on data availability and encourage the use of reporting guidelines, which can be reviewed during the process. Policies may need to address this if appropriate to the subject area.
- Clarify policy on content already posted on preprint servers.

Establish and Manage the Editorial Board and Reviewers

Review and confirm the roles and responsibilities of all editors involved in peer review. Editorial Board Members (EBMs) may make final decisions, supervise special issues, review manuscripts, and help identify suitable reviewers.
Build a strong editorial board comprising recognized experts relevant to the journal's aims and scope. Vet new EBMs before onboarding, checking for suspicious affiliations. For larger journals, vetting might focus on leadership, while for smaller ones, the entire board might be reviewed.
Implement processes for selecting, training, and assessing the performance of reviewers. Reviewers may take on the role without guidance and need training. Journals should have systems for assessing reviewer performance and removing those with unacceptable performance.
Ensure assigned reviewers are qualified, free of conflicts of interest, and have no known ethics or performance concerns. Assign reviewers whose expertise matches the scope of the content.
Manage reviewer activity. Reviewers should agree to review only if qualified and unbiased. They should be prompt in responding to invitations and informing the journal of delays.
If supervisors involve students or junior researchers for training, they must request permission from the editor, and the student should be acknowledged as the reviewer of record.

Implement Efficient Submission Handling and Screening

Work with the publisher/owner to determine efficient and appropriate submission processes, which can involve electronic systems. Electronic systems can help ensure authors provide required information but should not be overly complex.
Implement preliminary screening (pre-check) procedures before peer review. This involves confirming the manuscript fits the journal's scope and relevance.
Check for controversial topics.
Ensure language quality.
Verify compliance with instructions for authors, including ethical requirements (human/animal studies, consent, ethical approvals, trial registration), data availability, image quality, and copyright. Compliance with international ethics guidelines should be checked for international research.
Run plagiarism checks and verify revisions have addressed overlap.
Verify author affiliations and emails, check authorship consistency and contributions, and look for previously flagged concerns or misconduct history.
Ensure figures, tables, and references are appropriate, well-presented, properly cited, and of sufficient quality. Check author/journal self-citation rates.
Consult senior editors if pre-check criteria are not met. Pre-peer review rejection reasons can include being out of scope, poor quality, or ethical violations.

Manage the Peer Review Process

Ensure material submitted remains confidential while under review. Reviewers must respect confidentiality.
Provide reviewers with clear instructions, time allowed, formatting, and what to consider, including ethical aspects. Reviewers should be objective and constructive.
Monitor the process regularly to ensure timeliness and prevent delays. Prompt responses to invitations and timely reviews are important.
Ensure review reports are constructive and professional.
Have a process for managing conflicting reviewer decisions, which might involve seeking additional reports or involving EBMs.
Ensure editors making final decisions are free of conflicts of interest.
Implement a clear procedure for handling submissions where an editor or board member is an author, ensuring independent peer review to avoid impropriety. Transparency about this process is recommended.

Ensure Ethical Conduct and Transparency

Decisions to accept or reject should be based only on the paper's importance, originality, clarity, and relevance to the journal's scope, avoiding bias related to author origin or other characteristics. Do not link editorial decisions to internal targets or expedite decisions for metric purposes.
Develop guidelines for promptly responding to suspected ethical breaches by authors, reviewers, and editors.
Have clear processes for handling complaints against the journal, staff, editorial board, or publisher.
Be willing and have mechanisms in place for correcting, clarifying, revising, or retracting articles after publication when needed. Prompt retraction of seriously flawed articles is a responsible action.
Maintain a well-supported, maintained, and secure journal website, which is the public-facing statement of ethical practices.
Ensure the journal's name is unique and not misleading.
Clearly state the publishing frequency and content preservation plans.
Follow clear communication practices and do not disclose inappropriate information.

For a new journal, the initial assessment phase and establishing all policies and procedures from scratch are particularly critical. Consulting resources like COPE's Core Practices and using tools like the COPE Journal Audit are essential starting points.

The domain or discipline of the journal significantly impacts the types of ethical considerations that must be integrated into the peer review process (e.g., specific requirements for studies involving humans, animals, or clinical trials). It also dictates the necessary expertise of reviewers and editorial board members.

The size and resources of the journal influence the complexity and staffing of the editorial office. Smaller journals might need to rely more heavily on part-time staff or prioritize which EBMs to vet thoroughly. The chosen submission system and overall process efficiency might also be constrained by available resources.