The Problem with Perfect – Making Peer Review More User-Friendly

Stockholm University Press is a researcher-led publisher, and as such, we have to continuously improve and adapt our processes and practices in collaboration with authors, editors and reviewers to ensure that we provide the best possible opportunities for communication of good quality research. But it is of course not easy to make it perfect, considering all aspects of scholarly publishing!

By: Sofie Wennström, Managing Editor for books and journals at Stockholm University Press

peer review
Evidence-based policy after peer-review – Melbourne #MarchforScience on #Earthday 2017. Image from Flickr user Takver licensed with CC BY-SA 2.0

When it was decided in 2013 to found Stockholm University Press the Vice-Chancellor of Stockholm University made it clear that all books and journal articles had to be peer reviewed before publication. However, we soon learned that the practice of sending out book manuscripts for external review was far from standardised. The subsequent discussions with editors and authors on how we could manage such a process were many, and we still have to consider improvements.

The quality of the peer review process

Peer review is a standard with a purpose, but the process is not necessarily something that everyone is in agreement about. Included in guidelines for peer review according to the Committee on Publication Ethics (COPE) Core Practices are advice on how to work with transparency, timeliness, training for editors to manage the processes and structure for addressing misconduct such as descriptions of procedures to handle conflicts of interest and fraudulent behaviour. We have to ensure that all of these are included in the current practices at Stockholm University Press for both books and journals.

The critique against peer-review

Nevertheless, despite all available advice and best practices, there are objections to the assertion that peer review adds quality. Frequent comments claim that the process takes too long, that the feedback is biased, or not constructive enough. This is the reason why publishers have to be vigilant to address problems when they arise.

Other objections to the current practices claim that there are challenges about diversity and inclusion in editorial processes, as described in a blog post by Matt Hodgkinson from Hindawi, meaning that we need to ensure to include other perspectives when organising and managing peer review.  Working with books within the humanities and social sciences further emphasise the importance of diversity, as we have to ensure that the processes we put in place have an inclusive approach from all aspects of the academic spectrum.

User-informed work process at Stockholm University Press

The peer review process at Stockholm University Press has of course not been an exception from the rule, and we have received feedback from users indicating that our system is hard to use, that the feedback forms are not relevant or hard to understand and that the process is taking a long time.

sup-personer-EN
The editorial process as a collaborative unit with different actors, according to the Stockholm University Press structure. All of these actors add quality to the process. Image ©Stockholm University. Created by Karl Edqvist. Shared with a CC BY-SA 4.0 license.

This year’s focus for the Managing Editors working at Stockholm University Press has therefore been to collect user comments and to scrutinise the templates and forms to make them easier to use. We mapped our entire process for review of both book proposals and full manuscripts and found problems with instructions and feedback forms. Furthermore, we had to ensure that the structure of the process follows a standard suitable for evaluation of academic books such as:

  • For book proposals, we evaluate the relevance of the book to the intended audience, according to scope and focus.
  • For book manuscripts, we should provide simple feedback forms asking for relevant input about the book theme, the description of methods and data collection, and/or the relevance of the theories used. The book should also address current scientific problems relevant to the intended audience.

While checking forms and routines, we found that the feedback forms needed shortening and they were examined to remove inconsistent wording. Instructions were redesigned to encourage constructive comments and focused on the merit of each part of the review process.

What about the next steps?

We have received indications from our users that they find the process easier now, but can’t rest on our laurels yet. Constant improvements and monitoring of compliance with guidelines are required for us to maintain a high standard of quality assessments. Also, as our manuscript and review management system is still under development, we have to continue to carefully improve templates, instructions and processes maintaining a reliable service for authors, reviewers and editors.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s