Last year’s Peer Review Week proved to be a great success in raising awareness and starting discussions about peer review. This year, it’s back and the focus is on recognition for review.

There have been lots of surveys looking at perceptions of peer review. These surveys agree that peer review is valued and authors feel that the quality of their paper improves as a result. Nature’s annual author survey shows that after the reputation of the journal and relevance to the discipline, the quality of peer review was the third most important factor driving author’s choice of where to submit their article.

For Peer Review Week 2016, I thought I’d take another look at these surveys to see what they tell us about recognition for reviewing activity. I’m concentrating on three big surveys that were carried out in 2015 by Wiley, Taylor and Francis (T&F), and the Publishing Research Consortium (PRC). Sense about Science also conducted a survey in 2009.

Reviewers are motivated by a sense of contributing to the community

Top motivations for reviewers stemmed from a desire or sense of responsibility to actively contribute to the academic community. The two primary reasons given by reviewers in both the PRC and T&F surveys were playing their part as a member of the community, and reciprocating work by others, while seeing research ahead of publication (PRC) or improving papers (T&F) came third.

The Wiley survey also gave interesting insights when reviewers were asked about their reasons for accepting a specific invitation. The reputation of the journal was the primary factor determining whether they accepted a review invitation. Journal reputation also influenced the amount of time spent on the review, as well as their commitment to meeting the review deadline. Reviewers also said they valued the networking opportunity with the inviting editor. All this suggests that reviewers perceive a greater benefit when they review for a top-ranked journal.

The desire to see improvements is increasing

While peer review remains widely supported, with 82% of respondents to the PRC survey agreeing that “without peer review there is no control in scientific communication”, the number of people who want a thorough shake-up of the current peer review system has increased – the proportion of respondents who disagreed with the need for a complete overhaul of the peer review process fell from 35% in 2007 to 26% in 2015. They also received a very mixed response to “The current peer review system is the best we can achieve”.

There is an untapped pool of willing reviewers

The proportion of respondents to the PRC survey who see peer review as unsustainable because there are too few willing reviewers has risen by 9% since the Sense about Science 2009 survey. But contrast this with the fact that over two-thirds of authors in the T&F survey who had never peer reviewed said that they would like to, and we see a disconnect between perceived difficulties with finding reviewers and the number of willing reviewers. There could be many reasons for this, perhaps there’s a mismatch between the subject areas of willing reviewers and the papers seeking reviews? Or perhaps we need to find better ways of bringing new willing reviewers into the peer-review process very soon after they start to publish papers. This probably warrants a blog post of its own and I’d love to hear any suggestions for how to achieve this.

The Wiley survey demonstrates that reviewers with more than five years’ experience are shouldering more of the reviewing burden. Comparing the geographic distribution of reviewers to published articles shows that the US is disproportionately burdened and reviews 33% of life sciences papers while publishing 22%, whereas researchers in China are currently publishing twice as much as they are reviewing.

More training would be welcomed

There is clearly a demand for more training for peer reviewers. Over three quarters of reviewers surveyed by Wiley said they would like to receive more training on peer review. When asked how they started reviewing, 65% of reviewers were invited after publishing a paper, while 22% were invited to review as a result of a recommendation from their supervisor. Very few reviewers (12%) had received any kind of formal training. Over a third of reviewers in STM subjects in the T&F survey had been mentored by their supervisor when reviewing a paper and another third hadn’t but would like to. All the BES journals encourage this practice and I really recommend reading this post from two MEE Associate Editors.

Societies such as the BES are in a good position to provide training, and our Guides to better Science series was first launched with a guide to peer review in ecology and conservation. Over the past few years I’ve been involved in various workshops on peer review and publishing (find out more about the BES publications team panel discussion on different models of peer review in December) there is an appetite for more training.  As there is no need to undertake formal training before reviewing, this seems to imply that reviewers should inherently know what to do. Students and new reviewers are therefore concerned about asking questions in case they are seen as too basic or obvious.  I remember one workshop in particular when we polled participants on their level of experience with reviewing – after this the discussion really got going as everyone realised that many in the room had never reviewed or had only reviewed once so participants became less inhibited about asking questions. Workshops or mentoring options aren’t always available but there are plenty of great sources of advice online. Check out the Methods blog on tips and tricks for peer review (along with what not to do!).

Incentives and rewards – there are mixed feelings about payment and direct recognition

Reviewers felt strongly that reviewing activity should be acknowledged as a measurable output (Wiley survey). Consistent with the main community-based motivations for reviewing, the top reward or incentives favoured by reviewers were focussed around receiving feedback from the journal on the quality of their review and the decision reached for the paper, as well as receiving copies of the other review reports. A certificate acknowledging the review effort and acknowledgements published in the journal were also rated highly.

On the whole, reviewers seem uncomfortable about receiving direct monetary rewards. T&F found no consensus on whether payment for review would incentivise peer reviewers, but reviewers in STM were incentivised by receiving free access to the journal, waiving colour figure or open access article processing charges. Sense about Science drew a similar conclusion from their 2009 survey, which found that reviewers preferred discounts or waivers for publishing costs rather than direct payment. Reviewers in more mature markets (USA, UK, Germany, Australia, South Korea and Japan) showed a higher preference than high-growth markets (China, India and Brazil) for discounts or waivers on Open Access fees in the Wiley survey. However, high-growth markets ranked free access to papers or a digital badge to display on personal websites over acknowledgements in the journal. Reviewers in the T&F survey favoured appearing in a published list of reviewers. The survey also noted that some respondents welcomed recognition when not directly linked to a specific paper.

Overall, the importance that researchers place on quality peer review remains the same, but the demands for training, improvements to the process and formal recognition of this important professional service is growing, so this is a very worthy focus for the 2^nd global Peer Review Week.

Erika Newton, Managing Editor
@ErikaLNewton

Surveys

Publishing research consortium (2015) http://publishingresearchconsortium.com/index.php/prc-projects/peer-review-survey-2015

Taylor and Francis http://authorservices.taylorandfrancis.com/peer-review-global-view/

Nature (2015) https://figshare.com/articles/Author_Insights_2015_survey/1425362

Wiley (2015) http://olabout.wiley.com/WileyCDA/Section/id-828031.html

Sense about Science (2009) http://www.senseaboutscience.org/pages/peer-review-survey-2009.html