Fitting with our theme of #DiversityinEcology the theme of peer review week 2018 is diversity in peer review. To celebrate we decided to ask the people at the coalface of delivering the peer review process, our Associate Editors, why diversity in peer review is so important. At JAE our committed team of Associate Editors are responsible for selecting and inviting reviewers and subsequently evaluating the reports and delivering a recommendation to our Senior Editors to make a decision.
To introduce the post Senior Editor Nate Sanders gives his thoughts on the importance of diversity in peer review.
I’m a Senior Editor at Journal of Animal Ecology. And I’m biased. I have to admit, those are hard words for me to type. However, of course, we are all biased – authors, reviewers, Associate Editors, Senior Editors, and readers. As editors, we rarely see explicit bias by reviewers or Associate Editors. More common is implicit bias (or unconscious bias), if you think you don’t have our own implicit biases, go here and take the test or take a look at this great video from the Royal Society which we encourage everybody to watch including our Associate Editors.
What are my biases as a Senior Editor, and how do they manifest themselves? Honestly, I hope they rarely do. One way I try to ensure my biases, whatever they are, are kept in check is to hear from diverse voices about manuscripts that come across my digital desktop. That diversity of voices, mostly in our ever-diversifying editorial board, provides perspectives and insights that I might not have had. And that’s an important point, actually: a growing number of studies indicate that diverse teams are often better at solving complex problems than less diverse teams. You might have noticed that in your own collaborative work. The same should be true for journals: with a diverse editorial board and a diverse pool of peer reviewers, we should be able to better solve complicated tasks, like publishing the very best research on the ecology of animals. Moreover, having a diverse editorial board should help to attract a diversity of authors (and for the record, I’m using ‘diverse’ and ‘diversity’ to mean all dimensions of diversity). I’m proud of what Journal of Animal Ecology has been able to do over the past several years to increase diversity on the editorial board such through the open call for new Associate Editors (but we can and will do better). I am equally proud that we are attracting authors at various career stages (such as through the Sidnie Manton Award for Early Career Researchers, among other avenues) as well as attracting authors from different parts of the world (but again, we can and will do better). It’s also great that the BES journals now encourage collaborative peer review to enable ECRs to review their first manuscripts.
So, happy peer review week, all. Until next time, think about your own implicit biases and how to overcome them. And think about how Journal of Animal Ecology can do a better job of publishing the best research on the ecology of animals, by a diversity of authors with a diversity of perspectives.
Is diversity in peer review important?
The majority of our interviewed AEs agree that diversity in peer review is an essential part of the academic publishing process. Laura Prugh says “If the aim of publishing scientific findings is to reach a diverse audience, then it makes sense to have an equally diverse pool of peer reviewers”. She also believes that having a variety of perspectives may help reduce bias in peer review. Niels Dingemanse agrees. “This is the best way to acquire unbiased estimates of the average opinion in the population,” he says.
As well as reducing bias, diversity in peer review also allows a range of opinions and experiences to be included. “Taking opinions from people from different genders, institution types, scientific backgrounds, geographical location, career stage, and scientific approach means that a manuscript gets exposed to a greater range of critical assessment,” says Chris Harrod. “Most of us involved in peer review are positive (apart from the dreaded Reviewer 3!) and want to see work published, but we have different experiences. This means that the manuscript as originally submitted can convey different things to different people, and it’s important to make sure we maximise the diversity of those assessing the work and making suggestions for its improvement.” Mariano Rodiguez-Cabal agrees. “Peer review is fundamental for science, and diversity is fundamental for peer review,” he says. “Persons from different background approach things in different ways, and can give different advice during the review process.”
Not everyone agrees about the need for such diverse opinions. “For the quality of the paper, I think that diversity is mostly unimportant,” says Peter Hamback. “However, to get more people involved in the review process, it is important.”
But Mariano points out that it is not just diversity amongst reviewers that we should be working to improve. “Diversity must be applied in the greatest extent of its definition, and throughout the review process,” he says. “We do not only need a greater diversity in the reviewer community, but we also need more diversity at the Editor-in-Chief level and at the Associate Editors level. These are the first filters for any manuscript. Thus, we need more diversity at those levels to increase the fairness in the review process.” Mariano hastens to add that this is not because of people in such positions being deliberately unfair, but because there is low diversity at these levels. “So they just think alike,” he says.
How could the peer review system be changed to improve diversity?
So we agree that a diversity of peer reviewers is important – but the challenge is making it happen. Luckily, our AEs have some suggestions.
Firstly, establishing the scale of the challenge is important. “Some investment in studies that examine whether diversity does improve the peer review process might be useful,” says Chris. “Alternatively, if these already exist and I’m ignorant, then they should be highlighted,” Laura notes that multiple studies have already highlighted that implicit bias affects peer review. “Established male authors with Western last names tend to receive more favourable reviews than early-career, female, and authors with last names not of European origin,” she says.
As a result of these implicit biases, Laura is strongly in favour of double-blind peer review. “I feel the arguments against double-blind review do not outweigh the benefits,” she says. “It is the standard in most other fields of science and is used by some journals in our field, such as the American Naturalist. I recommend that the Journal of Animal Ecology switches to double-blind review.”
However, implicit bias can also affect other parts of the peer-review process. “The largest problem as an AE is to figure out who will be a good reviewer and one that will respond,” says Peter. “The common way to do this is either the suggestions made by the administrative system, by looking in reference lists, or by searching for relevant articles. All of these have biases.”
Mariano suggests that one solution could be to include authors in the act of reviewer selection. “I would request the authors to recommend five potential reviewers from different backgrounds (country, sex, young and established researchers, etc), and the system should not allow adding more than two reviewers from the same country, and at least one reviewer should be from a developing country,” he says. Mariano is also keen to better involve students and ECRs. “I would also propose that each journal have a program to have PhD students and postdocs reviewing papers with the help of their advisors,” he says. “The journals could invite a professor or researcher to evaluate a manuscript and given the option to co-review it with one of their students.” Mariano thinks this would not only increase diversity in peer review but also help teach young scientists about the review process and the way science is conducted.
Some ECRs are already taking the matter into their own hands, by trying to make themselves more available to journals and editors. “Via Twitter, I am aware of the Early Career Reviewer Database,” says Chris. “Interested ECRs can upload information regarding their expertise and make themselves available for peer review, allowing AEs access to people who are keen to review – as opposed to the superstar authors churning out 20 papers a year who are too busy to review other people’s work!”
What can we change now?
Our interviewed AEs are committed to improving diversity in peer review over the coming year.
Laura will be focusing on addressing her own implicit bias over the coming month. “Although I am aware of implicit bias, I realise that I am as susceptible to it as anyone else,” says Laura. “I will keep this at the forefront of my mind when reviewing papers, and try not to let the identity of the authors bias my review.” Meanwhile, Peter will turn his attention to geographic diversity. “I will try to invite some more non-European and non-American researchers,” he says.
Niels and Mariano will be focusing on improving diversity in general. “I will think much more actively about these issues when making decisions,” says Niels. Mariano agrees “As always, I will try to get the most diverse reviewers possible to evaluate a manuscript.”
Chris will be focusing on diversity both as an AE and as a reviewer. “I will continue to make sure that my requests to review are sent to a body of expert reviewers that represent our science across the wide diversity of gender, career stage, location, scientific approach, and institution,” he says. “When I cannot review a paper, I will also take this into consideration and suggest other reviewers in the same way.”