Anonymous complaints: the peer review story

Ok, the experiments are done, the journal is picked out and the manuscript is written. It’s time to make a sacrifice to whichever science god you follow, press the submit button on the online submission form and wait… and wait, and wait some more. Most will take close to a month, but my experience has been closer to 2 or 3 months. And that’s just for the first round of reviews!

Image Credit: Nick at http://www.lab-initio.com/

This may feel like forever, but peer review really does end up improving your paper. Plus, the alternative is a fast rejection letter! The editor is the first obstacle to pass, as he/she quickly screens incoming manuscripts. This is to decide if the article suits the journal’s focus and is of high enough quality (mostly the writing, but somewhat the science) to make it worth taking up the time of busy reviewers. If you send bad papers to scientists too often, they’re going to stop agreeing to do it.

If the article passes these minimum standards, it typically gets sent to 2 scientists in that field of study and they’ll have around 3 weeks to submit their opinions. Between the pre-screening, finding scientists willing to do the review, waiting for those (often late) reviews, and making an editorial decision based on them, it’s easy to see how this step can take some time.

To speed it up a bit, some journals ask for reviewer recommendations from the authors rather than having to search for experts. Otherwise, the editor will select people based on who’s publishing in that field (such as those cited in your article) or people they know from conferences. To be honest, I’ve never been involved in the editor side of things, so I’m getting most of this information from online resources.

What I do have a bit of first hand experience with being a reviewer. I’m lucky enough to have had a great PhD supervisor who recommended me to a low impact journal for a review that he wasn’t able to do. From there, I was asked to review a few other articles, bringing me to a grand total of… 3. Officially. I’ve also had a hand in a few others for bigger journals, since one of the training aspects of a postdoc is to help your supervisor with peer review. But, obvious disclaimer, I’m certainly no expert yet.

The peer review itself consists of just a few sections. First, a quick summary of the paper, including the overarching conclusions, the study’s strengths, any generalized weaknesses and whether it contributes to its field. Not too detailed, that’s for the next section: major edits. Here, it helps to have the criticisms numbered, as the authors are going to have to respond to them point by point. These should cover anything that makes it hard for you to believe their conclusions, read the paper or understand how experiments were done. I won’t go into details, as there are plenty of websites already discussing how to write peer reviews.

Next up is the third section, which is for minor edits. This is where you get all your nit picking out. Yes, it is hard to understand their data when an axis is mislabeled, but is that really a major concern? I’ve actually had one reviewer tell me as a major critique that there wasn’t enough space between the graphs in a figure. Ignoring that graph spacing can be fixed during the page layout phase before publication, is that honestly a make-or-break issue?

So I would hope it goes without saying, but don’t be a jerk, no matter how tempting it is. While in theory most reviews are single blind (ie: the authors don’t know who the reviewers are, but the reviewers can see the authors listed), it’s often all too easy to figure out who is picking apart your work. Maybe they’ll insist you add a reference, that just happens to be theirs. Or sometimes they’ll be critical of a widely accepted conclusion/assumption/method in the review, but are also publicly vocal about that opinion. Scientific fields tend to be quite small and nichey (Yes, that’s a word. Or at least I’m declaring it is now!), so hiding a mean review behind anonymity isn’t a good idea. Especially considering that the editor still knows who you are!

The last part of the peer review is the recommendation, which is confidential to the editor. This is just a couple of quick sentences summarizing the strengths and weaknesses, followed by what category it falls into:

Accept
Accept with minor revisions
Accept with major revisions
Revise and resubmit (Also called reject with hope, which I think sounds sort of mean. “Sorry Sally, I’m rejecting your offer of a date. But maybe if you become prettier in the next week, I’ll revise my opinion!”)
Reject

#1 is almost unheard of, because it means that it will get publishes completely as is. And really, no one’s that perfect. The next 2 are more common and require a response to the reviewers. I was going to go over that in this blog piece, but apparently I’m too long winded! So I’ll cover that next time.

#2 tend to not go back to the reviewers and can be decided by the editor. But #3 and 4 have to go through another round of peer review with the same reviewers, which means another month or more. You can see how the time adds up quickly! And if you’re racing with another lab to publish your data first, every week matters. In fact, there are stories of horrible people using this delay to their advantage and stealing an idea.

Lastly, there’s #5: full on rejection. The journal will not publish this study, no matter how much you fix it up. No need to say more. Just move on to another journal.

After the peer reviews are submitted, the editor steps in again and ultimately decides whether the manuscript will get accepted or rejected. But sometimes the reviews are polar opposites of each other (which happens more than you’d think in a supposedly objective discipline), and the editor can’t decide what to do. So they bring in the big guns: reviewer #3 (warning: foul, hilarious language).

But that’s my time, folks. Or rather, your time… Either way, this sucker is long enough! Next time, on Behind Lab Doors, can Dr. Author respond to his reviewers without pissing them off? How will Dr. Scientist react when she finds out that peer review fraud rings exist? Stay tuned!