Return to sender
Doesn’t it feel like an eternity since I started blogging about the peer review process? Guess what, it’s actually a fairly accurate of a timeline for the process! It’s long and drawn out, but that’s not necessarily a bad thing. Rush jobs are rarely done well, and in this case, can even be a sign of fraud!
Hearing of a researcher who faked data is unfortunately too common, but a pretty new concept (to me at least) is the idea of a fraudulent review. Rather than stand up to the scrutiny of your peers, some people have taken to writing their own reviews using fake names or false email addresses under a real scientist's name. The major clue that this deceit was happening was a quick response time (less than a day, sometimes even within hours) resulting in a (surprise!) glowing review. Considering half the battle for an editor is finding researchers willing to fit a peer review into their busy schedules, this seems really obvious. So ultimately, the fault of these fake reviews getting through the system lies at the feet of the editors in my opinion. No matter how busy you are, take the time to do your job properly.
The good news is that journals are becoming savvy to this fraud and are cracking down. Last year, a "peer review and citation ring" was discovered by the Journal of Vibration and Control and 60 articles were retracted. Currently, a major publishing company, BioMed Central, is in the process of retracting 43 papers due to fabricated reviews. This is crazy stuff, considering how uncommon it is to hear of even a single paper being retracted.
But back to what peer review looks like for the average, honest, non-horrible scientist. Assuming you have actual reviewers on your paper, you’re almost certainly going to need to make changes to the manuscript. The reviews typically come back as a brief summary of your paper, followed by a list of major issues to be addressed, then any minor edits. Each of these points need to be dealt with in the manuscript itself, then a Response to Reviewers must be written that addresses each point individually. This is where you can argue with their critiques, if you want. Sometimes it’s worth it! But not usually, and even then it’s a give-and-take. “I’ll give you the citation of what’s obviously your paper, but I’m taking the liberty of not doing the suggested long and boring experiment that won’t add anything to my conclusions.” That sort of thing.
Keep in mind that reviewers can pretty much ask you for anything, including additional experiments to further support your conclusions. In fact, it’s cause for celebration if a review comes back without extra experiments suggested! Editing is way easier (and quicker!) than having to do more bench work. It’s also occasionally acceptable to say, “Nope, I’m not doing all that work for this lower impact journal!” Ok, well, you may want to be more diplomatic. I’d recommend, “While the suggested experiments would likely generate interesting results, they are beyond the scope of this paper.”
It’s a delicate balance between refusing to do any extra work and agreeing to do everything. Refusal will often result in a rejection after the first round of revisions, which most certainly can happen. Yet agreement won’t necessarily get you published either. Sometimes Dr. Grumpy Reviewer will just find more roadblocks to throw in front of you until one of you capitulates. But I like to be optimistic about the nature of people. I truly think that peer review, for the most part, improves a paper and leads to better science. The critiques aren’t personal attacks, even though they can often feel like it.
My own exciting news is that yesterday I got an acceptance letter for a paper I wrote in my previous lab. Hurray! … after almost 6 months and 3 rounds of reviews. No joke. It’s quite good timing actually, as it demonstrates what I’m talking about in this post. No experiments were suggested, which was good since I’m literally a continent away from my samples! I’m sure other people in the lab would have been able to help out in exchange for a middle authorship, as I’ve done for others, but it was still a lucky break. Plus, reviewer #2 was quite happy with the work and only had 3 small comments. However, reviewer #1 was another story. Here’s a generalized, considerably paraphrased account of what happened. There were other critiques, but this was the main one.
“Why did you use these 2 tests for the basis of your paper? They’re commonly used for another process entirely, and there are other standardized tests you could have used instead. Moreover, the procedures are not explained in the text and the 2 tests were incorrectly administered. Test #1 usually requires [insert all kinds of technical details here].”
“While less common, test #2 has been characterized by several groups for this function when administered as we did. The manuscript has been updated to specify these differences (pg 5 and 8). *List of 4 supporting paper references* Test #1 is indeed conventionally used to study that other process. However, as the regular way didn’t work in our situation, we modified the data collection to also include this process. This is explained on pg 7-8. In separate, unpublished trials, we attempted to use those other tests. However, again this didn’t work in our situation. Therefore, we could not use any of the data collected and we did not pursue these tests further.”
“Regarding the first part of test #1, the authors do not mention any time they did this other part of the procedure. If that was correct, the test was invalid (see this paper reference). I am not convinced at all that this process can be accurately measured with test #1. By contrast, test #2 was correctly administered and I agree with the authors’ interpretation.”
“Test #1 did not include that part of the procedure. This is what we did instead. Details about this have been added to the methods section.”
We wore him/her down! But honestly, it improved my work. If our methods weren’t perfectly clear to a reader, I didn’t do my job properly. Especially if that means they doubted my conclusions, because that is the name of the game folks: Convince the audience you know what you’re doing!