Let’s Dissect Post-Publication Peer Review

---

This is the first of a two-part series on post-publication review. The second article, also by by Andy Tattersall, will present an overview of websites that facilitate post-publication review.

The idea of public post-publication peer review of journal articles would have been considered heresy just a couple of years ago, but in recent years there has been a growth in online post-publication reviewing options from publishers and others. The following blog post looks at a few of them and discusses the idea of leaving post-publication reviews and whether it’s all as bad as some in the academy community seem to think.

The journal publishing model has been quite consistent for many years now: authors submit their paper to a peer-reviewed journal, reviewers read and analyse it and give feedback, and depending on how good or bad that feedback is the paper is eventually accepted or dismissed for a variety of reasons, which may lead the authors to attempt a resubmission to another journal. For the papers that do get published, the cycle of conversation between peers ends here, unless the research is discussed at an event, such as a conference or appears in the media, where comments can be easily left. Comments on these more informal platforms are sometimes scathing, unjustified or plain unhelpful. And there is the rub, when does post-publication peer review become post-publication comment and how different are they?

Unlike many other things that appear on the Web, such as music, film, art and review, where comments and critique is normal and helps others make better informed judgments on what to consume, academic articles only appear above the surface for review and critique when it is used as part of someone’s research or teaching. Often there is no critique, good or bad of the research post-publication, just an important line, conclusion or otherwise to help build the hypothesis of new research; agreeing with or against previous works. But to say there  has been no post-publication review system previously would be untrue as the likes of BMJ editors and others have accepted e-letters and Rapid Responses about research published in their journals, and blogging and social media have more recently offered platforms for researchers to discuss other’s works.

From my experience many researchers feel uncomfortable on speaking about other people’s research, which is understandable. Nobody wants to hear ill of their own hard work, as this is what they are potentially opening themselves up to. Take for example the reviews on such as YouTube or the Daily Mail website, often comments can become personal, malicious and quite damning. The issue is that everyone has an opinion and that can be on everything from Syria to fracking, and the Web has facilitated that opinion culture to the point where ‘trolling’ is now an acknowledged and serious problem. Yet academic publishing is different, certainly academics are more than culpable for their barbed comments, but making unjustified ones online will help no one, especially in the advancement of knowledge via discourse.

The journal publishing model has been criticised for being out of touch with modern publishing, and rightly so in that a piece of research which can take years to complete can then take nearly as long to get published. So by that time things may have moved on in that topic of research, new methods, technologies and ideas may have surfaced. Post-publication reviews can help highlight this, and also may make researchers aware of potential future collaborators or similar research being undertaken.

For post-publication review to really be productive it has to be open, unlike sites such as YouTube which has allowed aliases and therefore trolls to flourish. Obviously not every piece of research commands a post-publication review and given the figures which range from about 12% to 90% of papers not being cited, it is pretty likely that not all papers will get reviews or at least have the mechanisms to be reviewed. We also have to remember that while some areas of research are less reliant on the journal publishing model, this does not mean post-publication review is not for them. In disciplines such as the humanities it may have just as much use.

There seems to be some difference between the notion of reviewing, discussing and commenting, something Kent Anderson in The Scholarly Kitchen wrote about earlier this year. With Anderson summerising that; “Today’s commentators seem to have many axes to grind. Far too often, commentary forums degrade into polemical attacks with win or lose dynamics at their heart. The pursuit of knowledge and science isn’t the goal. Capitulation of one combatant to another is.” Anderson questioned the validity of comments being championed by publications and websites and that they could never be considered in the same light as peer-review.

There is a need for both as comments can be insightful and highlight or spot useful content for the original authors or other readers without going into indepth reviews. On the flipside they can be malicious, unfounded and just clog up the whole knowledge process if left un-moderated and anonymous. Peer-review may not be perfect, but as the social web becomes more useful as a platform for discussion and knowledge sharing, it makes sense that other options are explored, even if they are supplementary. This is similar to the growing case for Altmetrics, first seen as an alternative to the traditional measurement of citations and now argued as more of an alternative indicator, rather than measurement. The real problem is that like many other technologies and platforms for communication we run the risk of not being able to see the wood for the trees. Post publication review platforms need to be explicit in their aims and explain that clearly to readers and reviewers. Like social media, it is unlikely that we will see every researcher using these unless they became standardised and part of the research cycle. It is an option, as with academic discussion lists, where the most insightful and on occasion barbed communications take place. Post-review commenting is happening right now and someone out there may have already commented on your research – whether you respond remains your choice.

---