We started the biOverlay experiment with the goal of testing detailed, portable comments for preprints. One hope was that authors would have access to portable reviews that could be taken to journals. We would then be able to link journal editors with reviewers to help speed papers through the review process with the fewest wasted reviews (i.e., those that are written for journals where a manuscript is ultimately rejected).

Our promise to associate editors was that they’d have the opportunity to ask their peers for detailed thoughts on what they found to be the most interesting papers of the moment. We expected that preprints reviewed on biOverlay would be among the highest profile contributions within our various disciplines. Many preprints covered by biOverlay were published in numerous journals that seek to publish work of broad interest, suggesting that our associate editors were choosing papers that others found interesting as well.

We’re closing biOverlay now for a few reasons. First, it may have made it more difficult for authors to publish strong papers. We are aware of one case where reviews from biOverlay were used, but the authors, in a subsequent blog post, noted that they were then expected to respond to five reviewers. We would prefer that editors and authors take all comments under consideration and take a nuanced perspective to request that authors focus their efforts on the most salient ones. While some comments were overlapping, this didn’t make an apparently strong paper of great general interest more straightforward to publish. Second, without information as to the current submission/review status of a preprint, it was unclear if biOverlay reviews would be helpful. Third, without a clear impact on subsequent publication, potential reviewers lacked one incentive to accept our invitations. All of these factors were cited by potential reviewers when declining invitations.

We found that posting a preprint is insufficient to signal to potential reviewers that authors welcome feedback. New efforts, such as Review Commons from EMBO and ASAPbio, eLife’s preprint review trial, and PREreview’s live-streamed journal clubs provide author-initiated platforms to produce a similar outcome. Additionally, bioRxiv and other preprint servers could provide a way for authors to signal that they welcome feedback or report a manuscript’s submission/review status.

We want to take a moment to thank all of the associate editors who worked behind the scenes to make this experiment possible, the reviewers who contributed their time and expertise to the experiment, and most of all the authors of the preprints who shared their work before peer review.

Sincerely, The biOverlay Managing Editors

Stephanie Hicks - Assistant Professor, Biostatistics - Johns Hopkins Bloomberg School of Public Health

Zach Hensel - Head of Single Molecule Microbiology Laboratory - ITQB NOVA

Casey Greene - Associate Professor, Systems Pharmacology and Translational Therapeutics - Perelman School of Medicine, University of Pennsylvania