On Wed, Oct 9, 2013 at 6:56 PM, Marco Lolicato <chimbio...@gmail.com> wrote:
> Anyway, for those reasons and more, I was wondering if maybe is nowadays > needed to revisit the peer-review process. > Apologies for the lengthy response, but I really do think the current publication system is broken, just not for the same reasons as others. One of the interesting suggestions put forth by Michael Eisen (PLoS co-founder, and author of the previously-linked rant), among others, is that "post-publication peer review" should become more widely used and partially substitute for the current system. This has always been done informally at conferences, journal clubs, ccp4bb emails, and so on, but this is all very unstructured and not always public. The only formal outlets are by writing a letter to the editor of a journal, which is a very time-consuming process, or by writing a more thorough follow-up article dissecting the problems. There are some advantages to this - the discovery of fraudulent structures would probably not have been as widely noticed if the analysis took the form of blog comments. However, the overhead (in time and effort) is so massive as to deter all but the most determined scientists. At a minimum, I'd like to see a more structured but very lightweight way to discuss *and track* problems with the literature, ideally at the source(s). For instance, if I go to this PDB entry, there is absolutely no indication of anything suspicious: http://www.rcsb.org/pdb/explore/explore.do?structureId=2hr0 If I follow the publication links, I can see that there is a "brief communication arising" associated with the article: http://www.nature.com/nature/journal/v444/n7116/full/nature05258.html but since Nature's editors watered down the letter and accepted a response that did nothing to address any of the questions, it's difficult for an non-expert to reach any conclusions. Few of the multiple derived databases have anything either; Proteopedia is the big exception. One has to do a surprising amount of digging to find out that the senior author's university publicly disclaimed the structure as fraudulent. There is also a very large volume of comment on the ccp4bb, including some excellent specific (illustrated!) examples of problems with the structure. But none of this is centrally available, because the journal (and databases) do not provide any mechanism other than the lengthy formal route. It's true that a little Googling will quickly uncover problems with this particular paper. However, from what I've seen there are depressingly many scientists who are unable to use Google even for the questions they already know to ask. And this is an exceptional case; there are many other problematic structures (anyone working in methods development probably has a long list) for which no such information is available, because we don't have time to write a formal letter to the journal editors, especially since there's no guarantee that they'll even pay attention to us. It would be far more efficient if I could simply post a comment at the source saying "ligand density does not support binding - see attached image". In the long term, if there was a better system for this kind of peer review, the current system could mostly go away. Post a manuscript on arXiv (or equivalent), let the community comment on it and rate it, and the eventual consensus determines its credibility and importance. Scientists would stop wasting months tailoring the paper to fit into arbitrary and obsolete length restrictions or to impress the editors of high-profile journals or please a handful of anonymous reviewers. There would be no disincentive to publish negative results or brief technical write-ups. Both publication and review would be immediate, inexpensive, and public. The scientific literature would become truly self-correcting over any time scale. Undoubtedly there are issues with this, and I'm sure there are other approaches that could work too. But the present system is both horribly inefficient and too permissive of outright junk, and I think it's really holding us back. -Nat
