I am puzzled by writer Stan Moore's query, "These are questions that enter my mind after perceiving many failures in peer review while reading the scientific literature over the years."
I have several key questions: 1. Which scientific literature has been read, and was being referred to? Is this the scientific discipline in which the writer was trained? 2. What was the nature of the perceived failures of review? Was he concerned by verifiable errors of fact? Clear errors in data analysis, statistical or otherwise? Clear errors in interpretation of the data? Were the paper's conclusions suspect, and if so, why? 3. I have no idea what is meant by the writer's query about "ultimate and subultimate goals." Has the writer himself served as a referee? After almost three decades of serving either as an Associate Editor (AE) or as reviewer for more than a dozen different journals, both in ecology as well as in general science (Science, Nature, PNAS), I have never once considered the review process to be a superficial exercise, and I certainly never considered it to be primarily a spell-checking and grammar review exercise. On the contrary, I view the role of reviewer as a critical component of the scientific process: he or she is expected by the journal in question to accept or reject manuscripts based upon a number of key criteria, including (but not limited to) suitability for the journal in question; the methodologies used; scientific soundness; consistency with existing knowledge; the quality of exposition; and the degree to which the authors' conclusions are supported by the data which were presented and analyzed in the manuscript being reviewed. As an AE, I always chose reviewers who were considered to be established, knowledgeable experts in the research topic covered by the manuscript being reviewed. As a reviewer, I will not accept an invitation to review any manuscript written by a close personal friend or close collaborator. Please remember, this is objective science we are talking about, and not subjective politics. In dealing with the general public, for example, I try to emphasize that scientific papers are not like articles in Time magazine. Opinions don't count: hypothesis testing, the use of rigorous and up-to-date scientific methodologies, and the collection of independently verifiable data are essential currencies of legitimate scientific inquiry. Fraud and deliberate manipulations or distortions of the data will ultimately be identified and revealed as falsehoods, and will be rejected by the scientific community. I urge the writer to view the Instructions for Reviewers of ESA journals at http://esapubs.org/esapubs/reviewers.htm, which I hope will help address many of his questions and concerns; similar guidelines for reviewers are available on line for many other scientific journals. As emphasized by the ESA guidelines, objective, high quality peer reviews are essential for insuring the quality of scholarly journals -- and thus for insuring the advancement of scientific knowledge. Val H. Smith Professor Department of Ecology and Evolutionary Biology University of Kansas Lawrence, KS 66045 At 03:53 PM 7/9/2007, Ben Bond-Lamberty wrote: >On 7/9/07 9:53 AM, "stan moore" <[EMAIL PROTECTED]> wrote: > > > What is the minimal obligation of any peer reviewer for any scientific > > publication? For instance -- must the peer reviewer read all the reference > > materials? >No: the time commitment could be enormous. But presumably the reviewer is >someone in the field, and so will be familiar with most of the references. >I would say the "minimal obligation" is to read and professionally evaluate >the ms, while disclosing any conflicts of interest, within the time period >requested by the journal. > > > How much time and effort should go into any peer review? >I expect, on average, to spend 2-5 hours on a ms, including writing the >evaluation. > > > > What should be the ultimate and subultimate goals of the reviewer? >?? > > > > For instance, is the overall objective of the paper under > review subject > > to question? >Sure. Perhaps it isn't appropriate for the journal, for instance. > > > > Is peer review more superficial and mostly a spell-checking and grammar > > review excercise? >Not in my experience. > > > > On what basis should peer reviewers be chosen -- species expertise, > > technical application of scientific method? >Either or both. Ideally a ms is read by many reviewers who bring multiple >points of view to bear. > > > > Does reviewer acquaintance, friendship, collaboration with author(s) > > constitute bias? >Yes, and thus it should be disclosed. But it doesn't necessarily disqualify >the reviewer. > > > > How should obvious errors in peer review after publication be dealt with? >Can you give a more concrete example? > > > These are questions that enter my mind after perceiving many failures in > > peer review while reading the scientific literature over the years. >I'd be interested in what for you constitutes a "failure" of peer review. >Such failures certainly exist, but the most notorious cases (Henrik Schon, >for instance) involve fabrication of data. Peer review can't be expected to >catch this: reviewers and editors must assume basic honesty on the author's >part. > >Ben Val H. Smith Professor Department of Ecology and Evolutionary Biology 6007 Haworth Hall 1200 Sunnyside Avenue University of Kansas Lawrence, KS 66047 phone: 785-864-4565 FAX: 785-864-5321 e-mail: [EMAIL PROTECTED]
