Last year (on Dec 11, 2000) I posted a message (on the subject: Re ClinMed NetPrints) about the publication rate for eprints posted at the ClinMed NetPrints website (http://clinmed.netprints.org/home.dtl)
As of Dec. 10, 2000, I estimated that about 25% of the eprints posted at this website had completed the entire publication process, and had subsequently appeared in a peer-reviewed journal. As of May 20, 2001, 45 eprints have been posted at the ClinMed NetPrints website. Of these, 19 were posted before the end of May, 2000 (i.e. about a year or more ago). Of the 19, I could identify 5 that had subsequently been published in peer-reviewed journals (a publication rate of 26%). I also checked, via PubMed and the ISI Citation Databases, for other publications by the author(s) of the 45 eprints. For only 8 of the 45 was I unable to find any previous publications by the author (nor by any of the co-authors, for multi-authored research reports). So, for 37/45 of the eprints (82%), one or more of the authors had some track record of prior publications in journals listed in PubMed, or in the ISI Citation Databases (in an attempt to be thorough, I included all three of these databases, not just the ISI Science Citation Index). Although track record is certainly not a highly reliable indicator of the quality of a research report, neither is success in peer review (especially in low-impact journals!). However, I'm especially interested in ways to assess the quality of an eprint archive as a whole, not just the quality of individual eprints. As in my previous message, I continue to ask: what criteria should be used to assess the quality of an eprint archive? I've seen sets of criteria for use in the evaluation of eHealth websites (see, for example, the an Information Quality Tool based on The Health Summit Working Group's "Criteria for Assessing the Quality of Health Information on the Internet" (http://hitiweb.mitretek.org/iq/default.asp), but I haven't seen an analogous set of criteria for use in the evaluation of eprint archives (and, especially, ones related to health research, such as the ClinMed NetPrints archive, and The Lancet's eResearch Archive, which, at present, contains only a few research reports; see: http://www.thelancet.com/era/epstatus). In a contribution to another thread (Re: Evaluation of preprint/postprint servers, in a message posted on 15 Dec 2000), Greg Kuperberg suggested that a main criterion in evaluating an archive should be "its suitability as part of the envisioned universal archive". Another criterion (it seems to me) should be its suitability for obtaining citation data. An example, based on the arXiv archive, is provided by the Cite-Base search service (http://cite-base.ecs.soton.ac.uk/cgi-bin/search) which (I gather) is based on OpCit citation data, and allows users to rank searches for reports in arXiv by citation impact or by hits (see The Open Citation Project, http://opcit.eprints.org/). Does anyone know of a web-accessible set of criteria for use in evaluating eprint archives? Jim Till University of Toronto
