As much fun as it is to bash Nature, Science and Cell, the evidence that they publish poorer quality structures doesn't actually hold up well. Gerard Kleywegt (cited below) and I tried to use that supposition as the basis of a "positive control" for our case-controlled validation paper in Acta D, but we were surprised that once you account for the fact that the high-profile journals tend to publish papers on bigger structures that generally diffract to lower resolution, there's actually very little evidence that those structures are worse than comparable lower-resolution structures in lower-impact journals.
They probably do have more than their fair share of retractions -- but then it's hard to control for the varying level of scrutiny applied to papers published in different journals. In support of Bayesian reasoning, it's good to see that the data could over-rule our prior belief that Nature/Science/Cell structures would be worse! ----- Randy J. Read Department of Haematology, University of Cambridge Cambridge Institute for Medical Research Tel: +44 1223 336500 Wellcome Trust/MRC Building Fax: +44 1223 336827 Hills Road E-mail: rj...@cam.ac.uk Cambridge CB2 0XY, U.K. www-structmed.cimr.cam.ac.uk On 18 Oct 2012, at 19:31, Bernhard Rupp (Hofkristallrat a.D.) wrote: > One might include independent prior evidence (Kleywegt, Brown @ Ramaswami) > showing that in general most other quality indicators are worse for high > impact journals. > > So, as a frequentist I agree that his correlation is significantly weak, as > a Bayesian I say it is reasonably probable. > > Cheers, BR > > -----Original Message----- > From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Ethan > Merritt > Sent: Thursday, October 18, 2012 11:11 AM > To: CCP4BB@JISCMAIL.AC.UK > Subject: Re: [ccp4bb] PNAS on fraud > > On Thursday, October 18, 2012 10:52:48 am DUMAS Philippe (UDS) wrote: >> >> Le Jeudi 18 Octobre 2012 19:16 CEST, "Bernhard Rupp (Hofkristallrat a.D.)" > <hofkristall...@gmail.com> a écrit: >> >> I had a look to this PNAS paper by Fang et al. >> I am a bit surprised by their interpretation of their Fig. 3: >> they claim that here exists a highly signficant correlation between >> Impact factor and number of retractations. >> Personnaly, I would have concluded to a complete lack of correlation... >> Should I retract this judgment? > > Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. > While a correlation coefficient of less than 0.3 is not "a complete lack of > correlation", it's still rather weak. > > The "highly significant" must be taken in a purely statistical sense. > That is, it doesn't mean "the measures are highly correlated", it means "the > evidence for non-zero correlation is very strong". > > Ethan
