Posted by Jonathan Adler:
Are Most Scientific Studies Sloppy?
http://volokh.com/archives/archive_2007_09_09-2007_09_15.shtml#1189863895
This week's [1]"Science Journal" column in the W$J (link for
subscribers) reports on the interesting and provocative research of
John Ioannidis, who argues that the results of most published
scientific studies are wrong.
Dr. Ioannidis is an epidemiologist who studies research methods at
the University of Ioannina School of Medicine in Greece and Tufts
University in Medford, Mass. In a series of influential analytical
reports, he has documented how, in thousands of peer-reviewed
research papers published every year, there may be so much less
than meets the eye.
These flawed findings, for the most part, stem not from fraud or
formal misconduct, but from more mundane misbehavior:
miscalculation, poor study design or self-serving data analysis.
"There is an increasing concern that in modern research, false
findings may be the majority or even the vast majority of published
research claims," Dr. Ioannidis said. "A new claim about a research
finding is more likely to be false than true." . . .
Statistically speaking, science suffers from an excess of
significance. Overeager researchers often tinker too much with the
statistical variables of their analysis to coax any meaningful
insight from their data sets. "People are messing around with the
data to find anything that seems significant, to show they have
found something that is new and unusual," Dr. Ioannidis said.
He further argues that only a fraction of incorrect studies are ever
corrected or retracted, meaning that there are many published studies
still "on the books" that support erroneous findings. It's an
interesting and provocative thesis.
References
1.
http://online.wsj.com/article/SB118972683557627104.html?mod=hps_us_editors_picks
_______________________________________________
Volokh mailing list
[email protected]
http://lists.powerblogs.com/cgi-bin/mailman/listinfo/volokh
