On Sun, 01 Apr 2012 15:29:21 -0700, Karl L Wuensch, > Referring to the file drawer problem and data falsification, the >president of the Association for Psychological Science recently wrote: "These >and related factors should tend to inflate 'false positives' (aka Type I >errors), which leads inexorably to the pessimistic conclusion that some unknown >(but considerably higher than .05) proportion of our field's published effects >are not true effects."
Karl is referring to an article by cognitive psychologist Douglas Medin who is currently one of the rotating presidents of APS. The article is on research misconduct and how to deal with it. The article can be accessed here: http://www.psychologicalscience.org/index.php/publications/observer/obsonline/a-science-we-can-believe-in.html NOTE: There is a comment area at the bottom of the webpage where comments, questions, and rants can be submitted. It is also useful to note the Medin's quote above is building upon the research presented in the following article: Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant Psychological Science October 2011 0956797611417632, first published on October 17, 2011 doi:10.1177/0956797611417632 Amd can be accessed here: http://pss.sagepub.com/content/22/11/1359 A popular media article on Simmons et al is provided on the Psychology Today website; see: http://www.psychologytoday.com/blog/the-good-life/201111/false-positive-psychology Simmons et al review the little "tricks" that researchers engage in to achieve statistical significance in a study but which often failures to replicate. This suggests that the false positive rate (i.e., the number of articles that represent false rejection of the null hypothesis) of journals is inflated relative to the 5% rate that might be expected if certain conditions for statistical testing are met (e.g., sample size, effect size, statistical power, etc.). Simmons et al can be seen as following in the tradition of John Ioannidis, whose most notorious article is: Ioannidis JP. (2005). Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124. Epub 2005 Aug 30. Which can be accessed for free through this website (see upper right hand corner): http://www.ncbi.nlm.nih.gov/pubmed/16060722/ The magazine "The Atlantic" has an article on Ioannidis and the implications of his research which might be worth reading; see: http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/ Some Tipsters might recognize a couple of the above references from previous postings here. > Am I misreading this, or does he imply (incorrectly) that use of the >.05 criterion of statistical significance would be expected to result in 5% of >published research findings being Type I errors? Short answer: Yes Perhaps one should make a comment on the webpage with Medin's article to point out this error. -Mike Palij New York University [email protected] --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=17083 or send a blank email to leave-17083-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
