I'm with you, Scott. I find "statistically reliable" every bit as troublesome as "statistically significant." Perhaps we should just stop reporting p values and report instead a confidence interval for the effect size estimator (standardized or not).
Cheers, Karl W. ________________________________ From: Lilienfeld, Scott O [mailto:[email protected]] Sent: Sunday, March 21, 2010 12:26 PM To: Teaching in the Psychological Sciences (TIPS) Subject: RE: [tips] Odds are, it's wrong Don - Interesting, because my take is different...I always cross out the term "reliable" when it's used in lieu of "significant," because I fear that the former term implies that the finding is likely to be consistent or replicable. In fact, in part because of issues of statistical power, many or most findings in the social sciences don't replicate: See, e.g., Sohn, D. (1998). Statistical significance and replicability: Why the former does not presage the latter. Theory and Psychology, 8, 291-311. So I fear that the term "reliable" will lead some students and readers to conclude that a statistically significant result is likely to be reliable in the psychometric sense (viz., consistent). But perhaps I'm in a minority on this score, and I'll be curious to hear from other TIPSTERs. ....Scott Scott O. Lilienfeld, Ph.D. Professor Editor, Scientific Review of Mental Health Practice Department of Psychology, Room 473 Psychology and Interdisciplinary Sciences (PAIS) Emory University 36 Eagle Row Atlanta, Georgia 30322 [email protected] (404) 727-1125 Psychology Today Blog: http://www.psychologytoday.com/blog/the-skeptical-psychologist 50 Great Myths of Popular Psychology: http://www.wiley.com/WileyCDA/WileyTitle/productCd-140513111X.html Scientific American Mind: Facts and Fictions in Mental Health Column: http://www.scientificamerican.com/sciammind/ The Master in the Art of Living makes little distinction between his work and his play, his labor and his leisure, his mind and his body, his education and his recreation, his love and his intellectual passions. He hardly knows which is which. He simply pursues his vision of excellence in whatever he does, leaving others to decide whether he is working or playing. To him - he is always doing both. - Zen Buddhist text (slightly modified) From: don allen [mailto:[email protected]] Sent: Sunday, March 21, 2010 12:14 PM To: Teaching in the Psychological Sciences (TIPS) Subject: Re: [tips] Odds are, it's wrong Hi Stephen- Thanks for the link. It's a pretty good summary of some of the "dirty little secrets" of statistical analysis. However I was surprized at the lack of mention of effect size as an important tool for dealing with the "significant equals important" isssue. Re this point; I always insited that students use the term "statistically reliable" rather than "statistically significant". While a small point I think that it helps people understand what their findings actually mean. -Don. ----- Original Message ----- From: [email protected] Date: Sunday, March 21, 2010 8:24 am Subject: [tips] Odds are, it's wrong To: "Teaching in the Psychological Sciences (TIPS)" <[email protected]> > Alerted by a colleague, I recommend an instructive if > depressing essay on the problematic use of statistics in science. > > http://www.sciencenews.org/view/feature/id/57091/title/Odds_ar > e,_its_w or http://tinyurl.com/yh7sk7r > > Teaser: > > "Supposedly, the proper use of statistics makes relying on > scientific results a safe bet. But in practice, widespread > misuse > of statistical methods makes science more like a crapshoot." > > Stephen > -------------------------------------------- > Stephen L. Black, > Ph.D. > Professor of Psychology, Emeritus > Bishop's > University > e-mail: sblack at ubishops.ca > 2600 College St. > Sherbrooke QC J1M 1Z7 > Canada > ----------------------------------------------------------------- > ------ > > --- > You are currently subscribed to tips as: [email protected]. > To unsubscribe click here: > http://fsulist.frostburg.edu/u?id=13157.966b795bc7f3ccb35e3da08aebe98f18&n=T&l=tips&o=1419or > send a blank email to > leave-1419-13157.966b795bc7f3ccb35e3da08aebe98...@fsulist.frostburg.edu > --- You are currently subscribed to tips as: [email protected]<mailto:[email protected]>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13509.d0999cebc8f4ed4eb54d5317367e9b2f&n=T&l=tips&o=1425 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-1425-13509.d0999cebc8f4ed4eb54d5317367e9...@fsulist.frostburg.edu<mailto:leave-1425-13509.d0999cebc8f4ed4eb54d5317367e9...@fsulist.frostburg.edu> ________________________________ This e-mail message (including any attachments) is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message (including any attachments) is strictly prohibited. If you have received this message in error, please contact the sender by reply e-mail message and destroy all copies of the original message (including attachments). --- You are currently subscribed to tips as: [email protected]<mailto:[email protected]>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13060.c78b93d4d09ef6235e9d494b3534420e&n=T&l=tips&o=1426 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-1426-13060.c78b93d4d09ef6235e9d494b35344...@fsulist.frostburg.edu<mailto:leave-1426-13060.c78b93d4d09ef6235e9d494b35344...@fsulist.frostburg.edu> --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=1429 or send a blank email to leave-1429-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
