Mike and others
I would not adopt the text, though I generally favor downplaying NHST. On the one hand, NHST and CI (e.g. t tests) are very similar, and going from one to the other is simply rearranging the arithmetic. However, Welkowitz et al Intro Stat text makes an interesting defense on NHST as cutting down on Type III errors - finding a p<.05 in the _opposite direction_. Her example: Lets say Null mu = 500, you get get a mean of 530 and find it significantly different from 500 .... yet unknown to the researcher the _real_ mu is in the opposite direction, say, 490. Its not a Type I as the null of 500 really is false. Not a Type II as you did get a significant effect. So having NHST as an initial screening cuts down on those directional errors. In general (if I understand it), Welkowitz defends NHST as a screening to eliminate lots of weak or nonexistent findings, as well as directional errors. I am still pondering the Type III (she did not originate the concept). In this example, a Type III is low probability given that the real mu is slightly to the left of the stated null and the cutoff for significance is to the right of the stated null .. but it is still possible. Though I agree 100% that NHST is not very well understood or used properly. In fact, many people treat the p as the probability of the null being true. Its actually the p of such-and-such a finding IF null was true ... I suspect NHST will stick around, especially in those cases where the null _might_ be true, as in a large clinical trial with random assignment ... topic worth pursuing on TIPS JK ========================== John W. Kulig, Ph.D. Professor of Psychology Coordinator, University Honors Plymouth State University Plymouth NH 03264 ========================== ----- Original Message ----- From: "Michael Palij" <[email protected]> To: "Teaching in the Psychological Sciences (TIPS)" <[email protected]> Cc: "Michael Palij" <[email protected]> Sent: Monday, February 20, 2012 3:21:23 PM Subject: Re: [tips] Quickie Poll On How to Teach Undergraduate Statistics On Mon, 20 Feb 2012 10:56:46 -0800, Nancy Melucci wrote: >On what basis does the author conclude that NHST causes confusion? Educational >research or experience-based but essentially pure opinion? Both. But one example of the research being used is this: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3095378/ Here is the abstract: |Frontiers in Psychology. 2010; 1: 26. |Published online 2010 July 2. Prepublished online 2010 April 18. |doi: 10.3389/fpsyg.2010.00026 |Copyright © 2010 Coulson, Healey, Fidler and Cumming. | |Confidence Intervals Permit, but Do Not Guarantee, Better Inference than |Statistical Significance Testing | |Melissa Coulson,1 Michelle Healey,1 Fiona Fidler,1 and Geoff Cumming1* |1Statistical Cognition Laboratory, School of Psychological Science, |La Trobe University, Melbourne, VIC, Australia |Received March 16, 2010; Accepted June 9, 2010. |This is an open-access article subject to an exclusive license agreement |between the authors and the Frontiers Research Foundation, which permits |unrestricted use, distribution, and reproduction in any medium, provided |the original authors and source are credited. | |Abstract |A statistically significant result, and a non-significant result may differ little, |although significance status may tempt an interpretation of difference. Two |studies are reported that compared interpretation of such results presented |using null hypothesis significance testing (NHST), or confidence intervals (CIs). |Authors of articles published in psychology, behavioral neuroscience, and |medical journals were asked, via email, to interpret two fictitious studies |that found similar results, one statistically significant, and the other non-significant. |Responses from 330 authors varied greatly, but interpretation was generally |poor, whether results were presented as CIs or using NHST. However, when |interpreting CIs respondents who mentioned NHST were 60% likely to conclude, |unjustifiably, the two results conflicted, whereas those who interpreted CIs without |reference to NHST were 95% likely to conclude, justifiably, the two results |were consistent. Findings were generally similar for all three disciplines. An |email survey of academic psychologists confirmed that CIs elicit better |interpretations if NHST is not invoked. Improved statistical inference can result |from encouragement of meta-analytic thinking and use of CIs but, for full benefit, |such highly desirable statistical reform requires also that researchers |interpret CIs without recourse to NHST. Make of it what you will. It probably goes without saying that one should read the original article in order to be sure of the details. -Mike Palij New York University [email protected] -----Original Message----- From: Michael Palij <[email protected]> To: Teaching in the Psychological Sciences (TIPS) <[email protected]> Cc: Michael Palij <[email protected]> Sent: Mon, Feb 20, 2012 8:13 am Subject: [tips] Quickie Poll On How to Teach Undergraduate Statistics Okay, I beg your indulgence and participation in an unscientific poll where ou can either post your response to the TiPS list (for discussion) or mail your response directly to me. I am finishing a book review n an undergraduate statistics textbook that (a) attempts to eliminate ll null hypothesis signitifcance testing (NHST) in favor of focusing n effect sizes (ES), confidence intervals (CI), and (old fashioned) eta-analysis nd (b) encourages research on "statistical cognition" which, according o the author, shows that teaching NHST causes greater confusion n students than an ES/CI approach iven that limited description, I'm going to make this into a -alternative forced choice question: Would you use such a textbook as the main textbook in he first/introductory statistics in psychology course? [ [ Yes ] No Comments? If you care to, you might comment on whether current intro stat extbooks do an adequate job of covering issues such as effect izes and confidence intervals (these days I use some version f Gravetter and Wallnau which, in my opinion, do an adequate ob introducing the topics which I assume lay the foundation for more advanced undergraduate course in statistical methods). Thanks in advance for your cooperation. -Mike Palij ew York University [email protected] P.S. And, no, this not about procrastinating on finishing the book eview. Well, mostly it's not. ;-) --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13338.f659d005276678c0696b7f6beda66454&n=T&l=tips&o=16162 or send a blank email to leave-16162-13338.f659d005276678c0696b7f6beda66...@fsulist.frostburg.edu --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=16164 or send a blank email to leave-16164-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
