I like the idea of providing the actual p value and leaving it to the reader to decide but I don't think many readers will, without any justification being provided, give much weight to a result with a probability greater than .05. And an author should never try the scam (which I have seen) of presenting the results like, "Group A scored higher than Group B, t(24)=2.31, p=.06" without pointing out to the reader that the alpha standard for significance was not met (even though they didn't actually use the word "significantly" in the presentation). In fact, with such a result, you would need to provide some additional justification for putting any stock in those results.
In addition to providing actual p values and letting the reader decide, the presentation of confidence intervals can give readers more help in deciding how much weight to give an experimental result. It provides the range in which the population mean is likely to be found. I am also always curious with z or t results with p-values less than .05 but greater than .025. I remember one time that I was looking closely at the results of a research article and I determined that, without mentioning it or really having any justification for it, the author had used a one-tailed probability, most likely because the two-tailed would not have been significant. If you have the time, such checks are now easy in many cases with the probability functions of many distributions calculated in Excel. I also agree that nonsignificant results give us a great opportunity to address the topic of power analysis. The higher the power, the more we might make from nonsignificant results being actually indicative of no effect in the population. The suggestions of the Task Force on Statistical Inference of the APA Board of Scientific Affairs on this and many other statistical issues is available at: http://www.apa.org/journals/amp/amp548594.html. It was originally published in the American Psychologist in August 1999, Vol. 54, No. 8, 594-604. The task force also repeatedly referenced the seminal work of Cohen: Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003. Rick Dr. Richard L. Froman Psychology Department John Brown University Siloam Springs, AR 72761 e-mail: [EMAIL PROTECTED] phone and voice mail: (479)524-7295 http://www.jbu.edu/sbs/rfroman.html -----Original Message----- From: Martin J. Bourgeois [mailto:MartyB@;uwyo.edu] Sent: Monday, November 11, 2002 10:38 AM To: Teaching in the Psychological Sciences Subject: RE: Marginally Significant? I like Bob Abelson's advice about this: report the alpha level (e..g, .05) along with the observed p value (.056) and the effect size, and let the journal editor and reader decide what to make of it. Abelson has a great explanation of the topic in Chapter 4 of his great book, Statistics As Principled Argument. He also discusses John Tukey's suggestion that we call results between p=.05 and p=.15 as leaning toward significance, and results from p=.15 to p=.25 as hinting toward significance. Thinking in this way avoids what Abelson calls categoritis regarding statistical tests and gets students to think about what statistics do. Marty Bourgeois University of Wyoming -----Original Message----- From: Rob Flint [mailto:flintr@;mail.strose.edu] Sent: Monday, November 11, 2002 5:40 AM To: Teaching in the Psychological Sciences Subject: Marginally Significant? One of my students doing her senior thesis ran her stats and got results of .056 and .08 for two different ANOVAs. In the past I have seen published studies indicating that these are "marginally significant." How do you deal with results of this nature? More importantly, do you have any citations (journals or books) that discuss the value of including/discussing results that seem to "approach significance"? Thanks, Rob Flint ------------------------------------------------------------- Robert W. Flint, Jr., Ph.D. The College of Saint Rose Department of Psychology 432 Western Avenue Albany, NY 12203-1490 Office: 518-458-5379 Lab: 518-454-2102 Fax: 518-458-5446 Behavioral Neuroscience Homepage: http://academic.strose.edu/academic/flintr/ Department of Psychology Homepage: http://academic.strose.edu/academic/psychology/index.htm --- You are currently subscribed to tips as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] --- You are currently subscribed to tips as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] --- You are currently subscribed to tips as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED]
