On 5/22/2006 5:25 AM, Peter Dalgaard wrote: > Uwe Ligges <[EMAIL PROTECTED]> writes: > > >>> A = 0.1846, p-value = 0.9059 >>> >>>> ad.test(rnorm(100,mean=5,sd=3)) >>> ... >>> A = 0.5138, p-value = 0.1887 >>> >>> I mistakenly had thought the p-values would be more stable since I >>> am artificially creating a random normal distribution. Is this expected >>> for a normality test or is this an issue with how rnorm is producing >>> random numbers? I guess if I run it many times, I would find that I >>> would get many large values for the p-value? >> Well, as many large values as small values, 5% significant differences >> for the 5% level.... >> >> The following looks alright: >> >> hist(replicate(1000, ad.test(rnorm(100,mean=5,sd=3))$p.value)) > > We see this misunderstanding worryingly often. Worrying because it > reveals that a fundamental aspect of statistical inference has not > been grasped: that p-values are designed to be (approximately) > uniformly distributed and fall below any given level with the stated > probability, when the null hypothesis is true.
I think it's the fallacious belief that the p-value measures the probability that the null hypothesis is true. This is currently misunderstanding #1 in the Wikipedia entry for P-values. (Google had me worried: I searched for "probability that the null hypothesis is true" and found > P-value - Wikipedia, the free encyclopedia > The p-value is the probability that the null hypothesis is true, justifying > the "rule" of considering as significant p-values closer to 0 (zero). ... > en.wikipedia.org/wiki/P-value - 17k - Cached - Similar pages This quote is preceded by: "All of the following [...] statements are false:" Context is important! :-) The vast majority of hits to that search also pointed out that this interpretation was incorrect. A couple of counterexamples were a "research methods" page at a department of psychology, and another at a medical school. I'll send a copy of this note to people there. Duncan Murdoch > > There is no mechanism to give you "fewer significant" or "more stable" > p-values, and a p-value close to one is no better an indication of a > true null hypothesis than one of 0.5 or 0.25. > ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
