In article <[EMAIL PROTECTED]>,
Rich Strauss <[EMAIL PROTECTED]> wrote:
>At 08:50 AM 2/2/01 -0500, Herman Rubin wrote:
>>But this is not the case with fixing a p value. Most
>>testing problems have the property that the appropriate
>>procedure to be used corresponds to a p value for that
>>problem AND THAT SAMPLE SIZE, but the p value to be used
>>depends quite substantially on the sample size.
>I'm not sure I understand your point. Are you saying that the p-value
>depends on sample size beyond the use of degrees-of-freedom to choose the
>appropriate null distribution?
>Rich Strauss
This is exactly what I am saying. As the sample size
increases, the probability of incorrect acceptance when the
hypothesis is false decreases for a fixed p-value, so some
of this improvement should be used instead to decrease the
probability of the type I error.
To give an example, let X_i be independent two dimensional
(only chosen for computational convenience) normal random
variables with mean vector \mu and covariance matrix qI.
Also, again for computational convenience, suppose the
importance of accepting the null hypothesis if \mu lies in
a set of area A not containing 0 to be A/(2\pi) times the
importance of rejecting the null hypothesis if it is true.
Other formulations will give similar results.
Now if we have a sample of size n, the mean has variance
v = q/n. As expected, rejection should take place if the
norm of the sample mean exceeds some value r. Then the
proability of incorrect rejection will be exp(-r^2/2v),
and the corresponding importance of incorrect acceptance
will be r^2/2, for a total "loss" of r^2/2 + exp(-r^2/2v).
If we minimize this with respect to r, we find that
r^2 = 2 ln(1/v), if v < 1, and 0 if v >= 1. So the
p-value for the optimal procedure would be min(v, 1).
Other scenarios are likely to be harder to calculate,
but the conclusions are similar. The p-value to use
will depend on the precision of the usual estimator,
which depends on the sample size, for any given type
of problem.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
[EMAIL PROTECTED] Phone: (765)494-6054 FAX: (765)494-0558
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
