[EMAIL PROTECTED] (dennis roberts) wrote
> most software will compute p values (say for a typical two sample t test of
> means) by taking the obtained t test statistic ... making it both + and -
> ... finding the two end tail areas in the relevant t distribution ... and
> report that as p
>
> for example ... what if we have output like:
>
>
> N Mean StDev SE Mean
> exp 20 30.80 5.20 1.2
> cont 20 27.84 3.95 0.88
>
> Difference = mu exp - mu cont
> Estimate for difference: 2.95
> 95% CI for difference: (-0.01, 5.92)
> T-Test of difference = 0 (vs not =): T-Value = 2.02 P-Value = 0.051 DF = 35
>
> for 35 df ... minitab finds the areas beyond -2.20 and + 2.02 ... adds them
> together .. and this value in the present case is .051
>
> now, traditionally, we would retain the null with this p value ... and, we
> generally say that the p value means ... this is the probability of
> obtaining a result (like we got) IF the null were true
>
> but, the result WE got was finding a mean difference in FAVOR of the exp
> group ...
>
> however, the p value does NOT mean that the probability of finding a
> difference IN FAVOR of the exp group ... if the null were true ... is .051
> ... right? since the p value has been calculated based on BOTH ends of the
> t distribution ... it includes both extremes where the exp is better than
> the control ... AND where the cont is better than the exp
>
> thus, would it be fair to say that ... it is NOT correct to say that the p
> value (as traditionally calculated) represents the probability of finding a
> result LIKE WE FOUND ... if the null were true? that p would be 1/2 of
> what is calculated
>
> this brings up another point ... in the above case ... typically we would
> retain the null ... but, the p of finding the result LIKE WE DID ... if the
> null were true ... is only 1/2 of .051 ... less than the alpha of .05 that
> we have used
>
> thus ... what alpha are we really using when we do this?
>
> this is just a query about my continuing concern of what useful information
> p values give us ... and, if the p value provides NO (given the results we
> see) information as to the direction of the effect ... then, again ... all
> it suggests to us (as p gets smaller) is that the null is more likely not
> to be true ...
>
> given that it might not be true in either direction from the null ... how
> is this really helping us when we are interested in the "treatment" effect?
>
> [given that we have the direction of the results AND the p value ...
> nothing else]
>
I fail to see the problem.
If the researcher has a priori expectations about the *direction* of the
effect, he should use a one-sided significance test.
That's what they are for, aren't they?
Chris
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
