[NOTE: this is CC'd to EDSTSAT-L]
Stan Brown wrote:
>... Now we come to the part I'm having conceptual trouble with: "Have
> you proven that one gas gives better mileage than the other? If so,
> which one is better?"
>
> Now obviously if the two are different then one is better, and if
> one is better it's probably B since B had the higher sample mean.
> But are we in fact justified in jumping from a two-tailed test (=/=)
> to a one-tailed result (>)?
>
> Here we have a tiny p-value, and in fact a one-tailed test gives a
> p-value of 0.0001443. But something seems a little smarmy about
> first setting out to discover whether there is a difference -- just
> a difference, unequal means -- then computing a two-tailed test and
> deciding to announce a one-tailed result.
>
> Am I being over-scrupulous here? Am I not even asking the right
> question? Thanks for any enlightenment.
Yes, you're being over-scrupulous. The "tailedness" of the test is a
matter of what p-value you are claiming; as I've argued before, under
ordinary circumstances the one-tailed t test should always be avoided
anyhow.
The tailedness of the interpretation is a completely separate matter.
It is true that there is a tiny risk of what ought to be called a "type
III error" (were that not a standing but slightly misleading jocoserium
referring to "testing the wrong hypothesis" or something of the kind) in
which the null is wrong in one direction but the sample suggests that it
is wrong in the other direction; the probability of this (conditional on
the actual distribution and choice of null) is always strictly less than
the p-value, so we need not be overly concerned.
Moreover, if we have determined that we have enough power that we may
think of the failure-to-reject outcome as implying that the difference
is not practically significant (which implies outside knowledge of what
constitutes practical significance), the conditional probability that we
draw a sample implying ">" when the reality is "<" by a practically
significant amount is tiny even compared with p.
My advice, for t tests: ALWAYS test two-sided; ALWAYS interpret
one-sided if you reject the null.
Happy New Year!
-Robert Dawson
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at
http://jse.stat.ncsu.edu/
=================================================================
