Responses to various folks. And to everyone touchy about one-tailed
tests, let me make it quite clear that I am only promoting them as a
way of making a sensible statement about probability. A two-tailed p
value has no real meaning, because no real effects are ever null. A
one-tailed p value, for a normally distributed statistic, does have a
real meaning, as I pointed out. But precision of
estimation--confidence limits--is paramount. Hypothesis testing is
passe.
Donald Burrill queried my assertion about one-tailed p values
representing the probability that the true value is opposite in sign
to what you observed. Don restated what a one-tailed p represents,
as it is defined by hypothesis testers, but he did not show that my
assertion was false. He did point out that I have to know the
sampling distribution of the statistic. Yes, of course. I assumed a
normal (or t) distribution.
Here's one proof of my assertion, using arbitrary real values. I
always find these confidence-limit machinations a bit tricky. If
someone has a better way to prove this, please let me know.
Suppose you observe a value of 5.3 for some normally distributed
outcome statistic X, and suppose the one-tailed p is 0.04.
Therefore the sampling distribution is such that, when the true value
is 0, the observed values will be greater than 5.3 for 4% of the time.
Therefore, when the true value is not 0 but something else, T say,
then X-T will be greater than 5.3 for 4% of the time. (This is the
tricky bit. Don't leap to deny it without a lot of thought. It
follows, because the sampling distribution is normal. It doesn't
follow for sampling distributions like the non-central t.)
But if X-T > 5.3 for 4% of the time, then rearranging, T < 5.3-X for
4% of the time. But our observed value is 5.3, so T < 0 for 4% of the
time. That is, there is a 4% chance that the true value is less than
zero. QED.
Don also wrote
>You had in mind, I trust, the _two-sided_ 95% confidence interval!
Of course. I only thing I've got against 95% confidence intervals is
that they are too damn conservative, by half. The default should be
90% confidence intervals. I think being wrong about something (here,
the true value) 10% of the time is more realistic in human affairs.
But obviously, in any specific instance, it depends on the cost of
being wrong.
Dennis Roberts wrote:
>1. some test statistics are naturally (the way they work anyway) ONE
>sided with respect to retain/reject decisions
Look, forget test statistics. What matters is the precision of the
estimate of the EFFECT statistics. If you keep that in front of
everything else, the question of hypothesis testing with any number
of tails just vanishes into thin air. The only use for a test
statistic is to help you work out a confidence interval. Don't ever
report them in your papers.
Herman Rubin wrote about my assertion:
>This is certainly not the case, except under highly dubious
>Bayesian assumptions.
Herman, see above. And the only Bayesian assumption is what you
might call the null Bayesian: that there is no prior knowledge of
the true value. But any Bayesian- vs frequentist-type arguments here
are academic.
Jerry Dallal wrote, ironically:
>If you're doing a 1 tailed test, why test at all? Just switch from
>standard treatment to the new one. Can't do any harm. Every field
>is littered with examples where one-tailed tests would have led to
>disasters (harmful treatments missed, etc.) had they been used.
As you well know, Jerry, 5% is arbitrary.
Will
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
