[EMAIL PROTECTED] wrote:
>
> In article <[EMAIL PROTECTED]>,
> Jerry Dallal <[EMAIL PROTECTED]> wrote:
>
> > (1) statistical significance usually is unrelated to practice
> > importance.
>
> I don't think so. I can think of many examples in which statistical
> inference plays an invaluable role in practical applications and
> instrumentation, or indeed any "practical" application of a theory etc.
> Not just in science, but engineering, e.g aircraft design, studying the
> brain, electrical enginerring. Certainly there are examples of
> statistical nonsense, e.g. polls, but i wouldn't go so far as to say it
> is usually like this.
Chris: That's not what Jerry means. What he's saying is that if your
sample size is large enough, a difference may be statistically
significant (a term which has a very precise meaning, especially to the
Apostles of the Holy 5%) but not large enough to be practically
important. [A hypothetical very large sample might show, let us say,
that a very expensive diet supplement reduced one's chances of a heart
attack by 1/10 of 1%.] Alternatively, in an imperfectly-controlled
study, it may show an effect that - whether large enough to be of
interest or not - is too small to ascribe a cause to. [A moderately
large study might show that some ethnic group has a 1% higher rate of
heart attacks, with amargin of error of +- .2% . But we might have, for
an effect of this size, no way of telling whether it's due to genes,
diet, socioeconomic factors, recreational drugs, or whatever.]
> I *would* argue that without some method to determine the likelihood of
> a difference b/w two conditions you have no chance of determining
> practical importance at all.
>
> > (2) absence of evidence is not evidence of absence
>
> Everyone who has done elementary statistics is aware of this edict. But
> what if your power is very high and/or you have very large N? I have
> always found it surprising that we can't turn it around and develop a
> probability that two groups are the same.
In a frequentist philosophy, we are not allowed to do this,
because the nature of the two populations has not been randomized in any
well-defined way, so the concept of "probability" does not apply.
The Bayesian approach, which permits probabilities to be assigned to
statements about parameters, *does* allow us to answer such questions.
However, it depends, in general, on the "prior distribution" of the
parameters that you select. In many cases, this makes it hard to make
definitive statements (though if you have a lot of data it may well be
that all plausible priors produce similar posterior distributions).
However, here - with continuous parameters - the probability that the
parameters of two disjoint groups are _the_same_ is easy to compute -
it's 0. Like the probability of two people being exactly the same
height.
If you want to ask, in a Bayesian framework, for the probability that
two population parameters are equal to within some specified tolerance,
go right ahead. Alternatively, within a frequentist framework, you can
test the hypothesis that the absolute value of the difference is less
than some specified level.
-Robert Dawson
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
