KKARMA wrote:
>
> As a teacher of research methodology in (music) education I am
> interested in the relation between traditional statistics and the
> bayesian approach. Bayesians claim that their approach is superior
> compared with the traditional, for instance because it does not assume
> normal distributions,
Sometimes it does; and sometimes "classical" stats doesn't.
> is intuitively understandable,
Some bits are. Perhaps it's more accurate to say that while the
premises of Bayesian statistics are more intuitive at first glance, and
the conclusions come closer to what people intuitively want to know
("what is the probability that this is correct?") Bayesian stats has
its own thorny philosophical problems.
Frequentist stats says that only statements about the outcome of random
variables have "probability"; it is legitimate to say that "the
probability that this die shows a 6 on this roll is 1/6" but not (unless
the die itself was drawn at random from a well-defined collection)
"there is an 80% probability that this die rolls 6 more often than 1".
Bayesians do allow the broader use of probability to describe levels of
belief about something that was not generated by a well-defined sampling
operation, but the cost of this is that "the pump must be primed" with a
"prior probability" representing your level of belief before the
observations, and this is necessarily subjective.
The cost is not as great as it appears, because as the data accumulates
the impact of the prior becomes less and less; that is, rational
observers with initially different beliefs come to more or less agree
after observing the data.
The justification for this is that a Bayesian interpretation of an
opinion poll can actually be "The probability that the Garden Party
would get more than 40% of the votes in this election is x% [if it were
held today and voting patterns matched polling response patterns]"
whereas - despite the fact that this is intuitively the answer to the
question people *want* to ask- frequentist stats cannot.
The frequentist can only assign probabilities to samples from
well-defined populations. So the frequentist analysis of the same poll
might be "IF the Garden Party would have exactly 40% popular support
[if the election were held today and polling response patterns matched
voting patterns] the probability of getting this result or one less
favorable to the Garden Party in an opinion poll done in this way would
be y%."
> works with small samples,
To some extent; and so does frequentist stats, to some extent. Both
tend to be inconclusive with small samples, and to get some of whatever
power they have from assumptions that the data cannot justify. That's
the way the universe works: you want answers, first get enough data.
> If this is so, why is it so rare in educational research? Are there some
> hidden flaws in the approach or are the researchers just ignorant?
(1) Propagation delay. What statisticians are writing about in
theoretical journals today will be used by statisticians in their
practical work in a few years. It'll be in upper-level stats textbooks
for stats majors in a decade. Maybe in two decades significant numbers
of social science PhD's will start to hear about it; maybe another ten
years later somebody will be bold enough to put it in an applied stats
textbook; at that point it becomes well enough known to be used widely.
Maybe.
(2) Encapsulation. The philosophical complications of the frequentist
method are well-hidden for most users inside phrases such as "confidence
interval" and "significance level". You can construct a 95% confidence
interval correctly while believing that this guarantees a 95%
probability that this particular interval contains the true value (which
is not so); and you can even state this in your paper and many referees
and editors will let it pass. Similarly, you can do a hypothesis test at
the 5% significance level while believing that 5% is "the probability
that your data are wrong" (it isn't.). If people were required to
truly understand hypothesis tests and confidence intervals before using
them there might be more impetus for change.
Note: This is NOT entirely a valid argument for change, any more than
saying that "if people were required to understand the workings of their
vehicles there would be a lot more bikes on the road and a lot fewer
cars" is. On the other hand, if there were no mechanics, it might be.
(3) Standardization: There are genuine advantages to everybody singing
from the same hymnbook, which tends to lead to change being slow.
-Robert Dawson
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
