In article <01e301bfa2ee$b69f42b0$[EMAIL PROTECTED]>,
Robert Dawson <[EMAIL PROTECTED]> wrote:
>Dennis Roberts asked, imagining a testing-free universe:
>>> what would the vast majority of folks who either do inferential work
>>and/or
>>> teach it ... DO????
>>> what analyses would they be doing? what would they be teaching?
>I wrote:
>> >* students would be told in their compulsory intro stats that
>> > "a posterior probability of 95% or greater is called
>> > "statistically significant", and we say 'we believe
>> > the hypothesis'. Anything less than that is called
>> > "not statistically significant", and we say 'we disbelieve
>> > the hypothesis'".
>and Herman Rubin responded:
>> Why? What should be done is to use the risk of the procedure,
>> not the posterior probability. The term "statistically significant"
>> needs abandoning; it is whether the effect is important enough
>> that it pays to take it into account.
>Dennis asked what _would_ happen, not what _should_. Most of the abuses we
>see around us are not the fault of hypothesis testing _per_se_, but of
>statistics users who believe:
> (a) that their discipline ought to be a science;
What is a science? The word means "knowledge".
> (b) that statistics must be used to make this so;
The problem is that they expect statistics to take in their
data and spew out the TRUTH. The capital letters are not
an exaggeration.
> (c) and that it is unreasonable to expect them to _understand_
>statistics just because of (a) and (b).
They have elevated statistics to a religion, and as in many
religions, the layman only has to carry out the sacrifices
ordered by the priest to get the blessings of the gods.
They do not need to understand statistical CALCULATIONS,
and they do not have to be able to produce the proofs. What
they need to understand are the concepts of probability and
decision making, so that they can accurately communicate
their problems to those who can help with the mechanics.
>Granted, if they did understand statistics, they would not test hypotheses
>nearly as often as they do. However, that said, I am not entirely persuaded
>that risk calculation is the whole story, either. In many pure research
>situations, "risk" is just not well defined. What is the risk involved in
>believing (say) that the universe is closed rather than open?
Both "hypotheses" are highly composite. In a situation
like this, what is the ADVANTAGE of assuming one rather
than the other? What action is going to be taken?
There may be a point in investigating the problem, but is
there one in drawing inferences?
> Moreover, suppose we elected Herman to the post of Emperor of Inference,
>(with the power of the "Bars and the Axes"?) to enforce a risk-based
>approach to statistics (not that he'd take it, but bear with me...), would
>the situation realy improve?
> My own feeling is that, in many "soft" science papers of the sort where
>the research is not immediately applied to the real world, but may affect
>public policy and personal politics, a "risk" aproach would be disastrous.
>If the researcher had to assign "risks" to outcomes that were merely a
>matter of correct or incorrect belief, it would be all too tempting to
>assign a large risk to an outcome that "would set back the cause of X fifty
>years" and conversely a small risk to accepting a belief that might be
>considered "if not true, at least a useful myth." (Exercise: provide your
>own examples). Everything would be lowered to the level of Pascal's Wager -
>surely the canonical example of the limitations of a risk-based approach?
It is precisely in these situations that a risk approach is
absolutely necessary. But the input to this, or any other,
sound risk approach must be made by those who will benefit
or suffer from the decision. In the case of medical
procedures, unless there is a public health question like
the spread of disease, it is the risk function of the patient
which is the one which should be used. For public policy,
it is the risk function of the government which is involved.
On this topic, there is a fair book by Clemen, _Making Hard
Decisions_.
> One might argue that in such a situation the rare reader who intends to
>take action, and not the writer, should do the statistics. Unfortunately, in
>the real world, that won't wash. People want simple answers, and with the
>flood of information that we have to deal with in keeping up with the
>literature in any subject today, this is not entirely a foolish or lazy
>desire.
As Einstein stated, make things as simple as possible, but
NO SIMPLER. It is a foolish desire, fanned by ignorance.
We should be teaching that statistics at least as difficult
as the Oracle of Delphi, and that understanding the Oracle
can be as difficult as solving the problem otherwise.
It is considered the author's responsibility to reach a conclusion,
>not just to present a mass of undigested data for posterity to analyze.
>Thus, it would be unrealistic to expect any discipline, forced to use
>risk-based inference, to do other than have the author guess at risks (and
>work with those guesses) in situations where objective measurements of risk
>don't exist.
The only general conclusion would be a summary of the likelihood
function, or a reduction of the data to a point where the loss of
information is not critical in computing a good approximation to
the "best" action. Approximations are necessary, as any problem
is going to be too complex for an exact solution. But this does
not say anything about reaching much in the direction of
pontificating solutions. It is that which is done, and that is
what should not be done.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
[EMAIL PROTECTED] Phone: (765)494-6054 FAX: (765)494-0558
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
