On Fri, 7 Apr 2000, dennis roberts wrote:
> i was not suggesting taking away from our arsenal of tricks ... but, since
> i was one of those old guys too ... i am wondering if we were mostly lead
> astray ...?
I've been wondering a lot about this myself lately. Chow (1988) and Meehl
(1978) have both argued that in agriculture the substantive theory and
statistical hypothesis are very closely tied. A situation that is not
true in Psychology, Education, and some other areas.
I tend to agree. Dennis's example of wanting to know if one treatment is
BETTER than another is typical of what we teach. We collect data and
conduct a hypothesis test to determine if the new treatment is DIFFERENT
from the old one.
Establishing that two treatments are DIFFERENT does not automatically
establish that one is necessarily BETTER. However, establishing a
difference is a necessary, but not sufficient, step to establishing an
improvement. Many non-statistical issues enter into the ultimate decision
about whether the new treatment is better (e.g., cost, harmful side
effects) and hypothesis testing does not answer those questions.
Hypothesis testing is only useful when we use it within a context. Why is
the null hypothesis always a hypothesis of no difference or no
correlation? There is nothing in the mathematics of statistical
procedures that requires a zero. But, by always using a zero value for
the null hypothesis we free ourselves from considering the context within
which we are using the test. We do not have to bother thinking about
whether or not zero is really the correct value for our hypothesis. In
fact, we don't have to think at all about the context within which we are
conducting the test. In the worst case there is a complete disconnect
between our real question and the statistical hypothesis.
Eventually people become troubled by this lack of connection between the
hypothesis test and what they really want to know and start wondering if
the world wouldn't be a better place without hypothesis tests.
But, the hypothesis test is only a tool (one of several as someone else on
this thread pointed out) and a tool is only useful if it is used in a
context where it is useful. (Is that sentence redundant enough for you?)
If hypothesis testing were banned today, on Monday we would have to begin
reinventing it. Hopefully we would reinvent it in a manner that makes it
more easily used in a useful fashion. (I tend to favor an emphasis on
prediction and goodness-of-fit. This approach incorporates parameter
estimation into the hypothesis testing practice and, I believe, forces the
user to consider more clearly the link between their substantive theory
and the statistical hypothesis.)
But whatever form hypothesis testing takes it must first and formost be
viewed in the context of the question being asked. Sometimes a hypothesis
test is all the information that is needed to answer a given question.
Sometimes it is only one piece of a larger collection of information. And
sometimes it is irrelevant to the question.
Michael
>
> the more i work with statistical methods, the less i see any meaningful (at
> the level of dominance that we see it) applications of hypothesis testing ...
>
> here is a typical problem ... and we teach students this!
>
> 1. we design a new treatment
> 2. we do an experiment
> 3. our null hypothesis is that both 'methods', new and old, produce the
> same results
> 4. we WANT to reject the null (especially if OUR method is better!)
> 5. we DO a two sample t test (our t was 2.98 with 60 df) and reject the
> null ... and in our favor!
> 6. what has this told us?
>
> if this is ALL you do ... what it has told you AT BEST is that ... the
> methods probably are not the same ... but, is that the question of interest
> to us?
>
> no ... the real question is: how much difference is there in the two methods?
>
> our t test does NOT say anything about that
>
> 1 to 6 can be applied to all sorts of hyp tests ... and most lead us
> essentially into a dead end
>
> At 04:23 PM 4/7/00 +0000, j. williams wrote:
> >Some of us in the minority would not follow what might be
> >"fashionable." I am
> >one of those who believe hypothesis testing is still an important and
> >integral
> >part of statistics. Hypothesis testing is but one tool in the whole arsenal
> >however. OTH, I'm an old guy who went through graduate school way back in
> >the
> >60s. Teaching old dogs new tricks is not easy, right? If such a vote were
> >taken today with the results suggested by Mr.Roberts, I know I have
> >successfully misled literally thousands of students. Would re-education
> >be the
> >answer?
> >
> >
> >[EMAIL PROTECTED] (dennis roberts) wrote in
> ><[EMAIL PROTECTED]>:
> >
> > >let's say that today ... we as the statistical community decided, by
> > >democratic vote, that the concept of 'hypothesis testing' ... which has
> > >essentially dominated statistical work for as long as i can remember
> > >(which, .... er um ... is a LOT of years!) ... is relegated to the 'we
> > >USED to do this stuff' category
> > >
> > >just THINK about this ....
> > >
> > >what would the vast majority of folks who either do inferential work
> > >and/or teach it ... DO????
> > >what analyses would they be doing? what would they be teaching?
> >
> >
> >===========================================================================
> >This list is open to everyone. Occasionally, less thoughtful
> >people send inappropriate messages. Please DO NOT COMPLAIN TO
> >THE POSTMASTER about these messages because the postmaster has no
> >way of controlling them, and excessive complaints will result in
> >termination of the list.
> >
> >For information about this list, including information about the
> >problem of inappropriate messages and information about how to
> >unsubscribe, please see the web page at
> >http://jse.stat.ncsu.edu/
> >===========================================================================
>
>
>
>
> ===========================================================================
> This list is open to everyone. Occasionally, less thoughtful
> people send inappropriate messages. Please DO NOT COMPLAIN TO
> THE POSTMASTER about these messages because the postmaster has no
> way of controlling them, and excessive complaints will result in
> termination of the list.
>
> For information about this list, including information about the
> problem of inappropriate messages and information about how to
> unsubscribe, please see the web page at
> http://jse.stat.ncsu.edu/
> ===========================================================================
>
*******************************************************************
Michael M. Granaas
Associate Professor [EMAIL PROTECTED]
Department of Psychology
University of South Dakota Phone: (605) 677-5295
Vermillion, SD 57069 FAX: (605) 677-6604
*******************************************************************
All views expressed are those of the author and do not necessarily
reflect those of the University of South Dakota, or the South
Dakota Board of Regents.
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
