On Tue, 11 Apr 2000, Robert Dawson wrote:
> Michael Granaas wrote (in part):
> > And if the data are consistent with the null we do what? Act as if the
> > null were true? Act as if nothing were true?
>
> In general, that last is exactly what not rejecting the null permits.
> Recall that failing to reject the null is (among other things) the almost
> certain result of using an undersized sample. Therefore, any general
> interpretation "if you do not reject the null then believe...." must cover
> that eventuality. If you have separate rules for interpretation depending
> on sample size you are getting back into interval estimation, a better place
> to be anyway.
The problem is that interval estimation and null hypothesis testing are
seen as distinct species. An interval that includes zero leads to the
same logical problems as failure to reject a false null.
>
> > Every research project should plausibly add to our knowledge base. But,
> > if the null is a priori false failure to reject is just that a failure and
> > waste of time.
>
> Exactly. And most nulls *are* a priori false.
>
But they are only false as a matter of practice, not necessity.
>
>
> I think you may have missed my point, which was that if we conclude that
> an effect is too small to be of interest, it should be because it is too
> small to be of interest, not because it is too small to detect with the data
> easily available. (And the fox, having tried to reach the grapes and failed,
> said "Aaaaah! Those grapes were not statistically significant anyway.)
Yes, I interpreted your original statement differently than this.
>
>
> >
> > > But if you talk the talk you should also
> > > walk the walk. Hypothesis testing does not give you any way to formally
> > > introduce the idea that a null is "plausibly true".
> >
> > In our best understanding of Fisher's work this would indeed seem to be
> > what he said. In the world of 1928 his notion of null hypotheses was
> > certainly a great leap forward. But just because Fisher said that there
> > was no way to introduce the idea that the null was plausibly true does not
> > make it so. Fisher also said that there was no such thing as power or
> > type II errors or alternative hypotheses. These are perhaps not
> > universally accepted concepts, but they certainly have a wide acceptance.
> >
> > Unfortunately in hypothesis testing where the null may not be plausibly
> > true we have no means of discovering truth, only falsehood. We conclude
> > that either the null is false and something else must be true or we
> > conclude that the evidence is inconclusive and we know nothing. In the
> > first case we know that something is true but we do not know what that
> > something is. In the second we know nothing.
>
> Whatever Fisher said - and I am not an historian of statistics -
> that last paragraph does indeed seem to sum up the situation; and it would
> seem like a good argument against hypothesis testing in many situations.
As Fisher concived of it, yes. Just because Fisher gave us a particular
logic 75+/- years ago does not mean we have to accept it as gospel today.
We have the benefit of hindsight if nothing more with which to modify
hypothesis testing so that it is a useful tool.
At this point I am thinking that you are not a fan of hypothesis testing
when the null is a priori false. Am I right? If I am we can focus on my
intended main point that hypothesis test reconceptualized can be useful.
>
> >If I am to know what is true I must first identify all of the things that
> >may plausibly be true and then systematically eliminate them until I find
> >one that I cannot eliminate. That one then must be true.
>
> *If* you can eliminate all but one, it would follow that the one was
> true.
> In practice, this is rarely if ever the case.
In some fields/sub-fields there is no systematic falsification of anything
for any number of reasons. At least one of those has to do with testing
only a priori false nulls, which fail to eliminate all but the weakest
competing explanation, in a belief that doing so leads to truth.
With an updated conceptualization of hypothesis testing this could change
for the better.
> In order to
> >achieve this I must first accept the plausibility of the hypothesis that I
> >am testing and make an effort to reject it. Failure to reject means that
> >it stays on the list of plausible truths, for now. The more failures to
> >reject the greater my confidence in the plausibility of the hypothesis.
>
> No! (or, at least, *my* confidence would not necessarily increase.)
> Suppose that the mean IQ of a group is in fact 100 with standard deviation
> 15; and I, using sample sizes of 10 repeatedly, try to test the hypothesis
> that the mean is 102. I will do this 1000 times (MINITAB simulation:
>
> MTB > random 1000 c1-c10;
> SUBC> normal 100 15.
> MTB > rmean c1-c10 c15
> MTB > rstd c1-c10 c16
> MTB > let c17 = abs(c15-102) < c16*2.262/sqrt(10)
> MTB > mean c17
> mean of c17 = 0.932
>
> and we see that 93% of my experiments fail to reject H0. Should I now
> believe it to be true?
In this case you have selected an example where the null was a priori
false and showed that low power fails to detect a false null.
The problem with this example is that it fails to provide what I refered
to earlier as a good faith effort to falsify the hypothesis. That is I
also believe that power (precision) is a necessary component of the
approach. Use the same data to compute interval estimates and you come to
no better conclusion.
>
> The problem is that failure to reject means *either* that the null is
> true *or* that the sample size too small *or* both; and a classical
> hypothesis test on its own does not tell you which.
If the classical hypothesis test has an a priori false null then only the
second conclusion is warranted. Only if the null is plausible are all
three interpretations legitimate. Since I advocate plausible nulls I
accept all three interpretations.
The a priori false null told you even less than that, so we have improved
the situation.
Interval estimates? They are still plagued by problems of low power
although those problems are much more apparent than in a hypothesis
testing approach. I like interval estimates because they give me a good
range for my plausibly true values for the null.
>
> -Robert Dawson
Michael
>
>
*******************************************************************
Michael M. Granaas
Associate Professor [EMAIL PROTECTED]
Department of Psychology
University of South Dakota Phone: (605) 677-5295
Vermillion, SD 57069 FAX: (605) 677-6604
*******************************************************************
All views expressed are those of the author and do not necessarily
reflect those of the University of South Dakota, or the South
Dakota Board of Regents.
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
