At 01:56 PM 4/10/00 -0400, Donald F. Burrill wrote:
>On Mon, 10 Apr 2000, dennis roberts wrote in part:
>
> > .. the fact that we create a null and test a null does NOT imply that
> > we are therefore testing some effect size ...
>
>Of course not. One does not TEST an effect size, one ESTIMATES it.
>And it is useful to do so only if one has found it not equal to some
>value (possibly, but not necessarily, zero) that would imply the effect
>size to be uninteresting.
who says? i can just as interestingly (or just as UNinterestingly) ask:
what IS the effect size ... go about my estimate of it (can one not build a
CI on the estimated effect?) ... and THEN depending on what that estimate
is ... make some judgement of whether this 'effect' of 'this treatment' is
worth pursuing ... you don't have to test this FIRST against a null of 0
(thought to be the uninteresting value).
> (Sometimes it is convenient to do both of
>these things at once, as in constructing a confidence interval. But
>if the interval includes values that are, a priori, uninteresting, there
>is little utility to pursuing the current estimate of the effect size.)
why do you necessarily assume that an effect size of 0 is UNinteresting?
seems like if one 'thought' that psychotherapy has an effect ... BUT, found
that the an effect size of 0 is a possible value ... that could be very
interesting ... might make psychotherapists mad but, what the hey!
>I beg to differ. Strenuously.
the ONLY way to differ, right?
>So long as the logical style of scientific argumentation is argument by
>elimination,
this is the large fallacy ... that scientific progress is only made by way
of elimination of competing hunches ... or in the vernacular of hypotheses
... a null. have we not carried this burden on our shoulders long enough?
does this imply EITHER that if we eliminate all possible alternatives that
we can think of (and we can never think of them all) that what is left over
(our research hunch) is TRUE? OR ... only that what is left over may be
CLOSER to the truth? i would opt for the latter ... not the former.
the assumption seems to always be we are looking for THE truth ... well, we
can't EVER find that. what we can perhaps do is to offer explanations or
'theories' for phenomena that seem to make more and more sense ... but the
idea that the rejection of more and more null hypotheses (that never seem
to change) GETS us closer to a truth that we can never find anyway ... is a
real liability in science ...
it is NOT UNscientific to think that null hypothesis testing is severely
limited in what it can tell us about progress and truth ... but what is
true is that science and research have become so dominated by the notion of
... formulating SOME null ... and then hoping ... actually PRAYING that one
can reject it ... that we have this super glued set of blinders on to the
fact that it is highly overrated ...
a colleague of mine and i were discussing just the general notion of a
research investigation ... and, when you look at the documentation of it
... there is a written product ... that contains components like ...
introduction, literature, methods, analysis, and conclusions ... and,
WITHin analysis ... there might be 'significance' testing. well, in the
scheme of things ... it is but one itty bitty part of the overall process.
but, guess what?? not only has the 'significance testing' part of analysis
taken over almost all of the analysis section ... it has taken over almost
the entire paper! without it ... readers think the research effort has NO
value ... ie, the efficacy of the product depends almost solely on our p
values from our significance tests ...
we are going backwards folks ... we need to get OFF of this misguided train
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
