Michael (and others interested):
I like the confidence idea, though there can be confusion over what the
probabilities mean. Technically, when we reject the Null at .05, it's not true
there is a 95% chance of a real difference, given the data. It's that the
probability of the data _given the null_ is less than .05 (and assumptions are
met). The probability refers to the data, not the hypothes(es). To reverse the
thinking and think about the probability of the hypothes(es) given the data is a
Bayesian way to think.
I still prefer "reliability". When we do traditional hypothesis testing, the
null says "no difference." When we reject null, we are make a decision with a
certain level of confidence that a difference really exists. This confidence
leads us to the expectation that if the study were re-done, we'd come to the
same decision (i.e. reject null). If we view "reliability" this way, I think
it's acceptable (and it also captures your emphasis on confidence - which I
think is an important part). I was going to wait and read Stuart's recomended
articles before re-entering this debate, but, my impulsive fingers on the
keyboard got the best of me. I promise to read the counter-arguments soon. I
like the emphasis on confidence - as long as we interpret .05 and .95 correctly.
[EMAIL PROTECTED] wrote:
> What is wrong with "confidence level"? If we can reject the null hypothesis
> at the .05 level, we are 95% confident that a real difference exists.
>
> Michael B. Quanty, Ph.D.
> Psychology Professor
> Senior Institutional Researcher
> Thomas Nelson Community College
> PO Box 9407
> Hampton, VA 23670
>
> Phone: 757.825.3500
> Fax: 757.825.3807
>
> -----Original Message-----
> From: Michael J. Kane [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, September 28, 2000 12:33 PM
> To: John W. Kulig; DAP Louw (Sielkunde)
> Cc: [EMAIL PROTECTED]
> Subject: Re: Clinical vs statistical significance
>
> At 09:42 AM 9/28/00 -0400, John W. Kulig wrote:
>
> >(snip)
> >Say, isn't it time we revived our discussion about how awkward the term
> >"significance" is for p statements? For the "n th" time, wouldn't
> >_reliability_ be the better word? If p < .05, we conclude the results of
> >the study will repeat if the experiment was replicated. That, in my book,
> >is the definition of reliability. This was we can dispense, once and for
> >all, with adjectives before the word "significance."
>
> Hi John,
>
> I'd be interested to hear what other Tipsters have to say about this. I
> don't
> teach stats, but my hunch is that using the word "reliability" would likely
> lead students to understand that p = .05 means "if we repeated the study
> 100 times,
> we'd get this result 95 times", when in fact, p = .05 means that *if the
> null hypothesis
> were true,* then we'd only expect to get this finding 5 times out of a 100.
>
> -Mike
>
> ************************************************
> Michael J. Kane
> Department of Psychology
> P.O. Box 26164
> University of North Carolina at Greensboro
> Greensboro, NC 27402-6164
> email: [EMAIL PROTECTED]
> phone: 336-256-1022
> fax: 336-334-5066
--
---------------------------------------------------------------
John W. Kulig [EMAIL PROTECTED]
Department of Psychology http://oz.plymouth.edu/~kulig
Plymouth State College tel: (603) 535-2468
Plymouth NH USA 03264 fax: (603) 535-2412
---------------------------------------------------------------
"What a man often sees he does not wonder at, although he knows
not why it happens; if something occurs which he has not seen before,
he thinks it is a marvel" - Cicero.
