On Sat, 3 Mar 2001, dennis roberts wrote:
> when we discuss things like power, beta, type I error, etc. ... we
> often show a 2 by 2 table ... similar to
>
> null true null false
>
> retain correct type II, beta
>
> reject type I, alpha power
Similar, but not the same.
I usually present a table correct error: Type II
of "states of affairs",
without probabilities; error: Type I correct
see table at right.
(And usually with the rows interchanged, so that "Type I error" LOOKS
like the first kind of error one encounters.) It seems to me that to
include the probabilities in the same 2x2 table as the "states of
affairs" would be actively to invite rampant (or at least, and more
alliteratively, couchant) confusion of the concepts.
I have another problem with writing "power" in the lower right cell,
apart from the fact that it's a probability and not a state of affairs.
I'm aware that many people think of power as a conditional probability
(of rejecting the null when it's false); but I came to understand it as
an UNconditional probability (of rejecting the null, period). This
definition permits drawing power curves that include the parameter value
specified by the null hypothesis: the power at that point (or, in that
case) is alpha. For a symmetric two-sided alternative, this is also the
minimum value of power. Since the value of power approaches alpha as the
parameter value approaches the value specified in the null hypothesis, it
seems a little silly to omit that one point from the continuous curve.
> i think that we need a bit of overhaul to this typical way of doing
> things ...
>
> 1. each cell needs to have a name ... label ... that reflects the
> consequence of the decision (retain, reject) that was made
>
> i propose something along the lines of
>
> null true null false
>
> retain type I correct, 1C type II error, 2E
>
> reject type I error, 1E type II correct, 2C
I've long been persuaded of the need to distinguish between the two
different kinds of errors. That there are two distinct kinds is not at
all obvious, evidently; some folks seem never to master the distinction.
But I am not convinced that we need to distinguish between two kinds of
correct decision. After all, the decisions themselves are different:
to reject, or to retain (though some folks prefer "accept" to "retain").
Knowing the decision, and that it is (at least hypothetically) correct,
is surely all one needs to know. "Correct rejection" or "correct
retention" (or "acceptance") of the hypothesis being tested seems to me
easier to handle and apprehend than "a Type I correct decision" or "a
Type II correct decision".
> then, we have names or symbols for probabilities attached to each cell
>
> null true null false
>
> retain WHAT NAME/SYMBOL FOR THIS?? beta
>
> reject alpha power
If you want to construct such a table, I'd recommend including the
marginal row, showing the column totals to be 1 (or, if one prefers,
100%). That helps to emphasize the conditional nature of the
probabilities being displayed: conditional on the state of nature, not
on the decision. And consistent with my understanding of power, I'd
present such a table thus:
State of nature
null true null false
P{retain} 1 - alpha beta
Power alpha 1 - beta
---------- --------
Total 1 1
Sometime along about now one really ought to point out that a 2x2 table
like this is grossly oversimplified. Beta (and therefore power) cannot
be evaluated for "null false". It can be evaluated only for a specified
particular value of the parameter that is different from the value
specified in the null hypothesis. And, ceteris paribus, the farther that
parameter value is from the null-hypothetical value, the smaller is beta
(and the larger is power). This leads more or less directly to the idea
of a power curve, and then to the variations in such a curve as a
function of alpha and sample size.
> DOES ANYONE HAVE SOME SUGGESTION AS TO HOW THE UPPER LEFT CELL MIGHT BE
> REFERRED TO via A SYMBOL??? OR, SOME NAME THAT IS DIFFERENT FROM POWER
> BUT ... STILL GIVES THE FLAVOR THAT A CORRECT DECISION HAS BEEN MADE
> (better than making an error)?
Do you have a reasoned objection to "1 - alpha"? In other contexts we
routinely use, e.g., "1 - Rsq" for the proportion of variance unexplained
by the model being considered. The "1 minus" construction shows the
logical and arithmetical connection between two quantities, which can
easily get lost if one uses very different-looking terms for those
quantities.
> 2. i think it would be helpful to first identify each cell with a
> distinctive label ... describing the decision (correct, error) and ...
> the type ... 1 or 2
>
> 3. i think it would be helpful to have a system where there are names
> for EACH cell (why should the poor upper left be "left" out in the
> cold??) ... FIRST ... then some OTHER name/symbol for the probability
> associated with that cell
>
> confusions that might be avoided would be like:
>
> a. saying type II error is the same as beta ...
As remarked above, this confusion may well be _engendered_ by setting up
2x2 tables that contain BOTH the state of affairs and the probability
thereof in each cell.
> b. saying that power is NOT a name for a decision but, rather, THE
> probability of making some particular decision
Now _I'm_ confused. In context, Dennis appears to be claiming that the
statement in "b." is a "confusion", or untrue. But the statement is
perfectly true. What "confusion" is it supposed to represent?
> we have special names for errors of the first and second kind ....
> type I and type II ... and we have symbols of alpha and beta to
> represent their associated probabilities
> we have power which is supposed to be the probability of making a
> certain kind of decision ... but, no special name for THAT cell like we
> have given to differentiate the two kinds of errors one can make ...
I'm not convinced that we need a special name for "1 - alpha". But I'd
be interested in people's reactions to using "confidence" or "confidence
level" for this probability. After all, the confidence level of a
confidence interval is ordinarily 1 - alpha, so the usage would at least
be consistent in that sense.
(In the service of Rule 17-A -- "Always state the obvious; just in case
it wasn't" -- my interest is in reasoned and/or logical reactions to the
proposed terminology, not impassioned emotional objections. Though I
suppose those too might be interesting, in their own inimitable way.)
-- Don.
----------------------------------------------------------------------
Donald F. Burrill [EMAIL PROTECTED]
348 Hyde Hall, Plymouth State College, [EMAIL PROTECTED]
MSC #29, Plymouth, NH 03264 (603) 535-2597
Department of Mathematics, Boston University [EMAIL PROTECTED]
111 Cummington Street, room 261, Boston, MA 02215 (617) 353-5288
184 Nashua Road, Bedford, NH 03110 (603) 471-7128
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
