"Werner W. Wittmann" wrote:
>
> > Herman and N.N.,
> > type III error means measuring the wrong construct or something
> > nonexistent.In my German book about evaluation research(1985) I cited the
> > following:
> > "Statistician worry about two types of errors......:
> > Type I error is rejecting a hypothesis when it should be accepted; Type II
> > error is accepting a hypothesis when it should be rejected ...
> > Evaluators commonly make two types of errors doing evaluations: Type III
> > error is measuring something that does not exist; Type IV error is
> measuring
> > something that is of no interest to management and policy maker."
I always worry about this when it gets beyond being an "in-joke" like
the Eleventh Commandment ("Thou shalt not get found out.") and reaches a
wider audience, because of the pedagogical risks of conflating two sorts
of "error". The terms "Type I error" and "Type II error" do seem to
lead novices to think of them as errors in the sense of something that
wouldn't have happened if you'd been more careful or sensible; and
lumping them with genuine errors in this sense compounds the confusion.
The fault is perhaps with the original terminology, not the joke. The
terms "Type N error" for N=I to II are a truly dreadful example of bad
coinage; they are practically impossible to define directly and almost
everybody who tries gets it wrong.
A Type I error is NOT 'rejecting a hypothesis when it should be
accepted'. A hypothesis test is a (byzantine and often wrong-headed, but
let that pass) process that converts data to decisions. If the data
point to rejection - even if the data happen to be atypical of the
population they represent - then the hypothesis "should" be rejected. If
you knew the population parameter ahead of time yu wouldn't have done
the test!
Indirect definitions such as "in such a case we shall say that a Type I
error has occurred" are less likely to be utterly wrong, but are not
much more satisfactory as a definition of "Type I error" than "When I
watch the news I tell myself that the world is going to the dogs"" is
satisfactory as a definition of "dog". Yes, the second usage is an idiom
that cannot be understood at the single-word level... so, essentially,
is the first.
In fact, the term "a Type I error" does not have a syntactically
self-contained definition at all. It is a "macro" that cuts across the
syntax of natural language and of probability, and its real meaning is
something like
rejecting a hypothesis, conditional upon its being true
which is only meaningful when you plug it into
"the probability of (-----)"
So you have something that appears syntactically to be an unconditional
probability, and which is, in fact, when "expanded", conditional upon a
certain parameter value!
This is like the sort of abuse of the C preprocessor that some
self-taught programmers revel in in which (for instance) you define "k"
to be "j++". Not only does the fake variable "k" yield a value while
apparently never being initialized, but it invisibly changes its value
(and that of j) every time it's used, so that for instance the equality
test "k == k" returns 0 (false)! Of course, programmers who insist on
doing things like this are unwelcome on teams trying to write
maintainable code.
In the case of Type I errors, it is not uncommon for students to take
"the probability of a Type I error" to be P( H_0 is rejected and mu =
mu_0) rather than P(H_0 is rejected | mu=mu_0). The fact that the first
probability is 0 (usually, to a bayesian) or undefined (to a
frequentist) does not stop them. Perhaps one of the reasons is that (
H_0 is rejected and mu = mu_0) at least _sounds_ like something you
could find the probability of.
So maybe we should redefine the first two, and let a Type I error be
"rejecting a null hypothesis instead of estimating effect size" while a
Type II error can become "failing to reject a null hypothesis and
claiming you have shown theta = theta_0" <grin>?
-Robert Dawson
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
