I have a question of my own - I think FAMILY is not, in general, the
same as EXPERIMENT.
On 01 Aug 2000 19:54:26 GMT, [EMAIL PROTECTED] (EAK5COLUMB) wrote:
> I am trying to find a definition of "familywise error".
> Am running a number of regressions on the same set of data. Assume "familywise
> error" means the chance of Type I error increases.
>
> Is a solution Bonferroni? Are there other solutions?
"familywise error" is contrasted to "comparisonwise error."
An online search with Google shows that "experimentwise error" is
often used exactly the same as "familywise error" and large number of
references (online) happen to use one term or the other. The
definition for each is, approximately: the chance rejecting at least
once, by chance, across a family of tests.
I spent most of an hour in scanning --
A number of sites use both terms, and merely say that they are the
same. Just a few references *seem* to draw a distinction, such as
an outline for lecture notes that includes separate topics for
Experimentwise and Familywise. But I never saw one that spelled out
the difference. So I would be happy to hear if people agree with what
I am about to opine, or (especially) if someone has a good source.
Familywise, and often Experimentwise: Most of the procedures
discussed on the pages assume that there was an overall ANOVA. Then,
the common error term is used for various contrasts between means.
Compared to Bonferroni testing, here is a distinct thing about
Scheffe's, Tukey's, etc. These are defined from a sampling
distribution of "correlated" tests, which are correlated because,
usually: (a) the same means are compared several times, and
always: (b) the same MS-error is the denominator.
(The need for a common MS-error is one of the reasons that the
Followup tests do not generalize readily to ANOVAs with repeated
measures, where paired-t tests are generally appropriate.)
These tests under one ANOVA make up a "family", in my opinion. (I
think that I garnered this opinion while reading some of the original
literature on these followup tests.)
The tests in one big ANOVA *might* define all of the tests in one
Experiment. Or, there might be multiplicity that is not in one ANOVA.
On the other hand, if you set up a multi-way ANOVA, I might be willing
to define each of two major factors as a separate "experiments" (for
instance, "drug" and "social therapy".) So, I have definitions for
tests in one Experiment, or tests in a Family, that I link to other
meanings of the words, and I keep them separate.
As to the question: running a number of regressions on one set does
not give a single ANOVA family, in the sense that I have described it.
So, if correction is needed, it is something on the order of
Bonferroni.
It might be that Bonferroni correction is needed, but quite often, for
various questions, it is not. - If there is an overall effect that
is clearly shown, the experimenter's problem is not one of
"significance" of the individual model, but rather, one of describing
the effects.
--
Rich Ulrich, [EMAIL PROTECTED]
http://www.pitt.edu/~wpilib/index.html
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
