On Fri, 17 Mar 2000 16:33:44 -0500, Bruce Weaver
<[EMAIL PROTECTED]> wrote:
< snip, ... my answer to a question >
> Rich,
> Is there not an important distinction to be made between the
> following situations:
>
> 1. A computer algorithm determines (based on the magnitude of partial or
> semi-partial correlations) the order in which variables are entered or
> removed, and which ones end up in the final model
>
> 2. The investigator determines a priori the order in which variables are
> to be entered or removed.
>
Sure. #1 is totally wretched, if you are hoping that it will tell
you something important about the variables. Except, assuming that
all the predictors are useful, "What is a useful, small set?"
And, #2 is a pragmatic way to devise testing. In my experience,
several of the variables being forced in first are for control and are
irrelevant to considerations of "number of tests being performed." If
gender is highly significant (say), including it serves to reduce the
error residual, and including it is necessary if there might be
confounding. But I can't include its significance to justify the
success of my experiment ... see below .
< snip, detail >
> Here's another observation that is relevant to this thread, I think. When
> one performs a 2-factor ANOVA, there are 3 independent F-tests: one for
> each main effect, and one for the intereaction. One can arrive at these
> same F-tests using the same regression model comparison approach that is
> described above (e.g., compare the FULL regression model to one without
> the AxB interaction to get F for the interaction term). I don't think
> I have EVER seen anyone correct for multiple comparisons in this case.
...
Well, SPSS does provide *two* versions of overall tests to control
for multiple comparisons, and I have used them both. There is the
test on "main effects," and also a test on the whole model. It is
legitimate to use one test, the other, or neither, depending on your
questions and assumptions.
Most often, there is just one Main effect that is being "tested" and
that we had written about as being of experimental *interest*
beforehand. There are covariates and other technical main-effects in
the analysis which will need to be mentioned, and might interest
someone, but they did not justify the experiment. So, their outcomes
do not justify the experiment, or go into an Overall test.
Or, there could be two main effects that *each* justify the
experiment. In that case, which seems to be what I hope you are
referring to, there are separate tests performed, without any
correction for being multiple tests.
I have had cause to use the Main effects test, when there were two
treatments that were being subsumed under the same experiment-wise
error -- this leaves out the Interaction. Usually, I regard the
Interaction as a disqualifying test; I have to live with it, if it
happens to meet the 5% level, all by itself. So, I worry about it,
but I don't include it in my Hypothesis test -- unless it has been
elevated, beforehand, to be a separate aspect of the main hypothesis.
--
Rich Ulrich, [EMAIL PROTECTED]
http://www.pitt.edu/~wpilib/index.html
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
