Hmmm. Let's see.
On Dec 6, 2010, at 6:02 PM, Seth W Bigelow wrote:
With respect to Hal Caswell's comment that:
"First, the idea of thinking about a model with almost the same AIC
(or, better, AICc) but fewer terms, in pursuit of "parsimony" is doing
parsimony twice. The AIC already accounts for the relative number of
parameters. If the model with fewer parameters has a worse AIC, the
result is saying that the better model is better even though it has
more parameters."
Models within 2 AICc units of one another have virtually the same
level of
support from the data,
"Virtually the same" is not "the same". There is nothing magical
about Delta AIC=2. If one really wants to deal with the nearly (but
not exactly) identical degrees of support, the proper way to do it is
via model averaging. In my experience, that can provide exactly what
we are really looking for here --- a way to account for the
uncertainty in estimation within a model, and the uncertainty in which
model is better.
so in my view it still makes sense to proceed with
the simpler model. By "proceed" I mean estimating and reporting
parameters
and confidence intervals. When I've taken the time to estimate
parameter
confidence intervals on simple and more complex models within 2 AICc
units
of each other, I've usually found that at least one of the parameter
confidence intervals of the complex models encompasses zero. And one
feels
pretty silly trying on the one hand, to claim to have found support
for a
model, yet on the other hand having to admit that one can't
distinguish
one of its parameters from zero.
This is a bad case of mixing significance testing with model
selection. Model selection is based on likelihood, not on confidence
intervals. If a model including term X is better than one excluding
term X, then that is telling you that it is more well supported by the
data than the one excluding term X, regardless of the confidence
intervals.
The shift from a signficance-testing perspective to a model-support
perspective is, I believe, much more important than people tend to
appreciate. The literature is full of mixtures of AIC and confidence
intervals and significance tests in ways that violate the assumptions
of all three approaches.
Hal
Dr. Seth W. Bigelow
Biologist, USDA-FS Pacific Southwest Research Station
1731 Research Park Drive, Davis California
[email protected] / ph. 530 759 1718
---------------------------------
Hal Caswell
Senior Scientist
Biology Department
Woods Hole Oceanographic Institution
Woods Hole MA 02543
508-289-2751
[email protected]
