There are a couple of strange things about the description of the
scenario.
First, the idea of thinking about a model with almost the same AIC
(or, better, AICc) but fewer terms, in pursuit of "parsimony" is doing
parsimony twice. The AIC already accounts for the relative number of
parameters. If the model with fewer parameters has a worse AIC, the
result is saying that the better model is better even though it has
more parameters. And, advantageously, it is doing it in an objective
way rather than some subjective feeling about parsimony.
Second, in regard to Bob's reply to the scenario, R^2 is a really weak
tool for comparing models. You can always improve R^2 by adding more
terms. The value of information criteria (or at least one of the
values) is escaping from that bind in a satisfactory way.
Finally, in regard to Davis's question about what to report, in at
least some parts of the literature, it is standard to report all the
models evaluated, ranked by their Delta AIC. That way the reader can
judge how much better the best model is than the second, third, etc.
best. But in the end, you need a model to use. Model averaging is a
really good procedure here, and it seems a little strange to rule it
out.
Hal Caswell
On Dec 2, 2010, at 10:40 AM, Bob ohara wrote:
This might shock some people, bit AIC does not give The Truth. If
you have a model that fits almost as well, but is simpler, then I
don't see a problem with using it. It's worth checking how much less
of the variation is explain (e.g. using R^2), and also how different
the fitted models are.
AIC has a tendency to give overly complex models (especially with
lots of data), so I often use BIC instead, which tends too far in
the other direction. Or, if the full model isn't too big, I don't
bother with model selection, and report the full model.
HTH
Bob
Bob O'Hara
Tel: +49 69 798 40226 (in Germany)
Mobile: +49 1515 888 5440
WWW: http://www.bik-f.de/root/index.php?page_id=219
Blog: http://blogs.nature.com/boboh/
Journal of Negative Results - EEB: www.jnr-eeb.org
Lee Davis <[email protected]> 12/01/10 23:44 PM >>>
I have what might seem to be a simple question regarding AIC and
parsimony,
and yet the answers I have found on the subject are unsatisfactory.
So,
opinions please.
Here is the scenario:
Let's say that one is using AIC for the selection of nested models
to avoid
multiple LRT comparisons. Should you always choose the model with
deltaAIC =
0 as the best? What if there is a model with deltaAIC <2 that has
fewer
terms? Should it be chosen in the pursuit of parsimony? Or should
you report
some support for both models? If so, what is the proper language in
this
case?
Let's assume that we are avoiding model averaging.
Thanks,
Lee
--
Lee Davis
Graduate Assistant
State University of New York
College of Environmental Science & Forestry
Department of Environmental & Forest Biology
452 Illick Hall, 1 Forestry Drive, Syracuse, NY 13210
---------------------------------
Hal Caswell
Senior Scientist
Biology Department
Woods Hole Oceanographic Institution
Woods Hole MA 02543
508-289-2751
[email protected]
