Bob: Here are a few thoughts on your comments:
> This might shock some people, but AIC does not give The Truth. Of course not; one cannot hope to find full truth from finite samples. > If you > have a model that fits almost as well, but is simpler, then I don't > see a problem with using it. You could use the more simple model if you wished, but it is far better to model average (not to mention that you could properly treat model selection uncertainty as part of the measure of precision). >It's worth checking how much less of the > variation is explain (e.g. using R^2), and also how different the > fitted models are. Adj R-sq might be worth mention; however, it is a very poor way to select models. Adj R-sq applies just to the fitted data set and is not able to reflect out-of-sample prediction. Akaike's predictive likelihood goes beyond this. > AIC has a tendency to give overly complex models (especially with > lots of data), What you say is correct; however, that is exactly why people should be using AICc -- afterall that was first published some 33 years ago (1978)! >so I often use BIC instead, which tends too far in the > other direction. BIC is very poor; disliked by likelihoodists and Bayesians alike. Its underlying assumptions are absurd. It is not related to information theory in the slightest. Almost a hoax. This is a poor approach. >Or, if the full model isn't too big, I don't bother > with model selection, and report the full model. Again, a very poor approach to valid inference unless the sample size is very large. I hope these comments will be useful. David Anderson
