At 05:42 AM 6/12/2009, Lindsay Banin wrote:
Hi there,
I am looking to compare nonlinear mixed effects models that have
different nonlinear functions (different types of growth
curve)embedded. Most of the literature I can find focuses on
comparing nested models with likelihood ratios and AIC. Is there a
way to compare model fits when models are not nested, i.e. when the
nonlinear functions are not the same?
Transform back into original units, if necessary, and compare
distributions of and statistics of residuals from fitted values in
original units.
This is not a significance-test, but instead a measure of the better
approximation to the observed model.
Types of measures: 1) rms residual, 2) max absolute residual, 3) mean
absolute residual.
In my opinion, models should be chosen based on the principles of
causality (theory), degree of approximation and parsimony. None of
these involve significance testing.
Choosing models based upon significance testing (which merely
identifies whether or not the experiment is large enough to
distinguish an effect clearly) amounts to admitting intellectually
that you have no subject matter expertise, and you must therefore
fall back on the crumbs of significance testing to get glimmers of
understanding of what's going on. (Much like stepwise regression techniques.)
As an example, suppose you have two models, one with 5 parameters and
one with only 1. The rms residual error for the two models are 0.50
and 0.53 respectively. You have a very large study, and all 4
additional parameters are significant at p = 0.01 or less. What
should you do? What I would do is select the 1 parameter study as my
baseline model. It will be easy to interpret physically, will
generalize to other studies much better (stable), and is almost
identical in degree of approximation as the 5 parameter model. I
would be excited that a one parameter model could do this. The fact
that the other 4 parameters have detectable effects at a very low
level is not important for modeling the study, but may conceivably
have some special significance on their own for future investigations.
So not being able to do significance testing on non-nested models is
not that big a loss, in my opinion. Such tests encourage wrong
thinking, in my opinion.
What I've expressed as an opinion here (which I am sure some will
disagree with) is similar to the philosophy of choosing the number of
principal components to use, or number of latent factors in factor
analysis. What investigation do people ever do on the small
eigenvalue principal components, even if their contributions are
"statistically significant"?
================================================================
Robert A. LaBudde, PhD, PAS, Dpl. ACAFS e-mail: r...@lcfltd.com
Least Cost Formulations, Ltd. URL: http://lcfltd.com/
824 Timberlake Drive Tel: 757-467-0954
Virginia Beach, VA 23464-3239 Fax: 757-467-2947
"Vere scire est per causas scire"
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
