On 9/5/05, Thomas Petzoldt <[EMAIL PROTECTED]> wrote: > Dear expeRts, > > there is obviously a general trend to use model comparisons, LRT and AIC > instead of Wald-test-based significance, at least in the R community. > I personally like this approach. And, when using LME's, it seems to be > the preferred way (concluded from postings of Brian Ripley and Douglas > Bates' article in R-News 5(2005)1), esp. because of problems with the > d.f. approximation. > > But, on the other hand I found that not all colleagues are happy with the > resulting AIC/LRT tables and the comparison of multiple models. > > As a compromise, and after a suggestion in Crawley's "Statistical > computing" one may consider to supply "traditional" ANOVA tables as an > additional explanation for the reader (e.g. field biologists). > > An example: > > one has fitted 5 models m1..m5 and after: > > >anova(m1,m2,m3,m4,m5) # giving AIC and LRT-tests > > he selects m3 as the most parsimonious model and calls anova with the > best model (Wald-test): > > >anova(m3) # the additional explanatory table
Whether or not this is a good idea will depend on what the differences in the models are. Two mixed-effects models for the same data set can differ in their random effects specification or in the fixed-effects specification or both. The anova() function applied to a single lmer model produces a sequential anova table for the fixed-effects terms. If the models differ in the random effects specification - say the full model has random effects for slope and intercept but the restricted model has a random effect for the intercept only - then a Wald test is not appropriate (and it is not provided). In those cases I would use a likelihood ratio test and, if necessary, approximate the p-value by simulating from the null hypothesis rather than assuming a chi-squared distribution of the test statistic. Recent versions of the mlmRev package have a vignette with extensive analysis of the Exam data, including MCMC samples from the posterior distribution of the parameters. The marginal posterior distribution of the variance components are quite clearly skewed (not surprisingly, they look like scaled chi-squared distributions). Testing whether such a parameter could be zero by creating a z-statistic from the estimate and its standard error is inappropriate. Changing both the fixed-effects and the random-effects specification is tricky. I would try to do such changes in stages, settling on the fixed-effects terms first then checking the random-effects specification. > > My questions: > > * Do people outside the S-PLUS/R world still understand us? > > * Is it wise to add such an explanatory table (in particular when the > results are the same) to make the results more transparent to the reader? > > * Are such additional ANOVA tables *really helpful* or are they (in > combination with a model comparison) just another source of confusion? > > > Thank you! > > Thomas P. > > ______________________________________________ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > ______________________________________________ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
