H Roger and Florian, thanks for your replies - I wanted to follow up on a few things:
On 10 Oct 2007, at 18:39, Roger Levy wrote:
1) you can always use cross-validation.
T. Florian Jaeger wrote:
predictive power? you can, of course, use predict() [you have to write your own version if you want to use the random effects]. and then use cross-validation to get a measure for unseen data.
Sure, and then use a non-parametric test to compare the two accuracies, or can we even assume (Central Limit Theorem) that the means obtained through cross-validation are approximately normally distributed?
Regarding log-likelihood and BIC, I also found a paper by Song & Lee: Model comparison of generalized linear mixed models Xin-Yuan Song, Sik-Yum Lee Statistics in Medicine 25 (10), pp 1685-1698, 2005 http://www3.interscience.wiley.com/cgi-bin/abstract/112100165/ABSTRACT .. which I'll have to look at.Thanks for your hint regarding quasi-likelihood and Laplace fits. I might go with that for now.
On 10 Oct 2007, at 18:39, Roger Levy wrote:
2) I've recently become aware of the "Vuong test" (Vuong 1989, Econometrika):http://links.jstor.org/sici?sici=0012-9682(198903)57:2%3C307:LRTFMS% 3E2.0.CO;2-Jbut I haven't had a chance to learn what it is. It might be useful. Would love to hear what you learn if you have a chance to look into it.
Vuong uses the Log-Likelihood Ratio to compare two non-nested models and discusses cases where models do not and do overlap partially. I note that one problematic assumption (A.1) in this paper is that all samples are i.i.d., which excludes mixed effects models with a grouping factor. So time-series and repeated measures experiments (by- subject and by-item analyses etc.) are not covered.
-- David Reitter ICCS/HCRC, Informatics, University of Edinburgh http://www.david-reitter.com
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ R-lang mailing list [email protected] http://pidgin.ucsd.edu/mailman/listinfo/r-lang
