Classification accuracy is an improper scoring rule, and one of the problems with it is that the proportional classified correctly can be quite good even if the model uses no predictors. [Hence omitting the intercept is also potentially problematic.]
Frank E Harrell Jr Professor and Chairman School of Medicine Department of Biostatistics Vanderbilt University On Wed, 11 Aug 2010, Michael Scharkow wrote:
Dear all, I have some growth curve data from an experiment that I try to fit using lm and lmer. The curves describe the growth of classification accuracy with the amount of training data t, so basically y ~ 0 + t (there is no intercept because y=0 at t0) Since the growth is somewhat nonlinear *and* in order to estimate the treatment effect on the growth curve, the final model is y ~ 0 + t + t.squared + t:treat + t,squared:treat this yields: t t.sq t:treat t.sq:treat 1.08 -0.007 0.39 -0.0060 This fits the data fairly well, but I have replicated data for 12 different classifiers. First, I tried 12 separate regressions which yielded results with different positive values for t and t:treat. Finally, I tried to estimate a varying intercept model using lmer lmer(y ~ 0+t+t.squared+t:treat+t,squared:treat+(0+t+t.squared+t:treat +t,squared:treat | classifier) The fixed effects are similar to the pooled regression, but most of the random effects for t and t:treat are implausible (negative). What's wrong with the lmer model? Did I misspecify something? Greetings, Michael ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.