Classification accuracy is an improper scoring rule, and one of the
problems with it is that the proportional classified correctly can be
quite good even if the model uses no predictors. [Hence omitting the
intercept is also potentially problematic.]
Frank E Harrell Jr Professor and Chairman School of Medicine
Department of Biostatistics Vanderbilt University
On Wed, 11 Aug 2010, Michael Scharkow wrote:
Dear all,
I have some growth curve data from an experiment that I try to fit using
lm and lmer. The curves describe the growth of classification accuracy
with the amount of training data t, so basically
y ~ 0 + t (there is no intercept because y=0 at t0)
Since the growth is somewhat nonlinear *and* in order to estimate the
treatment effect on the growth curve, the final model is
y ~ 0 + t + t.squared + t:treat + t,squared:treat
this yields:
t t.sq t:treat t.sq:treat
1.08 -0.007 0.39 -0.0060
This fits the data fairly well, but I have replicated data for 12
different classifiers. First, I tried 12 separate regressions which
yielded results with different positive values for t and t:treat.
Finally, I tried to estimate a varying intercept model using lmer
lmer(y ~ 0+t+t.squared+t:treat+t,squared:treat+(0+t+t.squared+t:treat
+t,squared:treat | classifier)
The fixed effects are similar to the pooled regression, but most of the
random effects for t and t:treat are implausible (negative). What's
wrong with the lmer model? Did I misspecify something?
Greetings,
Michael
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
