Hi Maureen,
On Aug 21, 2009, at 3:41 PM, Maureen Gillespie wrote:
Hi everyone,
I have been using weighted empircal logit linear regression (Barr,
2008) to analyze data from a number of agreement error production
experiments. (Just as a side note, I have run into lots of problems
trying to use logit mixed models for this data as errors are
extremely rare: certain conditions produce essentially no errors and
all other conditions rarely have higher than 15% error rates. If
anyone has a better solution than the emp.logit please let me know!)
Yes, I have had similar difficulties with logit models where at least
one condition is error-free. One thing you may be prone to running
into in these cases is the unreliability of the Wald z-statistic.
Search for "standard error is inflated" on this page:
http://userwww.sfsu.edu/~efc/classes/biol710/logistic/logisticreg.htm
That being said, I am running what is essentially a meta-analysis.
I have data from 5 experiments and 104 different items (some of
which appear in multiple experiments, some only appear in a single
experiment). My model has two continuous predictors and two random
effects (experiment and item).
lmer(emp.logit ~ IV1 + IV2 + (1|item) + (1|exp), data, weights)
When I run the model, my estimates, standard errors, and t-values
all appear reasonable (i.e., comparable to other single random
effect models I have run using this technique on similar data).
There is no colinearity or anything else to suggest that something
is wrong. But when I use pvals.fnc() to compute CIs and p values
for the estimates, I find that the experiment random effect has a
std. dev. of 0.0000 (5.0e-11 to be exact), and this seems to inflate
the CI of the intercept estimate (t = 17, but it's only marginal
significant w/ pvals from MCMC). If I run the same model excluding
the experiment random effect, estimates do not change and the CIs
and p values for the intercept appear normal. Strangely (or maybe
not) the two models have the exact same log likelihoods.
Is this just an extreme example of a random effect not being
necessary?
And, more on the conceptual end of things, why would a near-zero
st.dev. of a random effect inflate CIs w/MCMC sampling?
I'm not sure what you mean by inflating the CI -- do you mean making
the CI on the fixed-effect intercept larger than it is in a model
without the random effect of experiment?
With such a small random effect of experiment, the model probably *is*
telling you that you don't need it. Try comparing your model's
likelihood with the likelihood of a model that doesn't have the random
effect of experiment -- the likelihoods should be very similar.
(Technically it's best to use restricted maximum likelihood
(REML=TRUE) when doing this, but that is the default so it looks like
you're doing that already.
Best
Roger
Thanks in advance,
Maureen Gillespie
Northeastern University
_______________________________________________
R-lang mailing list
[email protected]
http://pidgin.ucsd.edu/mailman/listinfo/r-lang
--
Roger Levy Email: [email protected]
Assistant Professor Phone: 858-534-7219
Department of Linguistics Fax: 858-534-4789
UC San Diego Web: http://ling.ucsd.edu/~rlevy
_______________________________________________
R-lang mailing list
[email protected]
http://pidgin.ucsd.edu/mailman/listinfo/r-lang
