Hi Dan,

I’m having a bit of trouble figuring out exactly how your two comments comport 
with one another, but I think the crucial point here is that the procedure I 
outline in the paper is simply how to do exactly what is done in traditional 
ANOVA analyses.  In this approach, the expected effect size of, for example, 
ListenerType does not depend on the relative amounts of data in the various 
levels of SyntaxType (which is what I think you’re referring to by “balance of 
the levels”).

Your caveat regarding whether the main effect of a factor X1 necessarily has a 
sensible interpretation in the presence of the interaction between X1 and X2 is 
certainly appropriate.  In the beginning of the paper I have a few remarks on 
the caution that should be applied.  I do think that for factorial ANOVA 
analyses the main effect can often have a useful interpretation as the 
“across-the-board” effect that X1 has, regardless of the value of X2 (which 
once again is the traditional ANOVA interpretation of a main effect).

If my responses to your comments aren’t on target, I would be very glad for 
clarification!

Best

Roger


On Sep 19, 2016, at 7:31 PM, Daniel Ezra Johnson 
<danielezrajohn...@gmail.com<mailto:danielezrajohn...@gmail.com>> wrote:

If you follow this procedure, though, you'd be testing for the effect of 
Listener when SyntaxType is "in the middle" (unweighted) of the three levels. 
This quantity not only has no sensible interpretation, it also depends on the 
balance of the levels of SyntaxType in the data.

If the effect of Listener is in the same direction for each level of 
SyntaxType, such a test might be useful, but otherwise I don't think it would 
be?

Dan

On Sep 19, 2016, at 8:00 PM, Daniel Ezra Johnson 
<danielezrajohn...@gmail.com<mailto:danielezrajohn...@gmail.com>> wrote:



 it also depends on the balance of the levels of SyntaxType in the data.

Well not quite. The idea of testing the "middle level" is right, and whatever 
this means doesn't change if the balance of data changes across levels...

But you could have two data sets where the effects of listener for each level 
of SyntaxType are the same (between the two data sets), but the significance of 
this test changes…


On Mon, Sep 19, 2016 at 2:45 PM, Levy, Roger 
<rl...@ucsd.edu<mailto:rl...@ucsd.edu>> wrote:
Hi Rachel,

If your goal is to test the main effect of Listener in the presence of the 
Listener-SyntaxType interaction, as would typically be done in traditional 
ANOVA analyses, I recommend you read this brief paper I wrote a few years ago 
on how to do this:

  http://arxiv.org/abs/1405.2094

It is exactly targeted at this problem, and explains why you’re getting the 
behavior you report due to differences in how R treats factors versus numeric 
variables in formulae.  (Setting the contrasts on the factor has no impact.)

I have no explanation for your reported behavior of why you don’t get this 
problem when you test for the main effect of SyntaxType; if you give further 
details, we might be able to help further!

Best

Roger

On Sep 18, 2016, at 5:57 PM, Rachel Ostrand 
<rostr...@cogsci.ucsd.edu<mailto:rostr...@cogsci.ucsd.edu>> wrote:

Hi everyone,

I'm having trouble with some 2-factor glmer models that I'm trying to run, such 
that the model with one of the main effects removed is coming out identical to 
the full model. Some colleagues suggested that this might be due to the coding 
of my factors, specifically because I have a factor that has 3 levels, and that 
one needs to be treated differently, but I'm not sure how - or why - to do that.

Brief summary of my data:
-My DV (called Target_E2_pref) is a binary categorical variable.
-There are two categorical IVs: Listener (2 levels) and SyntaxType (3 levels).
-Listener varies by both subject and item (i.e., picture); SyntaxType only 
varies by subject.

When I dummy coded my variables using contr.treatment(), the model with the 
main effect of Listener removed from the fixed effects comes out identical to 
the full model:

SoleTrain = read.table(paste(path, "SoleTrain.dat", sep=""), header=T)
SoleTrain$Listener.f = factor(SoleTrain$Listener, labels=c("E1", "E2"))
contrasts(SoleTrain$Listener.f) = contr.treatment(2)
SoleTrain$SyntaxType.f = factor(SoleTrain$SyntaxType, labels=c("Transitive", 
"Locative", "Dative"))
contrasts(SoleTrain$SyntaxType.f) = contr.treatment(3)

# Create full model:
SoleTrain.full<- glmer(Target_E2_pref ~ Listener.f*SyntaxType.f + (1 + 
Listener.f*SyntaxType.f|Subject) + (1 + Listener.f|Picture), data = SoleTrain, 
family = binomial, verbose=T, control=glmerControl(optCtrl=list(maxfun=20000)))

# Create model with main effect of Listener removed:
SoleTrain.noListener<- glmer(Target_E2_pref ~ SyntaxType.f + 
Listener.f:SyntaxType.f + (1 + Listener.f*SyntaxType.f|Subject) + (1 + 
Listener.f|Picture), data = SoleTrain, family = binomial, verbose=T, 
control=glmerControl(optCtrl=list(maxfun=20000)))

> anova(SoleTrain.full, SoleTrain.noListener)
Data: SoleTrain
Models:
SoleTrain.full: Target_E2_pref ~ Listener.f * SyntaxType.f + (1 + Listener.f * 
SyntaxType.f | Subject) + (1 + Listener.f | Picture)
SoleTrain.noListener: Target_E2_pref ~ SyntaxType.f + Listener.f:SyntaxType.f + 
(1 + Listener.f * SyntaxType.f | Subject) + (1 + Listener.f | Picture)
                     Df    AIC    BIC  logLik deviance Chisq Chi Df Pr(>Chisq)
SoleTrain.full       30 2732.5 2908.5 -1336.2   2672.5
SoleTrain.noListener 30 2732.5 2908.5 -1336.2   2672.5     0      0          1

However, I don't have this problem when I test for the main effect of 
SyntaxType, and remove the SyntaxType.f factor from the fixed effects. (That 
is, this produces a different model than the full model.)

Someone suggested that Helmert coding was better for factors with more than two 
levels, so I tried running the same models except with Helmert coding 
[contrasts(SoleTrain$SyntaxType.f) = contr.helmert(3)], but the models come out 
identical to the way they do with dummy coding. So why does the model with the 
main effect of Listener removed the same as the model with the main effect of 
Listener retained?

Any suggestions as to what I'm doing wrong?

Thanks!
Rachel



Reply via email to