Hi Dan, I’m having a bit of trouble figuring out exactly how your two comments comport with one another, but I think the crucial point here is that the procedure I outline in the paper is simply how to do exactly what is done in traditional ANOVA analyses. In this approach, the expected effect size of, for example, ListenerType does not depend on the relative amounts of data in the various levels of SyntaxType (which is what I think you’re referring to by “balance of the levels”).

## Advertising

Your caveat regarding whether the main effect of a factor X1 necessarily has a sensible interpretation in the presence of the interaction between X1 and X2 is certainly appropriate. In the beginning of the paper I have a few remarks on the caution that should be applied. I do think that for factorial ANOVA analyses the main effect can often have a useful interpretation as the “across-the-board” effect that X1 has, regardless of the value of X2 (which once again is the traditional ANOVA interpretation of a main effect). If my responses to your comments aren’t on target, I would be very glad for clarification! Best Roger On Sep 19, 2016, at 7:31 PM, Daniel Ezra Johnson <danielezrajohn...@gmail.com<mailto:danielezrajohn...@gmail.com>> wrote: If you follow this procedure, though, you'd be testing for the effect of Listener when SyntaxType is "in the middle" (unweighted) of the three levels. This quantity not only has no sensible interpretation, it also depends on the balance of the levels of SyntaxType in the data. If the effect of Listener is in the same direction for each level of SyntaxType, such a test might be useful, but otherwise I don't think it would be? Dan On Sep 19, 2016, at 8:00 PM, Daniel Ezra Johnson <danielezrajohn...@gmail.com<mailto:danielezrajohn...@gmail.com>> wrote: it also depends on the balance of the levels of SyntaxType in the data. Well not quite. The idea of testing the "middle level" is right, and whatever this means doesn't change if the balance of data changes across levels... But you could have two data sets where the effects of listener for each level of SyntaxType are the same (between the two data sets), but the significance of this test changes… On Mon, Sep 19, 2016 at 2:45 PM, Levy, Roger <rl...@ucsd.edu<mailto:rl...@ucsd.edu>> wrote: Hi Rachel, If your goal is to test the main effect of Listener in the presence of the Listener-SyntaxType interaction, as would typically be done in traditional ANOVA analyses, I recommend you read this brief paper I wrote a few years ago on how to do this: http://arxiv.org/abs/1405.2094 It is exactly targeted at this problem, and explains why you’re getting the behavior you report due to differences in how R treats factors versus numeric variables in formulae. (Setting the contrasts on the factor has no impact.) I have no explanation for your reported behavior of why you don’t get this problem when you test for the main effect of SyntaxType; if you give further details, we might be able to help further! Best Roger On Sep 18, 2016, at 5:57 PM, Rachel Ostrand <rostr...@cogsci.ucsd.edu<mailto:rostr...@cogsci.ucsd.edu>> wrote: Hi everyone, I'm having trouble with some 2-factor glmer models that I'm trying to run, such that the model with one of the main effects removed is coming out identical to the full model. Some colleagues suggested that this might be due to the coding of my factors, specifically because I have a factor that has 3 levels, and that one needs to be treated differently, but I'm not sure how - or why - to do that. Brief summary of my data: -My DV (called Target_E2_pref) is a binary categorical variable. -There are two categorical IVs: Listener (2 levels) and SyntaxType (3 levels). -Listener varies by both subject and item (i.e., picture); SyntaxType only varies by subject. When I dummy coded my variables using contr.treatment(), the model with the main effect of Listener removed from the fixed effects comes out identical to the full model: SoleTrain = read.table(paste(path, "SoleTrain.dat", sep=""), header=T) SoleTrain$Listener.f = factor(SoleTrain$Listener, labels=c("E1", "E2")) contrasts(SoleTrain$Listener.f) = contr.treatment(2) SoleTrain$SyntaxType.f = factor(SoleTrain$SyntaxType, labels=c("Transitive", "Locative", "Dative")) contrasts(SoleTrain$SyntaxType.f) = contr.treatment(3) # Create full model: SoleTrain.full<- glmer(Target_E2_pref ~ Listener.f*SyntaxType.f + (1 + Listener.f*SyntaxType.f|Subject) + (1 + Listener.f|Picture), data = SoleTrain, family = binomial, verbose=T, control=glmerControl(optCtrl=list(maxfun=20000))) # Create model with main effect of Listener removed: SoleTrain.noListener<- glmer(Target_E2_pref ~ SyntaxType.f + Listener.f:SyntaxType.f + (1 + Listener.f*SyntaxType.f|Subject) + (1 + Listener.f|Picture), data = SoleTrain, family = binomial, verbose=T, control=glmerControl(optCtrl=list(maxfun=20000))) > anova(SoleTrain.full, SoleTrain.noListener) Data: SoleTrain Models: SoleTrain.full: Target_E2_pref ~ Listener.f * SyntaxType.f + (1 + Listener.f * SyntaxType.f | Subject) + (1 + Listener.f | Picture) SoleTrain.noListener: Target_E2_pref ~ SyntaxType.f + Listener.f:SyntaxType.f + (1 + Listener.f * SyntaxType.f | Subject) + (1 + Listener.f | Picture) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq) SoleTrain.full 30 2732.5 2908.5 -1336.2 2672.5 SoleTrain.noListener 30 2732.5 2908.5 -1336.2 2672.5 0 0 1 However, I don't have this problem when I test for the main effect of SyntaxType, and remove the SyntaxType.f factor from the fixed effects. (That is, this produces a different model than the full model.) Someone suggested that Helmert coding was better for factors with more than two levels, so I tried running the same models except with Helmert coding [contrasts(SoleTrain$SyntaxType.f) = contr.helmert(3)], but the models come out identical to the way they do with dummy coding. So why does the model with the main effect of Listener removed the same as the model with the main effect of Listener retained? Any suggestions as to what I'm doing wrong? Thanks! Rachel