Hello, I'm using Mplus for two-level regression analyses with multiply imputed data. I'm interested in performing likelihood ratio tests to check if slope variances are significantly different from zero (TYPE = TWOLEVEL RANDOM). For conventional models, Mplus now reports results of a combined LRT using the method described by Meng and Rubin (1992). I wonder whether I can use the same formulas for combining the single (say 5) log-likelihoods from my multilevel models (or have to use different ones). Any references related to my question would be highly appreciated, too.
Thank you! Jan ----- Jan Hochweber German Institute for International Educational Research (DIPF), Frankfurt/Main, Germany -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.utsouthwestern.edu/pipermail/impute/attachments/20090713/220a6cdb/attachment.htm From Craig.Enders <@t> asu.edu Mon Jul 13 11:02:05 2009 From: Craig.Enders <@t> asu.edu (Craig Enders) Date: Mon Jul 13 11:03:44 2009 Subject: [Impute] likelihood ratio test for multilevel models References: <b56feb1467768e4db0a6e78c60b283ed02bc18c...@domexc03> Message-ID: <[email protected]> Jan, I have attached a SAS macro program that combines LR tests, as described by Meng & Rubin (1992). You just enter the LR values and the complete-data df and the program does the rest. A couple of things ... The formula for combining LR tests appears in several different sources. If you look at the formulas closely, they are not all the same. There is apparently a typo in one of the early articles that has propagated into subsequent manuscripts/books. I emailed Dr. Meng some time ago to get some clarification, so I'm confident that the attached SAS program is using the correct formula. Second, I believe that the LR test that Mplus prints uses a different reference distribution than the original article. The webnote on the Mplus website says that the test uses a chi-square distribution with the same df as the original test, but the original article uses a complex df expression. Who knows whether this makes any practical difference, but its worth mentioning .... Best, Craig Enders ***************************************************************************** Dr. Craig Enders Associate Professor Arizona State University Department of Psychology Quantitative Psychology Concentration Box 871104 Tempe, AZ 85287-1104 Office (480) 727-0739 [email protected] http://www.asu.edu/clas/psych/people/faculty/cenders.htm ***************************************************************************** -----Original Message----- From: [email protected] on behalf of Hochweber, Jan Sent: Mon 7/13/2009 7:01 AM To: [email protected] Subject: [Impute] likelihood ratio test for multilevel models Hello, I'm using Mplus for two-level regression analyses with multiply imputed data. I'm interested in performing likelihood ratio tests to check if slope variances are significantly different from zero (TYPE = TWOLEVEL RANDOM). For conventional models, Mplus now reports results of a combined LRT using the method described by Meng and Rubin (1992). I wonder whether I can use the same formulas for combining the single (say 5) log-likelihoods from my multilevel models (or have to use different ones). Any references related to my question would be highly appreciated, too. Thank you! Jan ----- Jan Hochweber German Institute for International Educational Research (DIPF), Frankfurt/Main, Germany -------------- next part -------------- A non-text attachment was scrubbed... Name: Combining Likelihood Ratio Chi-Square Statistics From a MI Analysis.sas Type: application/octet-stream Size: 1706 bytes Desc: Combining Likelihood Ratio Chi-Square Statistics From a MI Analysis.sas Url : http://lists.utsouthwestern.edu/pipermail/impute/attachments/20090713/fc70466c/CombiningLikelihoodRatioChi-SquareStatisticsFromaMIAnalysis.obj From K.J.M.Janssen <@t> umcutrecht.nl Mon Jul 6 09:23:14 2009 From: K.J.M.Janssen <@t> umcutrecht.nl (Janssen, K.J.M.) Date: Thu Jul 16 20:23:51 2009 Subject: [Impute] weird question In-Reply-To: <[email protected]> References: <[email protected]> <[email protected]> Message-ID: <[email protected]> Dear all, I have attachted the manuscript that Frank mentioned, in which several methods to deal with missing values are compared, once a prediction model is applied in practice. Hope this may help you out. Best regards, Kristel J.M. Janssen PhD Clinical Epidemiologist Julius Center for Health Sciences and Primary Care University Medical Center Utrecht PO Box 85500 3508 GA Utrecht The Netherlands 0031-8875-51752 -----Oorspronkelijk bericht----- Van: [email protected] [mailto:[email protected]] Namens Frank E Harrell Jr Verzonden: zondag 5 juli 2009 17:46 Aan: Paul Allison CC: [email protected] Onderwerp: Re: [Impute] weird question Paul Allison wrote: > I completely agree with Frank Harrell that this is not, in general, a > good method. I haven't checked out all his references but, for me, the > definitive refutation was Jones' 1996 paper in the Journal of the > American Statistical Association. And I reference your excellent 2001 book, p. 9-11. > > Nevertheless, I still believe that this method may be useful in two > situations: > > 1. Data are "missing" because a variable doesn't apply or is undefined > for some fraction of cases. For example, suppose you have a measure > of marital happiness, dichotomized as high or low, but your sample > contains some unmarried people. Then it is entirely appropriate to > have a 3-category variable with values high, low, and unmarried. Nice example Paul. I've added that to my notes and book, with attribution. > > 2. The goal is to build a forecasting model, and it is anticipated > that a substantial fraction of the new cases to be forecast will have > missing data on one or more variables. Here, the goal is not to get > unbiased estimates of population parameters but to minimize some > function of prediction errors. A workable forecasting model must have > some way of dealing with the cases that have missing data. Maybe there > are better ways, but I've found almost no literature on this topic > (with the exception of Warren Sarle's unpublished paper). My colleagues Janssen, Donders, and Moons in The Netherlands are working on that. Averaging predictions over multiple imputations is one approach but there are others. There are some logistical problems to imputing especially with regard to updating the imputation rules. Cheers, Frank > > ----------------------------------------------------------------- > Paul D. Allison, Professor > Department of Sociology > University of Pennsylvania > 581 McNeil Building > 3718 Locust Walk > Philadelphia, PA 19104-6299 > 215-898-6717 > 215-573-2081 (fax) > http://www.ssc.upenn.edu/~allison > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Maria da > Conceicao-Saraiva > Sent: Saturday, July 04, 2009 9:19 AM > To: [email protected] > Subject: [Impute] weird question > > > > > Sorry about this question, > > I have been discussing with some people I am working about the need of > imputation with some of our data. What some of analysist are doing is > just to creating a category of missing values inside some variables, > they argue this is enough. It has been hard to argue with them that > this is not the best way to do. Specially in our variable income, we > have about 30% of missings. > Does anybody know about refereces discussing this approach of just > creating a category for missing values inside a variable? > > Maria > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > ~~~~~~ > ~~ > Maria da Conceicao P. Saraiva DDS, MSc, Ph.D Departamento de Clinica > Infantil e Odontologia Social e Preventiva Faculdade de Odontologia de > Ribeirao Preto-Universidade de Sao Paulo > > Aviso: Esta mensagem destina-se exclusivamente ao destinatario, sendo > confidencial. Se V. Sa. nao eh o destinatario, fique advertido de > que a divulgacao, distribuicao ou copia desta mensagem eh estritamente proibida. > Caso tenha recebido esta mensagem por engano, por favor avise > imediatamente seu remetente atraves de resposta por e-mail. Obrigado. > -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University _______________________________________________ Impute mailing list [email protected] http://lists.utsouthwestern.edu/mailman/listinfo/impute -------------- next part -------------- A non-text attachment was scrubbed... Name: Janssen KJ Dealing with missing predictor values when applying clinical prediction models Clin Chem 2009.pdf Type: application/octet-stream Size: 95243 bytes Desc: Janssen KJ Dealing with missing predictor values when applying clinical prediction models Clin Chem 2009.pdf Url : http://lists.utsouthwestern.edu/pipermail/impute/attachments/20090706/3d78a56d/JanssenKJDealingwithmissingpredictorvalueswhenapplyingclinicalpredictionmodelsClinChem2009-0001.obj From Seppo.Laaksonen <@t> stat.fi Tue Jul 21 03:58:00 2009 From: Seppo.Laaksonen <@t> stat.fi (Laaksonen Seppo) Date: Tue Jul 21 04:04:21 2009 Subject: Re(2): [Impute] weird question In-Reply-To: <[email protected]> References: <[email protected]> <[email protected]> Message-ID: <[email protected]> Just reading your comments after being on holiday. I think as Paul that Maria's approach is OK but not best in several cases. I have even called this solution as a starting imputation method that should be always done. If possible, several missingess codes are useful to create since there are often some information behind the missingness (like refusal, non-contact, does not know, does not want to answer, does not know well but approximately, missing for unknown reasons). After this kind of coding, it is of course good to try for getting better 'imputed values.' In the case of an exploratory analysis with a number of explanatory (independent) variables, I have not tried to use such an approach for a dependent variable (Has someone done?), but this approach is not bad if this happens for explanatory variables. If the mssingness has even been coded ('imputed') well, you can interpret the results quite well (e.g. when I have been explaining happiness by a number of variables, income being partially missing with several codes, fortunately, it has not been difficult to see for example, that some missingess income people are unhappier than the average; note that this is not necessarily my main explanatory variable but an important control variable; imputation for these missing categories could be dramatic since it would be a hard job but not still a main issue, and may lead to a bias). And as mentioned in my parenthesis example, this kind of a variable can be a control variable too. Note also that this is very often more advantageous than the two major alternatives: (i) data deletion when loosing a lot of observations (too often used in econometrics, for example) and (ii) unsuccessful imputation (it is also hard to know how well your imputation has been succeeded and your results can be problematic). Keep always in mind what are your main estimates. Be realistic and not always use MI. Seppo University of Helsinki >> Nevertheless, I still believe that this method may be useful in two >> situations: >> >> 1. Data are "missing" because a variable doesn't apply or is undefined > >> for some fraction of cases. For example, suppose you have a measure >> of marital happiness, dichotomized as high or low, but your sample >> contains some unmarried people. Then it is entirely appropriate to >> have a 3-category variable with values high, low, and unmarried. > >Nice example Paul. I've added that to my notes and book, with >attribution. > >> >> 2. The goal is to build a forecasting model, and it is anticipated >> that a substantial fraction of the new cases to be forecast will have >> missing data on one or more variables. Here, the goal is not to get >> unbiased estimates of population parameters but to minimize some >> function of prediction errors. A workable forecasting model must have >> some way of dealing with the cases that have missing data. Maybe there > >> are better ways, but I've found almost no literature on this topic >> (with the exception of Warren Sarle's unpublished paper). > >My colleagues Janssen, Donders, and Moons in The Netherlands are working >on that. Averaging predictions over multiple imputations is one >approach but there are others. There are some logistical problems to >imputing especially with regard to updating the imputation rules. > >Cheers, >Frank > >> >> ----------------------------------------------------------------- >> Paul D. Allison, Professor >> Department of Sociology >> University of Pennsylvania >> 581 McNeil Building >> 3718 Locust Walk >> Philadelphia, PA 19104-6299 >> 215-898-6717 >> 215-573-2081 (fax) >> http://www.ssc.upenn.edu/~allison >> >> >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Maria da > >> Conceicao-Saraiva >> Sent: Saturday, July 04, 2009 9:19 AM >> To: [email protected] >> Subject: [Impute] weird question >> >> >> >> >> Sorry about this question, >> >> I have been discussing with some people I am working about the need of > >> imputation with some of our data. What some of analysist are doing is >> just to creating a category of missing values inside some variables, >> they argue this is enough. It has been hard to argue with them that >> this is not the best way to do. Specially in our variable income, we >> have about 30% of missings. >> Does anybody know about refereces discussing this approach of just >> creating a category for missing values inside a variable? >> >> Maria >> >> >> >> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> ~~~~~~ >> ~~ >> Maria da Conceicao P. Saraiva DDS, MSc, Ph.D Departamento de Clinica >> Infantil e Odontologia Social e Preventiva Faculdade de Odontologia de > >> Ribeirao Preto-Universidade de Sao Paulo >> >> Aviso: Esta mensagem destina-se exclusivamente ao destinatario, >sendo >> confidencial. Se V. Sa. nao eh o destinatario, fique advertido de >> que a divulgacao, distribuicao ou copia desta mensagem eh estritamente >proibida. >> Caso tenha recebido esta mensagem por engano, por favor avise >> imediatamente seu remetente atraves de resposta por e-mail. Obrigado. >> >-- >Frank E Harrell Jr Professor and Chair School of Medicine > Department of Biostatistics Vanderbilt >University > >_______________________________________________ >Impute mailing list >[email protected] >http://lists.utsouthwestern.edu/mailman/listinfo/impute >
