Hi Anurag, list, Others on the list can weigh in with more authority, but perhaps this will get the discussion started.
I think your question gets at how the models are nested. To ensure that the likelihoods are chi-square distributed you also need to make sure that the parameter is not constrained against a limit (for instance, a Brownian process is only nested in an OU process by setting alpha to zero, and alpha is frequently constrained to be positive). In this case, you have a mixture of chi square and zero values (there's a rich literature on this, e.g. Ota et al. 2000 http://mbe.oxfordjournals.org/content/17/5/798.abstract, but for some reason it is frequently ignored). i.e. if it is constrained as in the OU/BM case, then you are only testing that the parameter estimate is significantly larger than zero and it's one-tailed, but not chi-squared. Recall that AIC values are a fequentist statistic: and they obey the very same same distribution as the likelihood ratio, (recall it is a difference log likelihoods, just shifted by the difference in the number of parameters (e.g. -2 [ log L1 - log L0 - (k1 - k0)]). Recall that the maximum likelihood estimate (MLE) is a biased estimate of the likelihood of your data and that AIC penalty is simply creating an asymptotically unbiased** estimator of the true model likelihood, which is a frequentist concept to begin with. Why we report confidence intervals/p-values in the case of one of these statistics but not the other is not obvious to me either. -Carl ** the MLE is biased in the frequentist sense, as follows: Simulate data under some "true" model and then evaluate the likelihood of that data under those "true" parameters. Now estimate the parameters from the data by maximum likelihood. This second likelihood will always be greater than or equal to the first "true" likelihood, making it a biased estimate (even though it converges to the true value). The corresponding AIC score should be symmetrically distributed about that "true" likelihood. On Wed, Jun 13, 2012 at 8:45 AM, Anurag Agrawal <aa...@cornell.edu> wrote: > Dear physigs, > I've been using likelihood ratio tests in various statistical models and > have seen mixed usage of two- vs. on-tailed tests of the difference in the > LL of two models. On the one hand, a one-tailed test seems reasonable > because a model can only reduce the model fit if we remove a parameter... > on the other hand, perhaps this is accounted for by the shape of a > chi-square distribution (which is bounded by zero on the left). > What should we be doing? I know I should be using AIC values, but I am > having difficulty escaping the frequentist paradigm. > Many thanks, -Anurag > > p.s. this is what Mark Pagel said a few years ago: "When your test allows > outcomes in either direction (plus or minus) you should set alpha in each > tail at alpha/2 to have a long run expectation of making Type I errors at > rate alpha." > > > [[alternative HTML version deleted]] > > _______________________________________________ > R-sig-phylo mailing list > R-sig-phylo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-phylo > -- Carl Boettiger UC Davis http://www.carlboettiger.info/ [[alternative HTML version deleted]] _______________________________________________ R-sig-phylo mailing list R-sig-phylo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
