[R-sig-phylo] Comparing DIC of phylogenetic and non-phylogenetic GLMM run with MCMC (MCMCglmm)

2018-06-20 Thread Liam Kendall
Dear all, 

I am conducting an analysis predicting insect body sizes using a co-varying 
trait and their biogeographic region within two model formulations using 
MCMCglmm.

The first model has the structure: log(Weight) ~ log(Trait)+ Biogeography + 
Family (i.e. Taxonomic family of species)

The second model has the structure: log(Weight) ~ log(Trait)+ Biogeography + 
(1|Species/Animal), pedigree = phylogeny, i.e. variance between species is 
constrained by the branch lengths between the species.

The aim of running these two models is compare which is more predictive and to 
increase usability: Including family is user-friendly (and easy for the end 
user, especially if they’re not a taxonomist) whereas the phylogenetic model is 
more attractive theoretically however from a predictive sense requires your 
species of interest to be contained within the phylogeny used to fit the model,

Therefore, my question is how best can I compare these two models in model 
selection? Can I compare them directly by their DIC weighting if the only 
difference is the phylogenetic random term? Or is there be a better way to 
compare them? So far, we are also comparing their performance based off k-fold 
cross validation and RMSE but in the ‘age of AIC’, DIC appears a good place to 
start for model selection.

Any advice would be much appreciated.

Best,
Liam


[[alternative HTML version deleted]]

___
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/


Re: [R-sig-phylo] estimate ancestral state with OUwie models

2018-06-20 Thread David Bapst
That sounds pretty different from doing ancestral reconstruction with some
fossil data, and hoping the resulting node estimates will reflect reality,
because fossil data is now involved. Look, I think every method has its
uses; I'm just railing against the idea that the concerns of any particular
method go away by tapping into the fossil record.

Personally I have a work-in-progess myself that involves parsimony
ancestral state reconstruction of some very slow-evolving discrete traits
in a fossil-rich group. I think in that particular case, I can defend my
approach, but I don't think it'd be applicable in many other cases, even
with other fossil-rich groups.

Cheers,
-Dave

On Thu, Jun 14, 2018 at 11:08 AM, Jacob Berv  wrote:

> What about situations in which fossil calibrations are used as priors to
> inform reconstructions for uncalibrated important/interesting nodes? Sure,
> there is great uncertainty, but that doesn’t necessarily imply we should
> entirely abandon hypothesis testing using this approach, does it? I like
> the idea of using priors to constrain different scenarios and then using
> model testing to compare alternative histories (though one’s model ranking
> may change depending on the underlying models too, I guess).
>
> Jake
>
>
> > On Jun 14, 2018, at 9:32 AM, David Bapst  wrote:
> >
> > Simone, Marguerite, others,
> >
> > I'll also add that I think there's a great deal to be skeptical of
> > ancestral trait reconstruction even when large amounts of fossil data is
> > available. You can try the exercise yourself: simulate pure BM on a
> > non-ultrametric tree with lots of 'extinct' tips, and you'll still find
> > pretty large confidence intervals on the estimates of the trait values.
> > What does it mean to do ancestral trait reconstruction, if our
> calculations
> > of uncertainty are that broad?
> >
> > That said, in the era of sampled-ancestor phylogenetics for fossil data,
> > the ability to examine quantitative support for ancestor-descendant
> > relationships among fossil taxa may allow alternative routes to
> considering
> > this issue.
> >
> > Cheers,
> > -Dave
> >
> > On Tue, Jun 12, 2018 at 2:43 AM, Simone Blomberg 
> > wrote:
> >
> >> I would add an extra caveat to Marguerite’s excellent post: Most
> >> researchers work with extant taxa only, ignoring extinction. This
> causes a
> >> massive ascertainment bias, and the character states of the extinct taxa
> >> can often be very different to the ancestral state reconstructions,
> >> particularly if the evolutionary model is wrong. Eg. there has been an
> >> evolutionary trend for example. Ancestral state reconstructions based
> only
> >> on extant taxa should be treated as hypotheses to be tested with fossil
> >> data. I wouldn’t rely on them for much more.
> >>
> >> Cheers,
> >> Simone.
> >>
> >> Sent from my iPhone
> >>
> >> On 12 Jun 2018, at 4:59 pm, Marguerite Butler 
> >> wrote:
> >>
> >> Aloha all,
> >>
> >> There is no requirement for an ultra metric tree in the formulae
> reported
> >> in Butler-King 2004. Interested investigators should in particular read
> the
> >> supplementary materials where the mathematical details are worked out.
> >>
> >> We do generally use ultrametric trees because as comparative biologists,
> >> it is more straightforward to think about evolution in units of time
> rather
> >> than in terms of mutational units, etc. However this is by choice, not
> any
> >> methodological limitation.
> >>
> >> Once the model parameters are found, the phylogenetic
> variance-covariance
> >> matrix defined by the alpha, thetas, and sigmas can be used to compute
> >> ancestral states using a weighted least squares reconstruction method
> >> (instead of the typical BM var-cov matrix). The mapping of the alphas,
> >> thetas, and sigmas onto the tree are incorporated into this V-COV
> matrix,
> >> so that accounts for the OU model.
> >>
> >> NOTES:
> >> 1) without knowing why you are doing this, I do feel compelled to warn
> you
> >> that it is unclear why one would want to estimate ancestral states for
> >> poorly-fitting models. Be careful!
> >>
> >> 2) I hope you realize that ancestral states are in general poorly
> >> estimated, even assuming the “correct” model. This is because there is
> less
> >> and less information to anchor the values as you get farther from the
> tips,
> >> similar to the root estimation problem described below.  This issue was
> >> clearly exposed in Schluter et al 1997 (and less famously so in Butler
> and
> >> Losos 1997). These depressing results were among the motivations for
> >> developing model-fit approaches in the first place.
> >>
> >> 3) In 2008/2009 the algorithms in OUCH, SLOUCH, and possibly other
> methods
> >> have changed in the estimation of the value of the root state (X0)
> which is
> >> an internal calculation in fitting the model.  Ho and Ane 2013, and
> Hansen
> >> et al 2008 both reported that the root state X0 is ill-defined (unless
> >> there are fossil data