Accidentally replied directly to Marguerite, instead of to the list:
> On Aug 9, 2019, at 10:01 AM, Jacob Berv <jakeberv.r.sig.ph...@gmail.com>
> wrote:
>
> Great points Marguerite. I agree that the idea of using ‘all possible’ models
> is probably a bit too overzealous— and as Brian pointed out, some models will
> probably generate totally bogus results. For example, if you had three states
> and only allowed one transition rate between two states-- clearly that would
> not make any sense.
>
> But perhaps it might be useful to explore a reasonable set of models which
> have plausible biological interpretations — like the ARD, ER, SYM models
> typically available (and perhaps a few custom models).
>
> I was thinking that model averaging in this context may be a useful way to
> integrate out some of the uncertainty in ‘model choice.’ AIC/BIC ranking is
> one way to do that, but in my mind ‘best statistical fit' may not be
> synonymous with ‘best,' at least without some hypothesis of the underlying
> biology. Though I suppose if you are weighting by AIC/BIC then you are still
> looking at 'statistical fit,’ just in another way.
>
> J
>
>
>> On Aug 9, 2019, at 2:06 AM, Marguerite Butler <mbutler...@gmail.com
>> <mailto:mbutler...@gmail.com>> wrote:
>>
>> Aloha Jacob, Liam, Brian,
>>
>> To add to Brian's and Liam's excellent comments - just remember that any
>> type of ancestral state reconstruction of any form is just a weighted
>> average of the information that you put into the problem. If you have no
>> fossil data or data back in time, it's a weighted average of the tips
>> dictated by the model and/or the algorithm that you use. So point one is
>> keep that in mind.
>>
>> Second, along the lines of Brian's comments, it is always worth looking at
>> the model outputs - first of all, are the estimates different? If not, then
>> there is no reason to model average, the solution is robust to the model.
>> If they are different, then would it make more sense to interpret each model
>> results one by one and think about what the scenarios imply? It might be
>> useful to consider the most distinct alternative hypotheses, for example.
>> Of course I always only include in the model set the ones that have a
>> biological interpretation (the most robust tests have the most distinct but
>> biologically interpretable hypotheses). It is a way of bracketing the
>> interpretation of an answer.
>>
>> Often model averaging will result in parameter estimates that are
>> intermediate in value - but if none of the models returned that estimate,
>> then what is it exactly? It's a point that is best fit for nothing. But the
>> solution along an entire tree is a solution that is part of a set (a single
>> evolutionary scenario). Mixing and matching from different solutions may
>> not make sense.
>>
>> So I've personally never found a situation where model averaging is useful
>> and sometimes can lead to non-critical thinking. But if you find something,
>> I'm all ears!
>>
>> Best of luck,
>> Marguerite
>>
>> On Thu, Aug 8, 2019 at 10:14 PM Jacob Berv <jakeberv.r.sig.ph...@gmail.com
>> <mailto:jakeberv.r.sig.ph...@gmail.com>> wrote:
>> Cool- thanks for the great comments Liam and Brian, as always!
>> J
>>
>> > On Aug 7, 2019, at 11:55 AM, Liam Revell <liam.rev...@umb.edu
>> > <mailto:liam.rev...@umb.edu>> wrote:
>> >
>> > Hi Jake.
>> >
>> > [Edit: I see just now that Brian has also responded to this inquiry. I
>> > have no doubt that his message is more insightful than mine - but I'll
>> > nonetheless send what I was writing anyway just in case it contributes
>> > anything useful to the discussion.]
>> >
>> > If you're simply interested in the states at nodes, you might consider
>> > just multiplying the marginal reconstructions under maximum likelihood
>> > by the Akaike weights of each model & summing them.
>> >
>> > The former are probabilities that the node is in each state conditioned
>> > on a model, and the latter are the probabilities that each of the models
>> > is the best of the set. If the models in the set genuinely comprise all
>> > possible ways in which your character could have evolved (they don't,
>> > but still), then the model weighted average marginal reconstructions
>> > should give the total probability that each node is in each state.
>> >
>> > If you really want to do stochastic mapping, you might consider some
>> > kind of rjMCMC in which you sample models & transition rates from their
>> > joint posterior distribution and then generate stochastic character maps
>> > based on this sample. This is not implemented in phytools::make.simmap
>> > (it does MCMC, but only given a model), but is not to hard to envision
>> > doing, so long as you are careful about designing the rjMCMC.
>> >
>> > Now to read Brian's answer....
>> >
>> > All the best, Liam
>> >
>> > Liam J. Revell
>> > Associate Professor, University of Massachusetts Boston
>> > Profesor Asistente, Universidad Católica de la Ssma Concepción
>> > web: http://faculty.umb.edu/liam.revell/
>> > <http://faculty.umb.edu/liam.revell/> <http://faculty.umb.edu/liam.revell/
>> > <http://faculty.umb.edu/liam.revell/>>, http://www.phytools.org
>> > <http://www.phytools.org/> <http://www.phytools.org/
>> > <http://www.phytools.org/>>
>> >
>> > Academic Director UMass Boston Chile Abroad (starting 2019):
>> > https://www.umb.edu/academics/caps/international/biology_chile
>> > <https://www.umb.edu/academics/caps/international/biology_chile>
>> > <https://www.umb.edu/academics/caps/international/biology_chile
>> > <https://www.umb.edu/academics/caps/international/biology_chile>>
>> >
>> > On 8/7/2019 11:32 AM, Jacob Berv wrote:
>> >> Dear R-sig phylo
>> >>
>> >> I’ve been running a few discrete character Mk type models using phytool's
>> >> SIMMAP — and I had the idea that it might be useful to try model
>> >> averaging across posterior probabilities for node states.
>> >>
>> >> Might this make sense to do, to avoid problems associated with model
>> >> ranking via AIC? Ie, average the node state probabilities based on AIC
>> >> weights? Is there some fundamental problem with this?
>> >>
>> >> I could imagine generating some code to generate all possible transition
>> >> models given a set of N states, and then rather than ranking with AIC,
>> >> model averaging for parameter estimates (though now that I think about
>> >> it, not sure how one might reasonably average a symmetrical rate and an
>> >> asymmetrical rate).
>> >>
>> >> Best,
>> >> Jake
>> >>
>> >> [[alternative HTML version deleted]]
>> >>
>> >> _______________________________________________
>> >> R-sig-phylo mailing list - R-sig-phylo@r-project.org
>> >> <mailto:R-sig-phylo@r-project.org> <mailto:R-sig-phylo@r-project.org
>> >> <mailto:R-sig-phylo@r-project.org>>
>> >> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-phylo&data=02%7C01%7Cliam.revell%40umb.edu%7Ce85015cb8c314c684bdc08d71b4c8fee%7Cb97188711ee94425953c1ace1373eb38%7C0%7C0%7C637007887976647805&sdata=F8qdVoT17J%2BuC6wgvFktdI1id%2Bf37hDLe1W17z6OhTw%3D&reserved=0
>> >>
>> >> <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-phylo&data=02%7C01%7Cliam.revell%40umb.edu%7Ce85015cb8c314c684bdc08d71b4c8fee%7Cb97188711ee94425953c1ace1373eb38%7C0%7C0%7C637007887976647805&sdata=F8qdVoT17J%2BuC6wgvFktdI1id%2Bf37hDLe1W17z6OhTw%3D&reserved=0>
>> >>
>> >> <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-phylo&data=02%7C01%7Cliam.revell%40umb.edu%7Ce85015cb8c314c684bdc08d71b4c8fee%7Cb97188711ee94425953c1ace1373eb38%7C0%7C0%7C637007887976647805&sdata=F8qdVoT17J%2BuC6wgvFktdI1id%2Bf37hDLe1W17z6OhTw%3D&reserved=0
>> >>
>> >> <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-phylo&data=02%7C01%7Cliam.revell%40umb.edu%7Ce85015cb8c314c684bdc08d71b4c8fee%7Cb97188711ee94425953c1ace1373eb38%7C0%7C0%7C637007887976647805&sdata=F8qdVoT17J%2BuC6wgvFktdI1id%2Bf37hDLe1W17z6OhTw%3D&reserved=0>>
>> >> Searchable archive at
>> >> https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.mail-archive.com%2Fr-sig-phylo%40r-project.org%2F&data=02%7C01%7Cliam.revell%40umb.edu%7Ce85015cb8c314c684bdc08d71b4c8fee%7Cb97188711ee94425953c1ace1373eb38%7C0%7C0%7C637007887976657803&sdata=ogCaep2DN92bk6%2FZwOOcpmHaCPQwX7BzNgKo2UMcWtw%3D&reserved=0
>> >>
>> >> <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.mail-archive.com%2Fr-sig-phylo%40r-project.org%2F&data=02%7C01%7Cliam.revell%40umb.edu%7Ce85015cb8c314c684bdc08d71b4c8fee%7Cb97188711ee94425953c1ace1373eb38%7C0%7C0%7C637007887976657803&sdata=ogCaep2DN92bk6%2FZwOOcpmHaCPQwX7BzNgKo2UMcWtw%3D&reserved=0>
>> >>
>> >> <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.mail-archive.com%2Fr-sig-phylo%40r-project.org%2F&data=02%7C01%7Cliam.revell%40umb.edu%7Ce85015cb8c314c684bdc08d71b4c8fee%7Cb97188711ee94425953c1ace1373eb38%7C0%7C0%7C637007887976657803&sdata=ogCaep2DN92bk6%2FZwOOcpmHaCPQwX7BzNgKo2UMcWtw%3D&reserved=0
>> >>
>> >> <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.mail-archive.com%2Fr-sig-phylo%40r-project.org%2F&data=02%7C01%7Cliam.revell%40umb.edu%7Ce85015cb8c314c684bdc08d71b4c8fee%7Cb97188711ee94425953c1ace1373eb38%7C0%7C0%7C637007887976657803&sdata=ogCaep2DN92bk6%2FZwOOcpmHaCPQwX7BzNgKo2UMcWtw%3D&reserved=0>>
>> >>
>>
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-phylo mailing list - R-sig-phylo@r-project.org
>> <mailto:R-sig-phylo@r-project.org>
>> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
>> <https://stat.ethz.ch/mailman/listinfo/r-sig-phylo>
>> Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/
>> <http://www.mail-archive.com/r-sig-phylo@r-project.org/>
>>
>>
>> --
>> ____________________________________________
>> Marguerite A. Butler
>> Professor
>>
>> Department of Biology
>> 2538 McCarthy Mall, Edmondson Hall 216
>> Honolulu, HI 96822
>>
>> Office: 808-956-4713
>> Dept: 808-956-8617
>> Lab: 808-956-5867
>> FAX: 808-956-4745
>> http://butlerlab.org <http://butlerlab.org/>
>> http://manoa.hawaii.edu/biology/people/marguerite-butler
>> <http://manoa.hawaii.edu/biology/people/marguerite-butler>
>> http://www2.hawaii.edu/~mbutler <http://www2.hawaii.edu/~mbutler>
>>
>>
>>
[[alternative HTML version deleted]]
_______________________________________________
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/
