Hi,
On Sun, Dec 2, 2018 at 2:05 PM Michael Duncan
<[email protected] <mailto:[email protected]>> wrote:
can any moses gurus comment on using the diversity scoring
options? for some bioinformatics work we are more
interested in evolving a minimal ensemble of models to
maximally represent the patterns distinguishing two sample
sets rather than just maximizing out of sample prediction
accuracy. in other words, we want to maximize the number of
unique features in a model ensemble of a given size and
accuracy. more generally, are there procedures for choosing
optimal ensemble models beyond combining the top n models
from different cross-validation runs?
Yes, moses can take into account model diversity when sorting the top n
models of the metapopulation.
I recall that it works very well, but of course that depends on how
moses is being used.
My advice would be to start by setting --diversity-pressure to 0.1, then
double till you get passed 10.
See if you obtain a more diverse population.
I think the the tool
eval-diversity
may help you to measure the diversity of each run (if moses isn't enough).
If you're not happy with the result it might mean that you don't let
moses evolve enough demes. Remember that diversity work at the
metapopulation level, thus it affects the choice of the next deme
exemplar, so you really need to let moses explore multiple demes to
build-up diversity.
Also, are you looking for feature set diversity? Or are you looking for
candidate diversity (expressing different output behaviors, regardless
of whether they use different features)?
If you're mostly interested in feature set diversity then I would
recommend to enable diversity at the feature selection stage, see
--fs-diversity-pressure
of course it's only gonna work if you're using feature selection to
begin with (which I would recommend).
The other option, that doesn't involve using any diversity flag, is to
use tune feature selection to be highly sensitive of random
fluctuations. I forgot how to do that but I could dig it up if you want
me to. The advantage is that it's gonna yield diversity at a really low
computational cost.
Anyway, hope it helps, feel free to send me you're moses commands and
data so I can provide you more guidance.
Nil
--
You received this message because you are subscribed to the Google Groups
"opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/opencog.
To view this discussion on the web visit
https://groups.google.com/d/msgid/opencog/9f7c03fa-0c0d-92e5-ac39-80bc3dd86bd2%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
