On Mon, Mar 29, 2021 at 10:03:10AM +0200, Oleksandr Nikolskyy wrote: > Dear all, I want to share a draft of the first part of my proposal for GSOC > (approx first month).
Hey Oleksandr, I took a look through. Thanks for putting this together. Just a couple thoughts: > There are improvements on scalability for CMA-ES to achieve up to linear > time and memory complexity. > > The idea is to refactor the existing CMA ES to a modular framework, which > allows the implementation and usage of different variants of CMA-ES. > Structurally the considered CMA ES variants obey the general form: > > 1. Sample new population > 2. Selection and recombination > 3. Step size control > 4. Covariance adaptation > > There are different approaches, e.g. LM-CMA-ES, do not store a covariance > matrix to reduce the memory size. Therefore, from a more general view, we > can summarize the ES structure as > > 1. SamplerStrategy > 2. SelectionStrategy > 3. AdaptationStrategy. > 4. Restart strategy (?) > > An important point while construction of the framework would be to provide > type traits to ensure a given combination of strategies can work together. > > One of the main advantages of this general framework is first of all > extensibility. > > A good implementation would allow a user to invent their own ES and give > them a possibility to experiment. If you are planning on implementing all these strategies, you probably want to think about how you will test that it gets the speedup you expect. I don't imagine it will take too long to build a test program for that, but you should probably account for that. If the claim is that these variants are much faster, we should verify that in the implementation. :) > A tutorial on the usage and extension of the framework should be provided. How will this fit into the existing ensmallen documentation? We've kept the ensmallen documentation intentionally short and only consider templates when we really have to, so at least what we put on the user facing documentation displayed on the website should be similarly concise and simple. Perhaps the deeper documentation is in the code itself and we point users there? > Do you think it is an overkill? I was reading through different papers > regarding improvements of CMA-ES and there are so many variants out there, > that extensibility is perhaps a good approach. Definitely extensibility is good! We should make it easy to implement other variants later, in my opinion. Hopefully these comments are helpful! -- Ryan Curtin | "Great party, isn't it?" [email protected] | - Delbert Grady _______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
