Hi Marcus, I have got familiar with the code base and the listed papers. And I think these are the modifications we should add to the original CMA-ES. * ACM-ES We can add a surrogate model into the original algorithm. We check whether the model is given during the iteration, and call it if so. * IPOP-CMA-ES The key idea is to restart the algorithm with a larger population when terminated. * Active-CMA-ES The key idea is to introduce new update functions to variance matrix and other parameters. And I think we can extract the update functions in the main algorithm as separated functions which can be overwritten in derived classes.
Here are steps that I think we need to do: Phase #1 1. making initialization, iteration and update rules separated functions. 2. implement IPOP-CMA-ES, which will be easier on basis of step #1. 3. test and test preparation for the other two algorithms 4. documentation Phase #2 1. implement Active-CMA-ES 2. test and documentation Phase #3 1. implement ACM-ES 2. test and documentation Please let me know if there is any suggestion on the plan. Thanks, Yihan Marcus Edel <[email protected]> 于2020年3月9日周一 上午7:08写道： > Hello Yihan, > > > * Enhance CMA-ES I have began to check the references listed, and I have > a > > question related to the current mlpack. Currently is there an original > CMA-ES > > algorithm in the mlpack? If there is none, I can begin from the original > > implementation. > > All mlpack optimizers are in another repository including the CMA-ES > optimizer: > https://github.com/mlpack/ensmallen and > > https://github.com/mlpack/ensmallen/tree/master/include/ensmallen_bits/cmaes > . > > > * Implement the Transformer in mlpack I think what we need to do is first > > implement an attention layer and then the transformer itself. For > testing, we > > can compare the result with results got from pytorch or so. > > Agreed, mlpack doesn't implement an attention layer. > > Let me know if I should clarify anything. > > Thanks, > Marcus > > > On 8. Mar 2020, at 07:54, Yihan Wang <[email protected]> wrote: > > > > Hi all, > > > > I am Yihan Wang, a final year student from Tsinghua University, with > more than a year's research experience in machine learning algorithms. I am > interested in participating in this year's GSoC. In particular I am > interested in these two topics. > > > > * Enhance CMA-ES > > I have began to check the references listed, and I have a question > related to the current mlpack. Currently is there an original CMA-ES > algorithm in the mlpack? If there is none, I can begin from the original > implementation. > > > > * Implement the Transformer in mlpack > > I think what we need to do is first implement an attention layer and > then the transformer itself. For testing, we can compare the result with > results got from pytorch or so. > > > > Is there any suggestion related to these two ideas? > > > > Best, > > Yihan > > _______________________________________________ > > mlpack mailing list > > [email protected] > > http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack > >
_______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
