Hello Chintan, thanks for the update.
> The above steps ensure that each run of the algorithm makes use of the > user provided initial point, and benefits from the random distribution > as well. This method can also be extended to provide a better initial > point for gradient descent: Really like the ideas, and I think this is something that could be explored, might be a neat research component. > I have set up a basic layout with some files (imitating the structure > of other optimizers) and started coding; it might take some time to > have a basic working PSO as I only have a few hours each weekday and > mostly get work done over the weekend. I will submit a PR as soon as > possible. Sounds good, don't feel obligated to work on the parts before GSoC has started, but it might be a good opportunity to get familiar with the codebase, your call. > Another doubt: the application guide mentions adding a link to > "Homepage", is it necessary to have one? No, if you have one, you could add a reference, if not that's fine, no worries this are optional informations. Thanks, Marcus > On 14. Mar 2018, at 05:09, Chintan Soni <[email protected]> wrote: > > Hello Marcus, > > I went through the PR you mentioned a bit more thoroughly; I had > missed the use of the velocity update policy. It's probably the best > way to incorporate multiple variants of the algorithm. > >>> Looks really interesting, the "Better initialization" point is definitely >>> something that might affect the overall training time in a positive way. > > Just had an idea regarding this, it involves slightly modifying the > way iterations are run as follows: > > 1. Randomly distribute particles over the search space (standard > initialization method) > > 2. User specifies an initial point, currently what f.GetInitialPoint() > does for the test functions. This is usually the weight matrix saved > from a run of a training algorithm. > > 3. We run 1 (or a very small number of) iteration(s) on the randomly > distributed particles and evaluate the fitness values of each > particle. > > 4. Pick the worst particle from the swarm and replace it with the user > provided initialization point (if it is better than this "worst", else > discard it completely). > > The above steps ensure that each run of the algorithm makes use of the > user provided initial point, and benefits from the random distribution > as well. This method can also be extended to provide a better initial > point for gradient descent: > > 5. Run the standard PSO algorithm for x% of the max iterations, and at > the end of the x% iterations, evaluate the best particle. This > particle will serve as the initialization point for the gradient > descent step. > > 6. Run GD for the next (100-x)% iterations. > > It goes without saying that the algorithm will return as soon as the > fitness reaches the tolerance threshold. > > The value of x depends mostly on the objective function complexity; if > the objective function is computationally straining to evaluate, a > smaller value of x will favour gradient descent over PSO and still > influence training time in a positive way, whereas if the objective > function is relatively simpler, PSO might converge faster. It might > help to make x configurable as a user input. > > Regarding the time and resources being consumed by the build process, > I have disabled tests entirely for the time being and linked against > the generated library with external code; this has brought down the > build time down to a few minutes and is much more resource friendly. > > I have set up a basic layout with some files (imitating the structure > of other optimizers) and started coding; it might take some time to > have a basic working PSO as I only have a few hours each weekday and > mostly get work done over the weekend. I will submit a PR as soon as > possible. > > Another doubt: the application guide mentions adding a link to > "Homepage", is it necessary to have one? > > Please let me know your thoughts about the approach mentioned above. > Will keep you posted about my progress. > > Thanks and regards, > Chintan _______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
