Hello Marcus, I went through the PR you mentioned a bit more thoroughly; I had missed the use of the velocity update policy. It's probably the best way to incorporate multiple variants of the algorithm.
>> Looks really interesting, the "Better initialization" point is definitely >> something that might affect the overall training time in a positive way. Just had an idea regarding this, it involves slightly modifying the way iterations are run as follows: 1. Randomly distribute particles over the search space (standard initialization method) 2. User specifies an initial point, currently what f.GetInitialPoint() does for the test functions. This is usually the weight matrix saved from a run of a training algorithm. 3. We run 1 (or a very small number of) iteration(s) on the randomly distributed particles and evaluate the fitness values of each particle. 4. Pick the worst particle from the swarm and replace it with the user provided initialization point (if it is better than this "worst", else discard it completely). The above steps ensure that each run of the algorithm makes use of the user provided initial point, and benefits from the random distribution as well. This method can also be extended to provide a better initial point for gradient descent: 5. Run the standard PSO algorithm for x% of the max iterations, and at the end of the x% iterations, evaluate the best particle. This particle will serve as the initialization point for the gradient descent step. 6. Run GD for the next (100-x)% iterations. It goes without saying that the algorithm will return as soon as the fitness reaches the tolerance threshold. The value of x depends mostly on the objective function complexity; if the objective function is computationally straining to evaluate, a smaller value of x will favour gradient descent over PSO and still influence training time in a positive way, whereas if the objective function is relatively simpler, PSO might converge faster. It might help to make x configurable as a user input. Regarding the time and resources being consumed by the build process, I have disabled tests entirely for the time being and linked against the generated library with external code; this has brought down the build time down to a few minutes and is much more resource friendly. I have set up a basic layout with some files (imitating the structure of other optimizers) and started coding; it might take some time to have a basic working PSO as I only have a few hours each weekday and mostly get work done over the weekend. I will submit a PR as soon as possible. Another doubt: the application guide mentions adding a link to "Homepage", is it necessary to have one? Please let me know your thoughts about the approach mentioned above. Will keep you posted about my progress. Thanks and regards, Chintan _______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
