Hello Marcus,
I have read the research paper you linked. In the paper, two variants of PSO are mentioned -- inertia weight and constriction factor based. It is stated that the local-best particle swarm optimizer (LBPSO) with constriction k produces the best results. I assume all variants must be implemented for GSoC, however, in the paper a modified version of PSO is presented (MPSO), which dynamically updates two hyper-parameters, k and c2 (acceleration constant for social elements in the swarm), should this be implemented as well? I suppose this won't be time consuming if vanilla PSO is already in place. Regarding the design of the optimizer itself, it was pointed out earlier by Ryan that the SDP (semidefinite program) optimizer supports constraints. In there, the constraints are specified as Armadillo matrices, and set using setters. I think the same methodology could be applied for PSO. For specifying whether the PSO is local or global, a boolean could be used. However, the constriction factor k should only be created in case of constriction based PSO, I'm not sure what would be the best design for this. Would it be possible for us to discuss the optimizer architecture in more detail on the mailing list? Thank you, Adeel ________________________________ From: Marcus Edel <[email protected]> Sent: Wednesday, January 17, 2018 5:39 PM To: Adeel Ahmad Cc: [email protected] Subject: Re: [mlpack] Query regarding constrained and unconstrained methods Hello Adeel, sorry for the slow reponse on this one. There are various approaches to solve constrained problems; one is the use of a penalty function. The constrained problem is transformed to an unconstrained one, by penalizing the constraints so that it can be solved using an unconstrained optimization method. You might take a look at: "Looking Inside Particle Swarm Optimization in Constrained Search Spaces" by Jorge Isacc Flores-Mendoza and Efrén Mezura-Montes they describe various PSO method to solve constrained problems. I apologize if I misunderstood what constrained problems are, but can't we apply constraints to the methods already present in "src/mlpack/methods/*" directory? Or, are these unrelated? In the latter case, are there some specialized methods for constrained problems that need to be implemented for this project? Currently, mlpack does not implement an optimizer that can handle constrained problems. So for example, if you like to solve the constrained (cube, line) Rosenbrock function: f(x, y) = (1 - x)^2 + 100(y - x^2)^2 with constraints (x - 1)^3 - y +1 < 0 and x + y - 2 < 0 Currently, there is no structure to represent the problem and there is no optimizer that can solve the constrained problem. Comming up with a structure is one part of the project implementing an optimizer (PSO) that can handle constrained problems is the other part. But as pointed out in the project idea, it's recommended to start with a PSO implementation for unconstrained problems and to extend the work later on. Regarding the test cases structuring, I've found that in some cases a test_function.cpp or <method_name>_test_function.cpp file is present in the main method directory, such as here (https://github.com/mlpack/mlpack/blob/master/src [https://avatars3.githubusercontent.com/u/10216045?s=400&v=4]<https://github.com/mlpack/mlpack/blob/master/src> mlpack/mlpack<https://github.com/mlpack/mlpack/blob/master/src> github.com mlpack: a scalable C++ machine learning library -- /mlpack/core/optimizers/gradient_descent/test_function.cpp). Later, an object of this class is created in the main tests directory ("src/mlpack/tests/*"), in this case, here (https://github.com/mlpack/mlpack/blob/master/src/mlpack/tests/g radient_descent_test.cpp). So, my question is this, what is the preferred structure for writing test cases? In this case, I think this could have been directly tested without the need of a separate GDTestFunction class, however, this might not have been a neat alternative. There is an open PR which consolidates different problems into one folder (https://github.com/mlpack/mlpack/pull/1151); the benefit for not implementing [https://avatars0.githubusercontent.com/u/4209744?s=400&v=4]<https://github.com/mlpack/mlpack/pull/1151> Optimization Test Problems by zoq · Pull Request #1151 · mlpack/mlpack<https://github.com/mlpack/mlpack/pull/1151> github.com Common functions used for testing optimization algorithms, will add more functions and test integrations once we agree on this. the test function inside the test itself, is that someone could reuse the functionality for other methods/tests. One example is the SGDTestFunction which is used to test Adam, SGD, RMSProp, etc. I hope this is helpful, let us know if we should clarify anything. Thanks, Marcus On 16. Jan 2018, at 19:58, Adeel Ahmad <[email protected]<mailto:[email protected]>> wrote: Greetings, I'm following a potential idea for GSoC 2018 titled "Particle swarm optimization". I have read a few documents and familiarized myself with the algorithm. It's listed in the idea description: "So this project is divided into two parts: First implement one or two unconstrained methods and afterwards takes a look at one --contained-- (constrained [?]) method". I apologize if I misunderstood what constrained problems are, but can't we apply constraints to the methods already present in "src/mlpack/methods/*" directory? Or, are these unrelated? In the latter case, are there some specialized methods for constrained problems that need to be implemented for this project? Regarding the test cases structuring, I've found that in some cases a test_function.cpp or <method_name>_test_function.cpp file is present in the main method directory, such as here (<https://github.com/mlpack/mlpack/blob/3d3d733ba3c41c4f51764f44185767384ab6d9c7/src/mlpack/core/optimizers/gradient_descent/test_function.cpp>https://github.com/mlpack/mlpack/blob/master/src/mlpack/core/optimizers/gradient_descent/test_function.cpp). Later, an object of this class is created in the main tests directory ("src/mlpack/tests/*"), in this case, here (<https://github.com/mlpack/mlpack/blob/97fd47de5e0a51adf3c01957f6646eb5cc3651d5/src/mlpack/tests/gradient_descent_test.cpp>https://github.com/mlpack/mlpack/blob/master/src/mlpack/tests/gradient_descent_test.cpp). So, my question is this, what is the preferred structure for writing test cases? In this case, I think this could have been directly tested without the need of a separate GDTestFunction class, however, this might not have been a neat alternative. Thank you, Adeel _______________________________________________ mlpack mailing list [email protected]<mailto:[email protected]> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
_______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
