On Mon, Jul 10, 2017 at 04:10:09PM +0000, federico vaggi wrote: > There is a fantastic library called lightning where the optimization > routines are first class citizens: > http://contrib.scikit-learn.org/lightning/ - you can take a look there. > However, lightning focuses on convex optimization, so most algorithms have > provable convergence rates.
Hi, I fully agree that lightning is fantastic :) but it might not be what Gürhan wants. It's true that lightning's api is designed around optimizers rather than around models. So where in scikit-learn we usually have, e.g., LogisticRegression(solver='sag') in lightning you would have SAGClassifier(loss='log') to achieve something close. But neither library has the oo-style separation between freeform models and optimizers such as you might find in deep learning frameworks. So, for instance, it's relatively easy to add a new loss function to the lightning SAGClassifier, but you would still be able to only use it with a linear model. This is by design in both scikit-learn and lightning, at least at the moment: by making these kinds of assumptions about the models, implementations can be much more efficient in terms of computation and storage, especially when sparse data is involved. Yours, Vlad > > Good luck! > > On Mon, 10 Jul 2017 at 09:05 Jacob Schreiber <jmschreibe...@gmail.com> > wrote: > > > Howdy > > > > This question and the one right after in the FAQ are probably relevant re: > > inclusion of new algorithms: > > http://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms. > > The gist is that we only include well established algorithms, and there are > > no end to those. I think it is unlikely that a PR will get merged with a > > cutting edge new algorithm, as the scope of scikit-learn isn't necessary > > "the latest" as opposed to "the classics." You may also consider writing a > > scikit-contrib package that basically creates what you're interested in in > > scikit-learn format, but external to the project. We'd be more than happy > > to link to it. If the algorithm becomes a smashing success over time, we'd > > reconsider adding it to the main code base. > > > > As to your first question, you should check out how the current optimizers > > are written for the algorithm you're interested in. I don't think there's a > > plug and play way to drop in your own optimizer like many deep learning > > packages support, unfortunately. You'd probably have to modify the code > > directly to support your own. > > > > Let me know if you have any other questions. > > > > Jacob > > > > On Mon, Jul 10, 2017 at 7:58 AM, Gürhan Ceylan <grhancey...@gmail.com> > > wrote: > > > >> Hi everyone, > >> > >> I am wondering, How can I use external optimization algorithms with > >> scikit-learn, > >> for instance neural network > >> <http://scikit-learn.org/stable/modules/neural_networks_supervised.html#algorithms> > >> , instead of defined algorithms ( Stochastic Gradient Descent, Adam, or > >> L-BFGS). > >> > >> Furthermore, I want to introduce a new unconstrained optimization > >> algorithm to scikit-learn, implementation of the algorithm and related > >> paper > >> can be found here <https://github.com/sibirbil/PMBSolve>. > >> > >> I couldn't find any explanation > >> <http://scikit-learn.org/stable/developers/contributing.html>, about the > >> situation. Do you have defined procedure to make such kind of > >> contributions? If this is not the case, How should I start to make such a > >> proposal/contribution ? > >> > >> > >> Kind regards, > >> > >> Gürhan C. > >> > >> > >> _______________________________________________ > >> scikit-learn mailing list > >> scikit-learn@python.org > >> https://mail.python.org/mailman/listinfo/scikit-learn > >> > >> > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn@python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
