@Jacob, I understand your concern about the new algorithms. It will be lost effort to make coding, updating, documentation for a unsuccessful algorithm. Thanks for the tips.
@federico, lightning library is, close to what in my mind but not the same. I think, there should be an easy way to see how optimizers affects learning algorithms. Thanks for the link. @Vlad, Thank you, for the clarification. Best, Gürhan 2017-07-10 20:37 GMT+03:00 Vlad Niculae <zephy...@gmail.com>: > On Mon, Jul 10, 2017 at 04:10:09PM +0000, federico vaggi wrote: > > There is a fantastic library called lightning where the optimization > > routines are first class citizens: > > http://contrib.scikit-learn.org/lightning/ - you can take a look there. > > However, lightning focuses on convex optimization, so most algorithms > have > > provable convergence rates. > > Hi, > > I fully agree that lightning is fantastic :) but it might not be what > Gürhan > wants. > > It's true that lightning's api is designed around optimizers rather > than around models. So where in scikit-learn we usually have, e.g., > > LogisticRegression(solver='sag') > > in lightning you would have > > SAGClassifier(loss='log') > > to achieve something close. But neither library has the oo-style > separation between freeform models and optimizers such as you might > find in deep learning frameworks. So, for instance, it's relatively > easy to add a new loss function to the lightning SAGClassifier, but > you would still be able to only use it with a linear model. > > This is by design in both scikit-learn and lightning, at least at the > moment: by making these kinds of assumptions about the models, > implementations can be much more efficient in terms of computation and > storage, especially when sparse data is involved. > > Yours, > Vlad > > > > > Good luck! > > > > On Mon, 10 Jul 2017 at 09:05 Jacob Schreiber <jmschreibe...@gmail.com> > > wrote: > > > > > Howdy > > > > > > This question and the one right after in the FAQ are probably relevant > re: > > > inclusion of new algorithms: > > > http://scikit-learn.org/stable/faq.html#what-are-the- > inclusion-criteria-for-new-algorithms. > > > The gist is that we only include well established algorithms, and > there are > > > no end to those. I think it is unlikely that a PR will get merged with > a > > > cutting edge new algorithm, as the scope of scikit-learn isn't > necessary > > > "the latest" as opposed to "the classics." You may also consider > writing a > > > scikit-contrib package that basically creates what you're interested > in in > > > scikit-learn format, but external to the project. We'd be more than > happy > > > to link to it. If the algorithm becomes a smashing success over time, > we'd > > > reconsider adding it to the main code base. > > > > > > As to your first question, you should check out how the current > optimizers > > > are written for the algorithm you're interested in. I don't think > there's a > > > plug and play way to drop in your own optimizer like many deep learning > > > packages support, unfortunately. You'd probably have to modify the code > > > directly to support your own. > > > > > > Let me know if you have any other questions. > > > > > > Jacob > > > > > > On Mon, Jul 10, 2017 at 7:58 AM, Gürhan Ceylan <grhancey...@gmail.com> > > > wrote: > > > > > >> Hi everyone, > > >> > > >> I am wondering, How can I use external optimization algorithms with > scikit-learn, > > >> for instance neural network > > >> <http://scikit-learn.org/stable/modules/neural_ > networks_supervised.html#algorithms> > > >> , instead of defined algorithms ( Stochastic Gradient Descent, Adam, > or > > >> L-BFGS). > > >> > > >> Furthermore, I want to introduce a new unconstrained optimization > > >> algorithm to scikit-learn, implementation of the algorithm and > related paper > > >> can be found here <https://github.com/sibirbil/PMBSolve>. > > >> > > >> I couldn't find any explanation > > >> <http://scikit-learn.org/stable/developers/contributing.html>, about > the > > >> situation. Do you have defined procedure to make such kind of > > >> contributions? If this is not the case, How should I start to make > such a > > >> proposal/contribution ? > > >> > > >> > > >> Kind regards, > > >> > > >> Gürhan C. > > >> > > >> > > >> _______________________________________________ > > >> scikit-learn mailing list > > >> scikit-learn@python.org > > >> https://mail.python.org/mailman/listinfo/scikit-learn > > >> > > >> > > > _______________________________________________ > > > scikit-learn mailing list > > > scikit-learn@python.org > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn@python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn >
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
