hi sklearners, FYI Danny Sullivan https://github.com/dsullivan7 will work at Telecom ParisTech with me as a scikit-learn engineer starting this summer. These topics (SGD improvements, averaging, SAG etc.) are part of the roadmap.
I think he will start by setting up a benchmark for online supervised estimators on sparse and dense data and then bench the different stochastic variants. However I am sure that any work done before he starts this summer will be useful :) Before Danny actually starts we'll send an email to the list to detail the plan and get feedback. Best, Alex On Sat, May 3, 2014 at 7:08 PM, Andy <t3k...@gmail.com> wrote: > On 05/03/2014 05:37 PM, Mathieu Blondel wrote: > > Same feeling as Andy. I'd favor implementing averaging instead. > > I totally forgot that we don't have averaging yet ^^ > I'd be in favor of geometric averaging: http://arxiv.org/abs/1212.2002 > > > > > Mathieu > > > On Sun, May 4, 2014 at 12:23 AM, Andy <t3k...@gmail.com> wrote: >> >> Hi Sean. >> For linear classifiers I'm not really aware of benefits in doing >> mini-batch training, and I don't think it is widely used (someone correct me >> if I'm wrong). >> Usually we only like to add features that have a clear benefit for the >> users, to prevent scikit-learn from becoming bloated. >> >> Do you have a particular use-case where it is important? >> >> Cheers, >> Andy >> >> >> On 05/01/2014 02:04 AM, Sean Violante wrote: >> >> Hi >> >> I was wondering if there is any interest in implementing minibatch/batch >> for the SGD algorithm. As I understand it, this is not implemented >> >> "There is a compromise between the two forms, which is often called >> "mini-batches", where the true gradient is approximated by a sum over a >> small number of training examples." >> >> http://en.wikipedia.org/wiki/Stochastic_gradient_descent >> >> >> >> This would be doing a partial_fit (on small number of training examples) >> but updating the weights only after each epoch rather than after each >> training sample >> >> >> as far as I can see it would only require a flag in the sgd_fast.pyx code. >> >> thanks >> >> Sean >> >> >> >> ------------------------------------------------------------------------------ >> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE >> Instantly run your Selenium tests across 300+ browser/OS combos. Get >> unparalleled scalability from the best Selenium testing platform >> available. >> Simple to use. Nothing to install. Get started now for free." >> http://p.sf.net/sfu/SauceLabs >> >> >> >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> >> >> >> >> ------------------------------------------------------------------------------ >> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE >> Instantly run your Selenium tests across 300+ browser/OS combos. Get >> unparalleled scalability from the best Selenium testing platform >> available. >> Simple to use. Nothing to install. Get started now for free." >> http://p.sf.net/sfu/SauceLabs >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> > > > > ------------------------------------------------------------------------------ > "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE > Instantly run your Selenium tests across 300+ browser/OS combos. Get > unparalleled scalability from the best Selenium testing platform available. > Simple to use. Nothing to install. Get started now for free." > http://p.sf.net/sfu/SauceLabs > > > > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > > > > ------------------------------------------------------------------------------ > "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE > Instantly run your Selenium tests across 300+ browser/OS combos. Get > unparalleled scalability from the best Selenium testing platform available. > Simple to use. Nothing to install. Get started now for free." > http://p.sf.net/sfu/SauceLabs > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > ------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available. Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
