On 05/03/2014 05:37 PM, Mathieu Blondel wrote:
Same feeling as Andy. I'd favor implementing averaging instead.
I totally forgot that we don't have averaging yet ^^ I'd be in favor of geometric averaging: http://arxiv.org/abs/1212.2002
MathieuOn Sun, May 4, 2014 at 12:23 AM, Andy <t3k...@gmail.com <mailto:t3k...@gmail.com>> wrote:Hi Sean. For linear classifiers I'm not really aware of benefits in doing mini-batch training, and I don't think it is widely used (someone correct me if I'm wrong). Usually we only like to add features that have a clear benefit for the users, to prevent scikit-learn from becoming bloated. Do you have a particular use-case where it is important? Cheers, Andy On 05/01/2014 02:04 AM, Sean Violante wrote:Hi I was wondering if there is any interest in implementing minibatch/batch for the SGD algorithm. As I understand it, this is not implemented "There is a compromise between the two forms, which is often called "mini-batches", where the true gradient is approximated by a sum over a small number of training examples." http://en.wikipedia.org/wiki/Stochastic_gradient_descent This would be doing a partial_fit (on small number of training examples) but updating the weights only after each epoch rather than after each training sample as far as I can see it would only require a flag in the sgd_fast.pyx code. thanks Sean ------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available. Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net <mailto:Scikit-learn-general@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available. Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net <mailto:Scikit-learn-general@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general ------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available. Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available. Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs
_______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
