Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Gael Varoquaux
On Fri, Feb 15, 2013 at 03:45:03PM +0100, Olivier Grisel wrote: > Alright for minimizing the RMSE on the training set but on the test > set the error might be minimized in between kinks as on the on real > distribution, the kinks location might be slightly off the positions > found on the training

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Olivier Grisel
2013/2/15 Gael Varoquaux : > On Fri, Feb 15, 2013 at 03:03:58PM +0100, Olivier Grisel wrote: >> > In my opinion, I am not sure that you want an interpolator that >> > interpolate the model parameters for specific alphas. You may want to >> > compute the RMSE at the knots of the path, because I thin

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Gael Varoquaux
On Fri, Feb 15, 2013 at 03:03:58PM +0100, Olivier Grisel wrote: > > In my opinion, I am not sure that you want an interpolator that > > interpolate the model parameters for specific alphas. You may want to > > compute the RMSE at the knots of the path, because I think that this is > > where they wi

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Olivier Grisel
2013/2/15 Gael Varoquaux : > On Fri, Feb 15, 2013 at 01:28:43PM +0100, Charles-Pierre Astolfi wrote: >> Just choose the alpha (from a fixed set) that minimizes the RMSE of >> the prediction of the last time step (or the last n time steps with >> exponential decay). Maybe I'm mistaken but there's no

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Gael Varoquaux
On Fri, Feb 15, 2013 at 02:08:14PM +0100, Olivier Grisel wrote: > I did not know about that. My previous answer is partially wrong then. And I didn't expect you to. I know this because I wrote the LassoLarsCV. This illustrates why I am reticent to add to much to this module: we need to balance the

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Olivier Grisel
2013/2/15 Gael Varoquaux : > On Fri, Feb 15, 2013 at 01:28:43PM +0100, Charles-Pierre Astolfi wrote: >> Just choose the alpha (from a fixed set) that minimizes the RMSE of >> the prediction of the last time step (or the last n time steps with >> exponential decay). Maybe I'm mistaken but there's no

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Olivier Grisel
2013/2/15 Charles-Pierre Astolfi : > On Fri, Feb 15, 2013 at 12:53 PM, Olivier Grisel > wrote: >> How do you evaluate which value of alpha is the best if you don't >> cross validate in one way or another? > > Just choose the alpha (from a fixed set) that minimizes the RMSE of > the prediction of t

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Gael Varoquaux
On Fri, Feb 15, 2013 at 01:28:43PM +0100, Charles-Pierre Astolfi wrote: > Just choose the alpha (from a fixed set) that minimizes the RMSE of > the prediction of the last time step (or the last n time steps with > exponential decay). Maybe I'm mistaken but there's no easy way to do > that with Lass

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Charles-Pierre Astolfi
On Fri, Feb 15, 2013 at 12:53 PM, Olivier Grisel wrote: > How do you evaluate which value of alpha is the best if you don't > cross validate in one way or another? Just choose the alpha (from a fixed set) that minimizes the RMSE of the prediction of the last time step (or the last n time steps wi

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Olivier Grisel
2013/2/15 Charles-Pierre Astolfi : > On Fri, Feb 15, 2013 at 11:30 AM, Gael Varoquaux > wrote: >> On Fri, Feb 15, 2013 at 11:22:02AM +0100, Charles-Pierre Astolfi wrote: >>> > However, before we do this, I'd like to understand: what is the usecase >>> > and the purpose for this function? >>> Which

Re: [Scikit-learn-general] Matlab linear classify vs SKlearn LDA

2013-02-15 Thread David Reed
Yes that is the method I was using, and its not giving the same results. I'm going to keep working on getting some sim data. On Fri, Feb 15, 2013 at 5:01 AM, Andreas Mueller wrote: > On 02/15/2013 02:10 AM, David Reed wrote: > > Could you link that? > > > http://www.mathworks.de/products/stati

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Charles-Pierre Astolfi
On Fri, Feb 15, 2013 at 11:30 AM, Gael Varoquaux wrote: > On Fri, Feb 15, 2013 at 11:22:02AM +0100, Charles-Pierre Astolfi wrote: >> > However, before we do this, I'd like to understand: what is the usecase >> > and the purpose for this function? >> Which? lars_path or lasso_path or my proposition

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Gael Varoquaux
On Fri, Feb 15, 2013 at 11:22:02AM +0100, Charles-Pierre Astolfi wrote: > > However, before we do this, I'd like to understand: what is the usecase > > and the purpose for this function? > Which? lars_path or lasso_path or my proposition? > What I propose in an faster and almost (in the sense that

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Olivier Grisel
2013/2/15 Charles-Pierre Astolfi : > On Fri, Feb 15, 2013 at 10:35 AM, Gael Varoquaux > wrote: >>> WDYT? >> I find that lars_path is already a function that is complicated-enough, >> with a signature difficult to understand, so I'd rather not make it more >> complex. If there is a need, we can add

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Charles-Pierre Astolfi
On Fri, Feb 15, 2013 at 10:35 AM, Gael Varoquaux wrote: >> WDYT? > I find that lars_path is already a function that is complicated-enough, > with a signature difficult to understand, so I'd rather not make it more > complex. If there is a need, we can add a similar function that calls > lars_path

Re: [Scikit-learn-general] Matlab linear classify vs SKlearn LDA

2013-02-15 Thread Andreas Mueller
On 02/15/2013 02:10 AM, David Reed wrote: Could you link that? http://www.mathworks.de/products/statistics/examples.html?file=/products/demos/shipping/stats/classdemo.html#3 "The classify function can perform classification using different types of discriminant analysis. First classify the da

Re: [Scikit-learn-general] Extreme Learning Machine implementation question

2013-02-15 Thread Olivier Grisel
2013/2/15 David Lambert : > > On Feb 14, 2013, at 6:52 PM, Lars Buitinck wrote: > >> 2013/2/15 : >>> how about softmax? >> >> The model is not intended for probability outputs. A predict_proba is >> not required; just implement a decision_function instead. > > Fair enough, and already done. For

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Gael Varoquaux
On Fri, Feb 15, 2013 at 10:03:55AM +0100, Olivier Grisel wrote: > I think it would be better to leave lasso_path as it is and add an > `alphas` option to lars_path instead that would trigger the > interpolation between the kinks automatically found by the lars > algorithm. > WDYT? I find that lar

Re: [Scikit-learn-general] Speeding up lasso_path by a factor of 2000

2013-02-15 Thread Olivier Grisel
2013/2/15 Charles-Pierre Astolfi : > That was my idea at first, but I was afraid of breaking things. > As of today, lasso_path returns a list of ElasticNet and that'd become > a list of LinearModel and it seemed like I would break > backward-compatibility. > > If that's not an issue, I definitely t