Hi, James:
If by 'alpha' you mean what the lasso literature refers to as 'lambda', my
recollection is that the maximum lambda is determined simply by the L1 norm
of the coefficients of the ordinary least squares solution, because any
value greater than that provides no constraint for the lasso solution.
This was mentioned in a talk at ICML this year:
http://techtalks.tv/talks/the-lasso-persistence-and-cross-validation/58279/
Regards,
Nick
On Mon, Oct 14, 2013 at 9:34 AM, James Jensen <jdjen...@eng.ucsd.edu> wrote:
> Thanks, Alex. That is helpful. Looks like the glmnet documentation says
> that this is how they do it as well. What they don't explain is how to
> find alpha_max in the first place. The only thing I've thought of is
> doing something like a binary search until you find the smallest alpha
> yielding the coef_ of zeros, with some limit on how many steps you do it
> in. But is there a better way?
>
> Also, how do you choose the smallest alpha value (or in other words, how
> do you choose eps)? I came across an unofficial third-party description
> of glmnet that said that if nobs < nvars, a higher value is chosen
> (0.01, I think), whereas if nobs > nvars, a smaller value is chosen
> (say, 0.0001). The basic idea makes sense, but it seems a bit ad hoc to
> me, and it seems like it would be sensible to have more than two
> possible values, based on the ratio of nobs to nvars. Any thoughts?
>
> > hi James,
> >
> > for a given value of l1_ratio, the grid of alphas is chosen in log scale
> > starting from alpha_max to alpha_max / 10**eps. Any value of alpha
> > larger than alpha_max will lead to a coef_ full of zeros.
> >
> > HTH
> > Alex
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
