@Olivier - no problem :)
@Alex
Appart from trying it ourselves to see how it fares, here're some findings:
The simulations that are done in the last mentioned paper by Ming Yuan and
Yi Lin,
http://www2.isye.gatech.edu/statistics/papers/05-25.pdf, they find that
the NG seems to do
generally better than the LASSO (figure 1)
Their second simulations they consider the 4 different models used in the
original LASSO paper (Tibshirani, 1996)
which they use to compare the NG with several other popular methods,
inluding the LASSO.
The results are shown in Table 2 of the above-mentioned paper from which
the NG does very well, often outperforming the other models and being the
most successful in variable selection.
They also include one real example using the prostate cancer dataset from
Stamey (1989) - the results of which
they use to confirm the theory that the path consistency of LASSO depends
on the correlationo of the design matrix whilst that of the NG is always
path consistent.
Thought it may be of interest
J
2012/4/11 Alexandre Gramfort <[email protected]>
> > The algorithm proposed in this paper, is rather similar to that of the
> Lars
> > LASSO, but with a complicating
> > factor being a non-negative constraint on the shrinkage factor. (See eq.
> (2)
> > in this paper)
> > Once you've computed your shrinkage factor, you basically have your
> > regression coefficients
> > seeing as your NG coefficient = shrinkage factor * regression coefficient
>
> that's exactly what I implemented.
>
> > He showed it to be a stable selection method and often outperforms it's
> > competitors like
> > subset regression and ridge regression.
> > The solution path of the NG is piece-wise linear and it's whole path can
> be
> > computed quickly.
>
> we could indeed have a path easily using LassoLars
>
> > It is also path-consistent (A solution that contains at least one
> desirable
> > estimate) given an appropriate initial estimate. The path-consistency of
> the
> > NG is highlighted to be in contrast to the fact that the LASSO is not
> always
> > path consistent (Peng Zhao & Hui Zou, personal communication). It is
> argued
> > that the NG has the ability to turn
> > a consistent estimate into an estimate that is both consistent in terms
> of
> > estimation and in terms of variable selection.
>
> hum. If it can be consistent on support and coef amplitudes that's neat.
>
> > A drawback is the NG's explicit reliance on the full least square
> estimate,
> > as a small sample size may cause it to perform poorly - however a ridge
> > regression is suggested as an initial estimate for defining the NG
> estimate,
> > instead
> > of the least square estimate.
>
> good to know
>
> it seems to me that it might be a good addition to the scikit if can
> convince ourselves with examples that it does better than a Lasso.
>
> Alex
>
>
> ------------------------------------------------------------------------------
> Better than sec? Nothing is better than sec when it comes to
> monitoring Big Data applications. Try Boundary one-second
> resolution app monitoring today. Free.
> http://p.sf.net/sfu/Boundary-dev2dev
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
Better than sec? Nothing is better than sec when it comes to
monitoring Big Data applications. Try Boundary one-second
resolution app monitoring today. Free.
http://p.sf.net/sfu/Boundary-dev2dev
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
