On Mon, Sep 17, 2012 at 1:37 PM, Andreas Mueller
<[email protected]>wrote:
> Hi Ariel.
>
>
>
> Thanks for your response and for pointing me to that commit - I didn't
> even know this was being actively discussed! I was looking exactly at that
> part of the doc-string you refer to in that commit. Now (in Alex's commit)
> the docstring is very strange: it looks as though rho doesn't matter at
> all! rho=0 is L1 and rho=1 is L1 too!
>
> I just fixed that 5 minutes ago ;)
>
> My understanding is that 'rho' should serve the purpose of what is
> referred to as '1-\alpha' in equation 3.54 in chapter 3 of the Elements of
> Statistical Learning (this one:
> http://www-stat.stanford.edu/~tibs/ElemStatLearn/). That is, when alpha
> is high, L2 is penalized and when alpha is low L1 is penalized.
>
> Yes, rho is 1-alpha.
>
> The code itself:
> https://github.com/scikit-learn/scikit-learn/blob/9987b61cf87aa8eeecd9c8e2fae6c29599892613/sklearn/linear_model/coordinate_descent.py#L190
>
> Also suggests that high rho would lead to high L1 regularization and low
> rho would lead to high L2 regularization, which is consistent with what I
> get in my computation. So, I am slowly coming to the conclusion that the
> docstring should read (e.g. on line 63):
>
>
> "[...rho=0...]the penalty is an L2 penalty. For rho = 1 it is an L1
> penalty"
>
> Does that make sense to you?
>
> Yes, I think it does. This is also how I understand it.
>
> Can you check if the docstring in current (as of right now ;) master
> reflects your understanding?
> If you find this inconsistency anywhere else, please let me know.
> In SGDClassifier rho=alpha atm but that is a (somewhat) other story.
>
Yeah - looks good for ElasticNet. What's the story of line 1004 in that
same file (the docstring for the MultiTaskElasticNet) should that really be
"L1/L2"?
Ariel
>
> Cheers,
> Andy
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
