On Wednesday, July 15, 2015, Sebastian Raschka <se.rasc...@gmail.com> wrote:
> Can this maybe potentially addressed by feature scaling? E.g,.
> standardizing all features but multiplying certain ones by a factor?
>
+1, exactly.
If you accept to weight each feature the same in both l1 and l2
contribution to the penalty.
To see it, call the weight vector w, the alpha vector alpha.
The initial problem is
min mse(Xw-y) + pen(diag(alpha) * w)
Write v = diag(alpha)* w. Then you get
min mse(X diag(alpha)^-1 v) + pen(v)
That means dividing each column of your design by the corresponding
alpha (won't work for 0 alpha of course).
Solve this using elasticnet with penalty 1, but then observe that you need
to get w back from v by remultiplying with alpha.
Is this understandable?
Michael
>
>
> On Jul 15, 2015, at 1:23 PM, Doaa Altarawy <daltar...@vt.edu
> <javascript:_e(%7B%7D,'cvml','daltar...@vt.edu');>> wrote:
>
> Hi,
>
> Is it possible some how to use the penalty term in the Elastic net (the
> alpha) as a vector instead of a scaler?
> Meaning that I want to have different penalties or shrinkage weights on
> each predictor.
>
> The purpose is to add prior knowledge where some predictors are known to
> already have a strong relation with the response, but the coefficient is
> not known.
>
> Adding different penalties was proposed in gene regulation problem is this
> paper:
> http://bioinformatics.oxfordjournals.org/content/29/8/1060.short
>
> They called it modified Elastic net.
>
> Is there an easy way to do it in scikit-learn without having to implement
> it?
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
>
> https://www.gigenetcloud.com/_______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> <javascript:_e(%7B%7D,'cvml','Scikit-learn-general@lists.sourceforge.net');>
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
>
------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
