[
https://issues.apache.org/jira/browse/SPARK-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weichen Xu closed SPARK-16638.
------------------------------
Resolution: Not A Problem
> The L2 regularization of LinearRegression seems wrong when standardization is
> false
> -----------------------------------------------------------------------------------
>
> Key: SPARK-16638
> URL: https://issues.apache.org/jira/browse/SPARK-16638
> Project: Spark
> Issue Type: Bug
> Components: ML, Optimizer
> Reporter: Weichen Xu
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> The original L2 is
> 0.5 * effectiveL2regParam * sigma( wi^2 )
> (wi is the coefficients we want to train)
> And in linearRegression code, when standardization == false, the code modify
> L2 into:
> 0.5 * effectiveL2regParam * sigma( ( w[i] / featuresStd[i] )^2 )
> It is obviously wrong, I think.
> As the purpose of author wrote in the code comment, the modification to L2
> reg should be:
> 0.5 * effectiveL2regParam * sigma( ( w[i] * featuresStd[i] )^2 )
> wi should not be divided by featuresStd[i], but should be multiplied by
> featuresStd[i]
> We can simply think this problem in the following way:
> assume a training data, there is a dimension k with very large std ( the
> value of featuresStd[k] is very large), so, we hope the training result
> coefficient w[k] to be small to keep the numerical stability. As a way of
> that, we can add a penalty to the L2 reg on this dimension.
> so that it should be surely w[k] * featuresStd[k], not w[k] / featuresStd[k]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
