zhengruifeng commented on SPARK-3181:

I am working on blockify+gemv/gemm for better performance, and found that the 
huber regression' s solution varies significantly when blockSize is changed.


Suggested by [~weichenxu123], I tested the stability by [shuffling the input 
 and found that:

the solution of LInearSVC and LinearRegression(SquaredError) looks stable when 
input is shuffled; while the solution of LinearRegression(huber) varies 


I also tested scikit-learn's {{HuberRegressor which also use LBFGSB}} as the 
solver, its solutions seem stable. So I guess there maybe something wrong in 
{{BreezeLBFGSB}}. Maybe using 'Replace {{\sigma}} to {{\exp(\alpha)' or 
'Pseudo-Huber loss' is an alternative.}}



> Add Robust Regression Algorithm with Huber Estimator
> ----------------------------------------------------
>                 Key: SPARK-3181
>                 URL: https://issues.apache.org/jira/browse/SPARK-3181
>             Project: Spark
>          Issue Type: New Feature
>          Components: ML
>    Affects Versions: 2.2.0
>            Reporter: Fan Jiang
>            Assignee: Yanbo Liang
>            Priority: Major
>              Labels: features
>             Fix For: 2.3.0
>   Original Estimate: 0h
>  Remaining Estimate: 0h
> Linear least square estimates assume the error has normal distribution and 
> can behave badly when the errors are heavy-tailed. In practical we get 
> various types of data. We need to include Robust Regression  to employ a 
> fitting criterion that is not as vulnerable as least square.
> In 1973, Huber introduced M-estimation for regression which stands for 
> "maximum likelihood type". The method is resistant to outliers in the 
> response variable and has been widely used.
> The new feature for MLlib will contain 3 new files
> /main/scala/org/apache/spark/mllib/regression/RobustRegression.scala
> /test/scala/org/apache/spark/mllib/regression/RobustRegressionSuite.scala
> /main/scala/org/apache/spark/examples/mllib/HuberRobustRegression.scala
> and one new class HuberRobustGradient in 
> /main/scala/org/apache/spark/mllib/optimization/Gradient.scala

This message was sent by Atlassian Jira

To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to