Hi Jeff,
Actually I have one implementation of robust regression with huber loss for
a long time (https://github.com/apache/spark/pull/14326). This is a fairly
straightforward porting for scikit-learn HuberRegressor.
The PR making huber regression as a separate Estimator, and we found it can
be merged into LinearRegression.
I will update this PR ASAP, and I'm looking forward your reviews and
comments.
After the Scala implementation is merged, it's very easy to add
corresponding PySpark API, then you can use it to train huber regression
model in the distributed environment.
Thanks
Yanbo
On Sun, Aug 20, 2017 at 3:19 PM, Jeff Gates <gatesa...@gmail.com> wrote:
> Hi guys,
>
> Is there huber regression in PySpark? We are using sklearn HuberRegressor (
> http://scikit-learn.org/stable/modules/generated/sklearn.
> linear_model.HuberRegressor.html) to train our model, but with some
> bottleneck in single node.
> If no, is there any obstacle to implement it in PySpark?
>
> Jeff
>
