[
https://issues.apache.org/jira/browse/SPARK-33520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weichen Xu updated SPARK-33520:
-------------------------------
Description:
Currently, pyspark support third-party library to define python backend
estimator, i.e., estimator that inherit `Estimator` instead of `JavaEstimator`,
and only can be used in pyspark.
CrossValidator and TrainValidateSplit support tuning these python backend
estimator,
but cannot support saving/load, becase CrossValidator and TrainValidateSplit
writer implementation is use JavaMLWriter, which require to convert nested
estimator into java estimator.
> make CrossValidator/TrainValidateSplit support Python backend estimator/model
> -----------------------------------------------------------------------------
>
> Key: SPARK-33520
> URL: https://issues.apache.org/jira/browse/SPARK-33520
> Project: Spark
> Issue Type: New Feature
> Components: ML, PySpark
> Affects Versions: 3.1.0
> Reporter: Weichen Xu
> Priority: Major
>
> Currently, pyspark support third-party library to define python backend
> estimator, i.e., estimator that inherit `Estimator` instead of
> `JavaEstimator`, and only can be used in pyspark.
> CrossValidator and TrainValidateSplit support tuning these python backend
> estimator,
> but cannot support saving/load, becase CrossValidator and TrainValidateSplit
> writer implementation is use JavaMLWriter, which require to convert nested
> estimator into java estimator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
