[
https://issues.apache.org/jira/browse/SPARK-61334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-61334:
------------------------------------
Assignee: Hyukjin Kwon
> Make the setters of the native pyspark.ml.connect classes set the value on
> the object and return it, as PySpark does
> --------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61334
> URL: https://issues.apache.org/jira/browse/SPARK-61334
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> In the Python drop-in, the setters of the native pyspark.ml.connect classes
> StandardScaler, MaxAbsScaler, StringIndexer, VectorAssembler,
> LogisticRegression, RegressionEvaluator, BinaryClassificationEvaluator,
> MulticlassClassificationEvaluator, Pipeline and CrossValidator returned a new
> object and left the receiver unchanged. So a call whose result was not kept
> was silently ignored: ev.setMetricName("r2"); ev.evaluate(df) returned the
> rmse, MulticlassClassificationEvaluator().setMetricName("accuracy") gave f1,
> Pipeline().setStages([...]) kept no stages, CrossValidator's
> setNumFolds/setParallelism/setSeed were ignored, and
> LogisticRegression().setLabelCol("y") then fit failed on the column label.
> PySpark's setters (Params._set) set the value on the object and return the
> object; the native KMeans, LinearSVC and LinearRegression already do. The fix
> makes every native setter do so, with the argument name value, as PySpark's.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]