Hyukjin Kwon created SPARK-61324:
------------------------------------
Summary: Native Connect ML VectorAssembler, Pipeline and
BinaryClassificationEvaluator send params the server rejects
Key: SPARK-61324
URL: https://issues.apache.org/jira/browse/SPARK-61324
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
The native VectorAssembler and Pipeline send inputCols and stages as Rust Debug
text, not as an array of strings, so every transform fails with "Argument is
not an array". The native BinaryClassificationEvaluator sends its score column
as scoreCol, which the JVM evaluator does not have, so "Param scoreCol does not
exist". Official PySpark 4.3.0 sends a list as an array literal and scores
rawPredictionCol: VectorAssembler(inputCols=["x", "y"]) gives [[1.0, 2.0],
[4.0, 5.0]] and the AUC of the scores is 0.75.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
