Hyukjin Kwon created SPARK-61711:
------------------------------------
Summary: Fetch the server defaults of the native ML classes as
PySpark's constructors do, and refuse Column arguments of their members
Key: SPARK-61711
URL: https://issues.apache.org/jira/browse/SPARK-61711
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
Part of completing the native ML classes of pyspark.ml.connect to the full
pyspark.ml 4.3.0 surface (fixes of the PyO3 base, SPARK-61697, that the area
conversions found):
- The defaults that PySpark's constructor fetches of the server, by the ML
helper (StopWordsRemover's default stop words and locale), are recorded as the
fetch in the API golden, and the native constructors fetch them of the active
session, raising PySpark's AssertionError without one.
- An argument of a member of a native model that is a Column, even of a
literal, raises PySpark's UNSUPPORTED_DATA_TYPE, as
pyspark.ml.connect.serialize raises it.
- The native classes of Params that neither fit, transform nor evaluate
(PrefixSpan) are virtual subclasses of pyspark.ml.param.Params.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]