Hyukjin Kwon created SPARK-61477:
------------------------------------
Summary: Raise NOT_IMPLEMENTED from the native Pipeline.fit of no
stages, and correct the seed and label docs of the native tree classifiers
Key: SPARK-61477
URL: https://issues.apache.org/jira/browse/SPARK-61477
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
Follow-up of SPARK-61399 and SPARK-61450. The native Pipeline(stages=[]).fit
raises CANNOT_INFER_ARRAY_ELEMENT_TYPE (and a test says PySpark does), while
any other stage list raises NOT_IMPLEMENTED; PySpark fits an empty pipeline
model. The docs of DecisionTreeClassifier say its seed breaks ties, but in
Spark the seed of a tree samples the input of the split candidates; the docs of
DecisionTreeClassifier and RandomForestClassifier say labels must be
contiguous, but Spark takes any class index below numClasses. The e2e test of
RandomForestRegressor keeps its own copy of the live session helper of the
common test module.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]