Hyukjin Kwon created SPARK-61759:
------------------------------------
Summary: Make the native ML summaries subclasses of the summaries
of their PySpark bases, of PySpark's metaclass, and construct them as PySpark's
JavaWrapper
Key: SPARK-61759
URL: https://issues.apache.org/jira/browse/SPARK-61759
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
The native summary classes of pyspark.ml.connect (LinearSVCTrainingSummary,
BinaryLogisticRegressionTrainingSummary, KMeansSummary, ...) differ from
PySpark's:
- A training summary was only a virtual subclass of its summary (registered by
an ABCMeta), and a Binary summary of its summary, where PySpark's are real
subclasses (BinaryLogisticRegressionTrainingSummary of both
BinaryLogisticRegressionSummary and LogisticRegressionTrainingSummary); their
metaclass was ABCMeta, where PySpark's is type.
- A summary could not be constructed: PySpark's constructs as
JavaWrapper.__init__(java_obj=None), of no server-side object, and its members
then read the java_obj as _extract_id_methods does (the AttributeError of
'NoneType' object has no attribute 'split', or the server's error of an id that
it does not hold).
This makes each native training and Binary summary extend its native base
summary (they add no fields, so a class of two bases keeps their layout), gives
the public classes the public classes of PySpark's bases of the module as real
bases and PySpark's metaclass, and gives every summary PySpark's constructor
and errors.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]