Hyukjin Kwon created SPARK-61688:
------------------------------------
Summary: Add the server-side ML objects to the Rust client: fetch,
summaries, save and load
Key: SPARK-61688
URL: https://issues.apache.org/jira/browse/SPARK-61688
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
Add the server-side objects of Spark Connect ML to the Rust client, as PySpark
4.3.0 reads and writes them over Spark Connect: a public MlRef of a fitted
model (Model::ml_ref), a fetch of the methods of an object, of a method chain
such as a model's summary, or of the ML helper (Fetch, with CreateSummary when
the server has lost a summary), the DataFrames of a method (AttributeRelation),
the values of a fetch as deserialize_param reads them (MlValue, byte for byte
as PySpark's literals), and the save and load of estimators, transformers,
evaluators and models (Write, Read).
This is the base of completing the native ML classes to the pyspark.ml surface
(typed model accessors, summaries, persistence).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]