Hyukjin Kwon created SPARK-61688:
------------------------------------

             Summary: Add the server-side ML objects to the Rust client: fetch, 
summaries, save and load
                 Key: SPARK-61688
                 URL: https://issues.apache.org/jira/browse/SPARK-61688
             Project: Spark
          Issue Type: Improvement
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


Add the server-side objects of Spark Connect ML to the Rust client, as PySpark 
4.3.0 reads and writes them over Spark Connect: a public MlRef of a fitted 
model (Model::ml_ref), a fetch of the methods of an object, of a method chain 
such as a model's summary, or of the ML helper (Fetch, with CreateSummary when 
the server has lost a summary), the DataFrames of a method (AttributeRelation), 
the values of a fetch as deserialize_param reads them (MlValue, byte for byte 
as PySpark's literals), and the save and load of estimators, transformers, 
evaluators and models (Write, Read).

This is the base of completing the native ML classes to the pyspark.ml surface 
(typed model accessors, summaries, persistence).




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to