[ 
https://issues.apache.org/jira/browse/SPARK-61749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hyukjin Kwon reassigned SPARK-61749:
------------------------------------

    Assignee: Hyukjin Kwon

> Take the paths of the saves and loads of the native ML classes as PySpark 
> does, and read the JVM class of a subclass's name
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-61749
>                 URL: https://issues.apache.org/jira/browse/SPARK-61749
>             Project: Spark
>          Issue Type: Improvement
>          Components: Connect
>    Affects Versions: connect-rust-4.3.0
>            Reporter: Hyukjin Kwon
>            Assignee: Hyukjin Kwon
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: connect-rust-4.3.0
>
>
> The native ML classes of spark-connect-rust (pyspark.ml.connect.*) took the 
> path of save, load, write().save and read().load as a str only, and raised 
> PyO3's TypeError of any other value ("'int' object is not an instance of 
> 'str'"), where PySpark takes it as the protobuf field of its Write and Read 
> takes it (bytes of their UTF-8, None as the empty path that the server 
> refuses, upb's "bad argument type for built-in operation" of anything else, a 
> pathlib.Path too), and of the Python format of a Pipeline or a validator as 
> os.path.join takes it (an os.PathLike, os.path's TypeError of anything else).
> Also, the load of a Python subclass read the JVM class of the native class, 
> where PySpark's RemoteMLReader names the JVM class of the Python class, which 
> the server refuses (CONNECT_ML.UNSUPPORTED_EXCEPTION); and getParam of a name 
> that is not a str raised PyO3's TypeError where PySpark's getattr raises 
> "attribute name must be string, not 'int'".



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to