[ 
https://issues.apache.org/jira/browse/SPARK-62134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated SPARK-62134:
-----------------------------------
    Labels: pull-request-available  (was: )

> Move the client of the Python client into pyspark.sql.connect.client.core, of 
> PySpark's layout and names of the package and its core
> ------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-62134
>                 URL: https://issues.apache.org/jira/browse/SPARK-62134
>             Project: Spark
>          Issue Type: Improvement
>          Components: Connect
>    Affects Versions: 5.0.0
>            Reporter: Hyukjin Kwon
>            Priority: Major
>              Labels: pull-request-available
>
> In pyspark-client 4.3.0, pyspark/sql/connect/client/core.py defines the 
> client: SparkConnectClient, ChannelBuilder, DefaultChannelBuilder and 
> RpcDeadlines (its __all__), PlanObservedMetrics, AnalyzeResult, ConfigResult 
> and its other names; the package's __init__.py has no __all__ and runs `from 
> pyspark.sql.connect.client.core import *` and `from 
> pyspark.sql.connect.logging import getLogLevel`. The Python client 
> (pyspark-client-rust) has it the other way around: its package __init__.py 
> defines the client and its helpers, of an __all__ of its own, and core.py is 
> a shim to which the package assigns its classes. So the class of 
> SparkSession.client is pyspark.sql.connect.client.SparkConnectClient, where 
> PySpark's is pyspark.sql.connect.client.core.SparkConnectClient, which type 
> checkers, help(), inspect.getsource and the pickles of its functions show, 
> and the package has names that PySpark's has not.
> Expected: as pyspark-client, the client in core.py of PySpark's names and 
> __all__, and the package's __init__.py as PySpark's.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to