[
https://issues.apache.org/jira/browse/SPARK-62134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-62134:
-----------------------------------
Labels: pull-request-available (was: )
> Move the client of the Python client into pyspark.sql.connect.client.core, of
> PySpark's layout and names of the package and its core
> ------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-62134
> URL: https://issues.apache.org/jira/browse/SPARK-62134
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: 5.0.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
>
> In pyspark-client 4.3.0, pyspark/sql/connect/client/core.py defines the
> client: SparkConnectClient, ChannelBuilder, DefaultChannelBuilder and
> RpcDeadlines (its __all__), PlanObservedMetrics, AnalyzeResult, ConfigResult
> and its other names; the package's __init__.py has no __all__ and runs `from
> pyspark.sql.connect.client.core import *` and `from
> pyspark.sql.connect.logging import getLogLevel`. The Python client
> (pyspark-client-rust) has it the other way around: its package __init__.py
> defines the client and its helpers, of an __all__ of its own, and core.py is
> a shim to which the package assigns its classes. So the class of
> SparkSession.client is pyspark.sql.connect.client.SparkConnectClient, where
> PySpark's is pyspark.sql.connect.client.core.SparkConnectClient, which type
> checkers, help(), inspect.getsource and the pickles of its functions show,
> and the package has names that PySpark's has not.
> Expected: as pyspark-client, the client in core.py of PySpark's names and
> __all__, and the package's __init__.py as PySpark's.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]