[
https://issues.apache.org/jira/browse/SPARK-61246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61246:
---------------------------------
Fix Version/s: connect-rust-4.3.0
> Add the ChannelBuilder classes, the session registry, the observed
> server-side session ID and the other Connect session and client members of
> PySpark that the drop-in lacks
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61246
> URL: https://issues.apache.org/jira/browse/SPARK-61246
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> pyspark.sql.connect.client of the drop-in has no ChannelBuilder or
> DefaultChannelBuilder, so
> SparkSession.builder.channelBuilder(DefaultChannelBuilder(url)) cannot be
> written and the official test_connect_session.py fails at import. The Connect
> SparkSession lacks its class-level registry (_default_session,
> _active_session, _get_default_session(), _lock), release_session_on_close,
> and direct construction SparkSession(connection=..., userId=...);
> SparkConnectClient lacks _proto_to_string, and its _server_session_id returns
> the client's own session ID and cannot be set (official: None until a
> response carries one, settable). The official test_connect_context.py and
> test_connect_error.py fail on these.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
