[
https://issues.apache.org/jira/browse/SPARK-61097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-61097:
------------------------------------
Assignee: DB Tsai
> Cancel pending streaming listener registration and avoid callback deadlocks
> during shutdown
> -------------------------------------------------------------------------------------------
>
> Key: SPARK-61097
> URL: https://issues.apache.org/jira/browse/SPARK-61097
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: DB Tsai
> Assignee: DB Tsai
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> The initial shared session manager and synchronous query-start delivery
> landed in SPARK-61162 (PR #353). This follow-up proposes the remaining
> listener lifecycle fixes in
> https://github.com/apache/spark-connect-rust/pull/384.
> Terminal SparkSession.stop() cancels local listener registration and dispatch
> without joining callbacks, releases the server session, and closes the client
> even when release fails. SparkConnectClient.close() cancels local
> registration and dispatch without releasing the session. Neither terminal
> path performs a listener-removal RPC. The current event's listener snapshot
> may finish after terminal return; later events are not dispatched. A
> registration whose RPC event stream is established but not yet acknowledged
> can be cancelled; initial stream establishment retains the transport deadline
> and retry policy.
> Lifecycle RPCs and worker joins run outside the state lock. Ordinary failed
> listener removal or close preserves registrations and worker state for retry,
> and generations prevent an old worker from clearing newer listeners. Real
> workers hold weak bus references so releasing the last native session or
> manager can release a quiet subscription. Regression coverage exercises
> callback reentrancy, terminal races, removal retry, stale generations,
> progress-handler lock reentrancy, cancellation wiring and real-worker
> ownership release.
> The fix retains upstream synchronous-start, duplicate-registration and
> foreign-session removal semantics. Listener-held Python session cycles and
> event-stream error warning parity remain limitations. This follow-up
> preserves connect-rust-4.3.0 release tracking.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
