zeruibao opened a new pull request, #57410: URL: https://github.com/apache/spark/pull/57410
### What changes were proposed in this pull request? This PR makes `AsyncEventQueue` shutdown robust to thread interruption. In `AsyncEventQueue.stop()`, two blocking operations are made uninterruptible: - Enqueuing the POISON_PILL now uses `Uninterruptibles.putUninterruptibly(eventQueue, POISON_PILL)` instead of eventQueue.put(POISON_PILL). - Joining the dispatch thread now uses Uninterruptibles.joinUninterruptibly(...), preserving the existing `spark.scheduler.listenerbus.exitTimeout` semantics (wait indefinitely when the timeout is 0, otherwise wait up to the configured timeout). Both operations restore the caller's interrupt status before returning, so interruption is not swallowed. ### Why are the changes needed? `AsyncEventQueue.stop()` can be invoked from a thread that has already been interrupted. For example, a Structured Streaming execution thread being torn down. In that case `eventQueue.put(POISON_PILL)` throws InterruptedException before the poison pill is enqueued. Because stopped has already been flipped to true, no further poison pill is ever enqueued, and the dispatch thread (spark-listener-group-*) blocks forever on `eventQueue.take()`, leaking the thread and preventing a clean shutdown. Making the enqueue and join uninterruptible guarantees the poison pill is delivered and the dispatch thread is joined even when the caller is interrupted. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Added a regression test in SparkListenerSuite, "removing the last listener is uninterruptible", which removes the last listener (triggering a queue stop) from an already-interrupted thread and asserts that the removal does not throw, the caller's interrupt status is preserved, and the queue is stopped. ### Was this patch authored or co-authored using generative AI tooling? Cursor (Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
