[
https://issues.apache.org/jira/browse/SPARK-61353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-61353:
-----------------------------------
Labels: pull-request-available (was: )
> Raise KeyboardInterrupt promptly on SIGINT during SparkSession.stop and the
> native KMeans fit, in the Python client as PySpark does
> -----------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61353
> URL: https://issues.apache.org/jira/browse/SPARK-61353
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> SPARK-61259 made the waits of the actions, writers, commands, streams,
> catalog calls, ML fits and pipelines of the Python client raise the
> KeyboardInterrupt of a SIGINT promptly, as PySpark does. Two waits still
> ignore it:
> - SparkSession.stop(): it waits for the release of the session and for the
> retries of that release (about 10 minutes with the default retry policy
> against an unreachable server since SPARK-61319). PySpark's stop() raises
> KeyboardInterrupt at once from the sleep between the retries, and leaves the
> session open. A wait that nothing is executing on the server for cannot be
> interrupted on the server, so the wait of SPARK-61259 keeps waiting until the
> release ends.
> - The fit of the native KMeans of SPARK-61229, which merged after SPARK-61259
> was written and still waits without polling the signals.
> Let a wait end at once when a signal handler raised and no execution is in
> flight (Watch::Abort, honored by a new block_on_abortable that the release of
> stop() uses), so that stop() raises the exception at once and leaves the
> session open as PySpark's does, and make the KMeans fit interruptible as the
> other fits are.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]