[ 
https://issues.apache.org/jira/browse/SPARK-61353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hyukjin Kwon reassigned SPARK-61353:
------------------------------------

    Assignee: Hyukjin Kwon

> Raise KeyboardInterrupt promptly on SIGINT during SparkSession.stop and the 
> native KMeans fit, in the Python client as PySpark does
> -----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-61353
>                 URL: https://issues.apache.org/jira/browse/SPARK-61353
>             Project: Spark
>          Issue Type: Bug
>          Components: Connect
>    Affects Versions: connect-rust-4.3.0
>            Reporter: Hyukjin Kwon
>            Assignee: Hyukjin Kwon
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: connect-rust-4.3.0
>
>
> SPARK-61259 made the waits of the actions, writers, commands, streams, 
> catalog calls, ML fits and pipelines of the Python client raise the 
> KeyboardInterrupt of a SIGINT promptly, as PySpark does. Two waits still 
> ignore it:
> - SparkSession.stop(): it waits for the release of the session and for the 
> retries of that release (about 10 minutes with the default retry policy 
> against an unreachable server since SPARK-61319). PySpark's stop() raises 
> KeyboardInterrupt at once from the sleep between the retries, and leaves the 
> session open. A wait that nothing is executing on the server for cannot be 
> interrupted on the server, so the wait of SPARK-61259 keeps waiting until the 
> release ends.
> - The fit of the native KMeans of SPARK-61229, which merged after SPARK-61259 
> was written and still waits without polling the signals.
> Let a wait end at once when a signal handler raised and no execution is in 
> flight (Watch::Abort, honored by a new block_on_abortable that the release of 
> stop() uses), so that stop() raises the exception at once and leaves the 
> session open as PySpark's does, and make the KMeans fit interruptible as the 
> other fits are.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to