Hyukjin Kwon created SPARK-61407:
------------------------------------
Summary: Let Ctrl-C end a call that waits for an unreachable or
retrying server, in the Python client as PySpark does
Key: SPARK-61407
URL: https://issues.apache.org/jira/browse/SPARK-61407
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
A follow-up of SPARK-61259, SPARK-61353 and SPARK-61388, found in a post-merge
review. A signal that arrives while no execution of the call is in flight (the
plan compression lookup and its retries before the first execution, or a hook)
does not end the wait, which ignores Watch::Abort for the waits of
interruptible calls, and after the interrupt of an execution the wait no longer
checks signals. So spark.range(1).collect() on an unreachable server blocks
through the whole retry budget after Ctrl-C, where PySpark raises
KeyboardInterrupt from its retry sleep within about 2 seconds, and an execution
that keeps retrying is not ended by a second Ctrl-C. A progress handler's
done-call error is also dropped when a signal ends the call.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]