Hyukjin Kwon created SPARK-61526:
------------------------------------
Summary: Let a signal end the wait for the next event of
streamListenerEvents, and run the interrupt tests with the SIGINT handler that
a background start ignores
Key: SPARK-61526
URL: https://issues.apache.org/jira/browse/SPARK-61526
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
>From a review of SPARK-61470 (Ctrl-C ends every wait of the Python client for
>the server) and its follow-ups:
1. The next() of spark.streams.streamListenerEvents(), an extra of this client,
still waited for the server's next event with a bare py.detach. A Ctrl-C was
ignored until an event came or the reattach retries ended. It now waits through
interruptible, with the session of the stream, as the other waits do.
2. The interrupt tests of the Python suites dropped every SIGINT when pytest
was started as a background job of a non-interactive shell (`cmd &`, `nohup ...
&`). Such a job inherits SIGINT as ignored, and Python leaves it ignored, so
each SIGINT test waited out its fallback (the retry budget, a subprocess
timeout) and failed. This is the cause of the runs where all ~20 SIGINT tests
failed together in about 30 minutes. python/tests/conftest.py now installs
Python's default handler when SIGINT is ignored at startup.
3. After SPARK-61470's follow-up, the first signal already ends the wait of
addListener for the server's acknowledgement. So the test of a second signal
during that wait no longer sent its second signal. The test now makes the
interrupt of the first signal retry, so that only the second signal ends the
wait.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]