This is an automated email from the ASF dual-hosted git repository.
gaogaotiantian pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 3fce4cfa3421 [SPARK-57073][SS][PYTHON][TEST] Catch AnalysisException
for test_parity_listener
3fce4cfa3421 is described below
commit 3fce4cfa34213adb30633ee942ceffcc9d58b922
Author: Tian Gao <[email protected]>
AuthorDate: Thu Jun 11 15:44:42 2026 -0700
[SPARK-57073][SS][PYTHON][TEST] Catch AnalysisException for
test_parity_listener
### What changes were proposed in this pull request?
Catch `AnalysisException` for the test and ignore it because it's
acceptable (table not created yet).
### Why are the changes needed?
The test has been flaky on the Build / Python-only (master, Python 3.12,
MacOS26) scheduled workflow:
2026-05-23 —
https://github.com/apache/spark/actions/runs/26346300968/job/77556662680
2026-05-25 —
https://github.com/apache/spark/actions/runs/26423905857/job/77783724134
Both failed with AnalysisException: TABLE_OR_VIEW_NOT_FOUND on
listener_terminated_events: the onQueryTerminated callback fires asynchronously
after q.stop() returns and writes the table via saveAsTable, but on slower
macOS runners the read races the write.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Local test passed. This fails on MacOS more frequently so we need to
observe it in scheduled tests.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #56309 from gaogaotiantian/test-parity-listener.
Authored-by: Tian Gao <[email protected]>
Signed-off-by: Tian Gao <[email protected]>
---
.../pyspark/sql/tests/connect/streaming/test_parity_listener.py | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/python/pyspark/sql/tests/connect/streaming/test_parity_listener.py
b/python/pyspark/sql/tests/connect/streaming/test_parity_listener.py
index b9583c4a11fa..a8b46c38db4f 100644
--- a/python/pyspark/sql/tests/connect/streaming/test_parity_listener.py
+++ b/python/pyspark/sql/tests/connect/streaming/test_parity_listener.py
@@ -18,6 +18,7 @@
import time
import pyspark.cloudpickle
+from pyspark.errors import AnalysisException
from pyspark.sql.tests.streaming.test_streaming_listener import
StreamingListenerTestsMixin
from pyspark.sql.streaming.listener import StreamingQueryListener
from pyspark.sql.functions import count, lit
@@ -257,7 +258,13 @@ class
StreamingListenerParityTests(StreamingListenerTestsMixin, ReusedConnectTes
@eventually(timeout=60, catch_assertions=True)
def load_event(event_name, table_name):
- table = self.spark.read.table(table_name).collect()
+ try:
+ table = self.spark.read.table(table_name).collect()
+ except AnalysisException as e:
+ # It's possible that the table has not been created yet
+ if e.getCondition() == "TABLE_OR_VIEW_NOT_FOUND":
+ return False
+ raise e
if len(table) == 0:
return False
events[event_name] = pyspark.cloudpickle.loads(table[0][0])
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
