This is an automated email from the ASF dual-hosted git repository.
uros-b pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new fb2360b2dffa [SPARK-58681][CORE][SQL][TEST] Fix race in job
cancellation tests
fb2360b2dffa is described below
commit fb2360b2dfface0d51579a9e6866c9d9da85c9aa
Author: Manu Zhang <[email protected]>
AuthorDate: Sat Aug 29 15:41:46 2026 +0000
[SPARK-58681][CORE][SQL][TEST] Fix race in job cancellation tests
### What changes were proposed in this pull request?
This pull request updates the test listeners in
`SparkSessionJobTaggingAndCancellationSuite` and `JobCancellationSuite` so they
increment `jobEnded` before releasing the semaphore that wakes the test thread.
A short comment documents the required ordering.
### Why are the changes needed?
`Semaphore.release()` could wake the test thread before
`jobEnded.incrementAndGet()` completed. The test could then observe the
previous counter value and fail intermittently with `2 did not equal 3`.
Updating the counter before releasing the semaphore establishes the
required ordering between the listener callback and the waiting test thread.
JIRA: https://issues.apache.org/jira/browse/SPARK-58681
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The affected existing tests passed with:
```
build/sbt 'core/testOnly org.apache.spark.JobCancellationSuite -- -z "job
tags"'
build/sbt 'sql/testOnly
org.apache.spark.sql.SparkSessionJobTaggingAndCancellationSuite -- -z
"Cancellation APIs in SparkSession are isolated"'
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)
Closes #57884 from manuzhang/codex/fix-spark-session-cancellation-test-race.
Authored-by: Manu Zhang <[email protected]>
Signed-off-by: Uros Bojanic <[email protected]>
(cherry picked from commit 099a4534965befba19beca26bc5dfe90255706f3)
Signed-off-by: Uros Bojanic <[email protected]>
---
core/src/test/scala/org/apache/spark/JobCancellationSuite.scala | 3 ++-
.../apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/core/src/test/scala/org/apache/spark/JobCancellationSuite.scala
b/core/src/test/scala/org/apache/spark/JobCancellationSuite.scala
index da6b57a0bccb..a4c4aaf8f4c2 100644
--- a/core/src/test/scala/org/apache/spark/JobCancellationSuite.scala
+++ b/core/src/test/scala/org/apache/spark/JobCancellationSuite.scala
@@ -317,8 +317,9 @@ class JobCancellationSuite extends SparkFunSuite with
Matchers with BeforeAndAft
}
override def onJobEnd(jobEnd: SparkListenerJobEnd): Unit = {
- sem.release()
+ // Update the counter before waking the test thread so it observes
the new value.
jobEnded.incrementAndGet()
+ sem.release()
}
})
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala
index 85de26b7a567..7e5da11256ae 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala
@@ -160,8 +160,9 @@ class SparkSessionJobTaggingAndCancellationSuite
}
override def onJobEnd(jobEnd: SparkListenerJobEnd): Unit = {
- sem.release()
+ // Update the counter before waking the test thread so it observes
the new value.
jobEnded.incrementAndGet()
+ sem.release()
}
})
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
