This is an automated email from the ASF dual-hosted git repository.
dbtsai pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 76c581db97cc [SPARK-57482][CORE][TEST] Fix flaky
SparkLauncherSuite.testInProcessLauncher under CI load
76c581db97cc is described below
commit 76c581db97cc7b556ea98183f883135affd45050
Author: DB Tsai <[email protected]>
AuthorDate: Mon Jun 15 21:54:15 2026 -0700
[SPARK-57482][CORE][TEST] Fix flaky
SparkLauncherSuite.testInProcessLauncher under CI load
### What changes were proposed in this pull request?
`SparkLauncherSuite.testInProcessLauncher` waits for the connection between
the in-process app and the launcher to be established by polling the app handle
state with `eventually(Duration.ofSeconds(5), Duration.ofMillis(10))`. Under
heavy CI load this 5-second window is too short: the handle can remain in
`UNKNOWN` for longer, causing the test to fail with:
```
java.lang.IllegalStateException: Failed check after 476 tries: expected:
not equal but was: <UNKNOWN>.
at org.apache.spark.launcher.BaseSuite.eventually(BaseSuite.java:88)
at
org.apache.spark.launcher.SparkLauncherSuite.inProcessLauncherTestImpl(SparkLauncherSuite.java:162)
at
org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher(SparkLauncherSuite.java:130)
```
This change increases the timeout to 30 seconds with a 100ms poll interval,
consistent with `waitForSparkContextShutdown` (30s/100ms) and the other
`eventually` calls in this suite (60s/1000ms).
### Why are the changes needed?
`SparkLauncherSuite.testInProcessLauncher` is flaky under CI load. The
change only relaxes a test timeout; it does not change production behavior.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing test `SparkLauncherSuite.testInProcessLauncher`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8
Closes #56529 from dbtsai/dbtsai/fix-flaky-inprocess-launcher.
Authored-by: DB Tsai <[email protected]>
Signed-off-by: DB Tsai <[email protected]>
(cherry picked from commit 1cf08e6e09f9e1e4807e51883987b16e981fce6d)
Signed-off-by: DB Tsai <[email protected]>
---
core/src/test/java/org/apache/spark/launcher/SparkLauncherSuite.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git
a/core/src/test/java/org/apache/spark/launcher/SparkLauncherSuite.java
b/core/src/test/java/org/apache/spark/launcher/SparkLauncherSuite.java
index 6c1e84f6157c..055eaf6ccae6 100644
--- a/core/src/test/java/org/apache/spark/launcher/SparkLauncherSuite.java
+++ b/core/src/test/java/org/apache/spark/launcher/SparkLauncherSuite.java
@@ -152,8 +152,10 @@ public class SparkLauncherSuite extends BaseSuite {
// SPARK-23020: see doc for InProcessTestApp.LOCK for a description of
the race. Here
// we wait until we know that the connection between the app and the
launcher has been
// established before allowing the app to finish.
+ // Use a generous timeout because, under heavy CI load, establishing
the connection
+ // between the in-process app and the launcher can take longer than a
few seconds.
final SparkAppHandle _handle = handle;
- eventually(Duration.ofSeconds(5), Duration.ofMillis(10), () -> {
+ eventually(Duration.ofSeconds(30), Duration.ofMillis(100), () -> {
assertNotEquals(SparkAppHandle.State.UNKNOWN, _handle.getState());
});
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
