[
https://issues.apache.org/jira/browse/SPARK-57991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-57991:
-----------------------------------
Labels: pull-request-available (was: )
> Deflake BarrierTaskContextSuite wall-clock skew assertions on macOS-26
> ----------------------------------------------------------------------
>
> Key: SPARK-57991
> URL: https://issues.apache.org/jira/browse/SPARK-57991
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Tests
> Affects Versions: 5.0.0
> Reporter: Hyukjin Kwon
> Priority: Minor
> Labels: pull-request-available
>
> The `Build / Maven (Scala 2.13, JDK 21, MacOS-26)` scheduled workflow fails
> on the `core` module with:
> {code}
> BarrierTaskContextSuite:
> - successively sync with allGather and barrier *** FAILED ***
> 1078 was not less than or equal to 1000 (BarrierTaskContextSuite.scala:122)
> {code}
> Three tests in `BarrierTaskContextSuite` capture `System.currentTimeMillis()`
> in each task right after a `barrier()` / `allGather()` global sync and assert
> `times.max - times.min <= 1000`. That skew is bounded below by the barrier
> client's own polling granularity, not by sync correctness:
> `BarrierTaskContext.runBarrier` waits for the coordinator RPC in a
> `Thread.sleep(1000)` loop, so two tasks can observe the release up to ~1s
> apart from the poll interval alone, before any thread-scheduling or GC jitter
> on a busy CI host.
> SPARK-49983 previously observed 1038 and only halved the pre-barrier sleep,
> which reduces arrival spread but cannot reduce the poll-granularity skew, so
> the flake persists on slower runners (macOS-26 arm64 observed 1078).
> Reproduced locally on macOS 26 arm64 / JDK 21: post-sync skew reached 994ms
> while idle.
> Widen the tolerance to a documented constant that accounts for the poll
> interval plus jitter, keeping the test's intent (tasks stay loosely in
> lockstep; none races an extra sync ahead) without coupling to a razor-thin
> wall-clock margin.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
