[
https://issues.apache.org/jira/browse/FLINK-40076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40076:
-----------------------------------
Labels: pull-request-available (was: )
> RescaleTimelineITCase.testRecordNonTerminatedRescaleMergingWithNewRecoverableFailureTriggerCause
> fails intermittently: snapshot taken before recoverable failover is merged
> into in-progress rescale
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-40076
> URL: https://issues.apache.org/jira/browse/FLINK-40076
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 2.4.0
> Reporter: Martijn Visser
> Priority: Major
> Labels: pull-request-available
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=76751&view=results
> (leg: test_cron_adaptive_scheduler core)
> {code}
> org.opentest4j.AssertionFailedError:
> expected: RECOVERABLE_FAILOVER
> but was: UPDATE_REQUIREMENT
> at
> org.apache.flink.runtime.scheduler.adaptive.timeline.RescaleTimelineITCase.lambda$testRecordNonTerminatedRescaleMergingWithNewRecoverableFailureTriggerCause$17(RescaleTimelineITCase.java:582)
> at
> org.apache.flink.runtime.scheduler.adaptive.timeline.RescaleTimelineITCase.runAdaptedParameterizedAssertion(RescaleTimelineITCase.java:630)
> at
> org.apache.flink.runtime.scheduler.adaptive.timeline.RescaleTimelineITCase.testRecordNonTerminatedRescaleMergingWithNewRecoverableFailureTriggerCause(RescaleTimelineITCase.java:574)
> {code}
> The test fails fast (~0.4s), commit 3196a998cf0. Distinct from the earlier
> races fixed in this class (FLINK-39902, FLINK-39903, FLINK-40009,
> FLINK-40010) and from the open FLINK-40067, which all target other methods.
> Root cause: after {{miniCluster.terminateTaskManager(0)}} the test
> synchronizes via {{waitForVertexParallelismReachedAndJobRunning(...,
> PARALLELISM)}}. {{CommonTestUtils.waitUntilCondition}} evaluates its
> predicate before sleeping, and immediately after the TaskManager loss the job
> is still observed RUNNING at PARALLELISM, so the wait returns on stale
> pre-failover state and provides no synchronization (after the failover the
> job runs at lower parallelism, so the condition can only ever match the stale
> state). The {{ExecutionGraphInfo}} snapshot can therefore be taken before
> {{AdaptiveScheduler.recordRescaleForJobRestarting}} merges the failover into
> the in-progress UPDATE_REQUIREMENT rescale and re-stamps its trigger cause to
> RECOVERABLE_FAILOVER. Product behavior is correct; the test synchronization
> is wrong.
> Proposed fix: before snapshotting, wait until the failover has actually been
> merged (rescale history size 2 with latest trigger cause
> RECOVERABLE_FAILOVER), gated on the enabled-history parameterization (the
> disabled-history leg asserts an empty history and needs no wait). Assertions
> unchanged. Verified locally with 25 iterations (50 executions across both
> parameterizations), all green.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
