smengcl opened a new pull request, #11244:
URL: https://github.com/apache/ozone/pull/11244
Generated-by: Claude Code (Opus 4.8)
## What changes were proposed in this pull request?
`TestContainerReconciliationWithMockDatanodes` fails intermittently in
`testContainerReconciliationFailureContainerScan`:
```
[ERROR] testContainerReconciliationFailureContainerScan Time elapsed: 0.005
s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
at
...TestContainerReconciliationWithMockDatanodes$MockDatanode.scanContainer(TestContainerReconciliationWithMockDatanodes.java:561)
at
...TestContainerReconciliationWithMockDatanodes.testContainerReconciliationFailureContainerScan(TestContainerReconciliationWithMockDatanodes.java:316)
```
The container replicas and the `OnDemandContainerScanner` are created once
in `@BeforeAll` and shared by every test method. The
`MockDatanode.scanContainer` helper asserts that
`OnDemandContainerScanner.scanContainerWithoutGap(...)` returns a present
`Future`. That method returns an empty `Optional` when the container is already
registered as in progress in `containerRescheduleCheckSet`; the entry is only
removed by the scan executor task after the scan finishes.
`KeyValueHandler.reconcileContainer` schedules a fire-and-forget scan via
`scanContainerWithoutGap` in a `finally` block, so every reconciliation test
leaves one scan in flight. Those tests wait via
`waitForExpectedScanCount(...)`, which polls the `numContainersScanned` metric
that is incremented inside the scan, before the executor thread runs
`removeContainerFromScheduledContainers`. When the next test's synchronous
`scanContainer` runs before that removal, no scan is scheduled, the `Optional`
is empty, and the assertion fails. The `withoutGap` helper uses `minScanGap =
0`, so the scan-gap path cannot cause the empty `Optional`, and this test
introduces no corruption, so the volume-failed path is not exercised either;
the only cause is the still-registered prior scan.
The fix makes `MockDatanode.scanContainer` wait until
`scanContainerWithoutGap` returns a scheduled `Future` before blocking on it,
instead of asserting the first call always schedules one. The single-thread
scan executor is guaranteed to drain the prior scan, so this is deterministic
by construction and keeps the per-call scan count at exactly one, preserving
the existing `waitForExpectedScanCount` assertions.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16423
## How was this patch tested?
Ran the full test class (which runs the reconciliation tests that leave a
scan in flight before the failing test) offline:
```
mvn -pl :hdds-container-service test \
-Dtest='TestContainerReconciliationWithMockDatanodes' \
-DskipShade -DskipRecon -DskipDocs -Dsurefire.failIfNoSpecifiedTests=false
```
A single green run cannot prove the absence of a race, so the fix is
deterministic by construction: the only transient reason
`scanContainerWithoutGap` returns an empty `Optional` here is a prior scan
still registered in `containerRescheduleCheckSet`, and the single-thread scan
executor is guaranteed to drain it, after which the retry schedules exactly one
scan.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
