Siyao Meng created HDDS-16135:
---------------------------------
Summary: Intermittent failure in
TestKeyLifecycleService$Normal#testNestedFSODirectoryScanResume
Key: HDDS-16135
URL: https://issues.apache.org/jira/browse/HDDS-16135
Project: Apache Ozone
Issue Type: Sub-task
Components: Ozone Manager, test
Reporter: Siyao Meng
{code}
[ERROR] Tests run: 148, Failures: 1, Errors: 1, Skipped: 6, Time elapsed: 99.18
s <<< FAILURE! -- in
org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal
[ERROR]
org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal.testNestedFSODirectoryScanResume(BucketLayout,
int)[2][2] -- Time elapsed: 0.336 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
at
org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal.testNestedFSODirectoryScanResume(TestKeyLifecycleService.java:797)
[ERROR]
org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal.testNestedFSODirectoryScanResume(BucketLayout,
int)[2][3] -- Time elapsed: 5.580 s <<< ERROR!
java.util.concurrent.TimeoutException:
{code}
The FSO directory-scan resume mechanism itself works, but the test races the
shared periodic {{KeyLifecycleService}} (fires every ~300ms) against a one-shot
fault injector. Only the first scan aborts at the intended resume point; the
next tick resumes and overwrites the persisted {{OmLifecycleScanState}}, so the
exact-value assertions at lines 785-808 depend on reading inside the ~300ms
window (the {{[2][2]}} {{lastScannedKey}} failure). {{getNumKeyIterated()}} is
a service-wide cumulative counter that concurrent re-scans inflate, so the
exact-equality wait can never settle, giving the {{[2][3]}} timeout. Same
fragility family as HDDS-16033 (different method). Suggested hardening: park
the scan at the resume point (blocking injector) for deterministic inspection,
and make the metric wait monotonic ({{>=}}) instead of exact.
Seen in:
-
https://github.com/adoroszlai/ozone-build-results/blob/master/2026/08/05/51974/integration-om/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal.txt
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
