Huang Kuan Hao created HDDS-16908:
-------------------------------------
Summary: Intermittent failure in
TestOzoneManagerDoubleBufferWithDummyResponse metrics assertions
Key: HDDS-16908
URL: https://issues.apache.org/jira/browse/HDDS-16908
Project: Apache Ozone
Issue Type: Bug
Reporter: Huang Kuan Hao
Assignee: Huang Kuan Hao
TestOzoneManagerDoubleBufferWithDummyResponse intermittently fails at the
queue-size metrics assertion:
{noformat}
expected: <101> but was: <100>
TestOzoneManagerDoubleBufferWithDummyResponse.java:122
{noformat}
The cumulative flushed count, internal flushed count, and bucket-table
row-count assertions at 101 pass before this failure.
The test casts MutableStat.lastStat().total() to long and compares it with the
cumulative transaction count. SampleStat.total() is calculated as mean *
numSamples. With flush batches [1, 1, 98, 1], Hadoop 3.4.3 produces
100.99999999999999, which truncates to 100 even after all 101 transactions have
flushed. This was reproduced using the existing OzoneManagerDoubleBufferMetrics
implementation.
The assertion also depends on metrics sampling intervals, which may reset
between batches. The cumulative counter is updated before queueSize in
updateFlush(), so waiting for that counter does not guarantee that all metrics
have been updated. CI logs do not identify which of these mechanisms occurred
in this particular run.
The proposed test-only fix waits for flush completion before verifying
cumulative counters and persisted entries, and verifies interval statistics
separately using an unregistered metrics instance with explicit sampling and
floating-point tolerance. Production code remains unchanged.
Failure:
https://github.com/rich7420/ozone/actions/runs/38119024900/job/114411941856
Earlier reports for this test, HDDS-2368 and HDDS-4767, are resolved; this
issue tracks the current metrics assertion failure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
