[
https://issues.apache.org/jira/browse/HDDS-16884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-16884:
----------------------------------
Labels: ozone-flaky-test pull-request-available (was: ozone-flaky-test)
> Intermittent timeout in
> TestSnapshotBackgroundServices#testBackupCompactionFilesPruningBackgroundService
> --------------------------------------------------------------------------------------------------------
>
> Key: HDDS-16884
> URL: https://issues.apache.org/jira/browse/HDDS-16884
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Huang Kuan Hao
> Assignee: Huang Kuan Hao
> Priority: Major
> Labels: ozone-flaky-test, pull-request-available
>
> {{TestSnapshotBackgroundServices#testBackupCompactionFilesPruningBackgroundService}}
> intermittently times out waiting for the backup SST directory's file count
> to decrease after pruning is resumed.
> Observed CI failure:
> https://github.com/apache/ozone/actions/runs/35732766354/job/106767352881
> {code}
> TestSnapshotBackgroundServices.testBackupCompactionFilesPruningBackgroundService:504
> -> checkIfCompactionBackupFilesWerePruned:574
> -> TimeoutException
> {code}
> The unchanged full class also reproduced the timeout locally on master
> {{6818707698}} with JDK 21 (4 tests, 1 error):
> {code}
> mvn -pl :ozone-integration-test test -Dtest=TestSnapshotBackgroundServices
> -DskipShade -DskipRecon -DskipDocs
> {code}
> Snapshot creation does not guarantee a completed tracked compaction. The
> fixture relies on automatic compaction and shared-cluster state to produce
> eligible backup SST inputs. Passing diagnostic runs removed inputs from a
> previous test. A net directory count can also hide removals when new backup
> files are created.
> Disabling automatic compaction with the existing {{DBProfile.TEST}}
> reproduced the missing precondition: the backup directory, compaction log and
> DAG remained empty, and the original wait timed out. This establishes a
> fixture defect; the historical CI log does not contain enough metadata to
> confirm its exact trigger.
> The proposed fix forces a keyTable compaction using
> {{CompactDBUtil.compactTable}} with bottommost {{kForce}} while pruning is
> suspended, captures existing backup inputs from newly recorded compactions,
> asserts that they are non-empty, and waits for those files to disappear after
> pruning resumes. It preserves the leadership-transfer and snapshot-diff
> checks and the existing 10-second timeout. Default {{compactDB()}} produced
> no new inputs in a verification run, so forcing bottommost compaction is
> needed.
> Local validation of the test-only fix:
> * The controlled no-automatic-compaction probe fails with the original test
> and passes with the fix.
> * Five separate runs of the normal-profile full class passed: 20 tests, 0
> failures/errors/skipped.
> * Checkstyle passed with 0 violations.
> Related: HDDS-9206 (resolved older pruning timeout), HDDS-10306 (shared test
> cluster).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
