jojochuang opened a new pull request, #11671: URL: https://github.com/apache/ozone/pull/11671
## What changes were proposed in this pull request? Add `hadoop.hdds.db.rocksdb.compaction.deletion.ratio` to request automatic compaction when the fraction of deletion tombstone entries in a newly created SST reaches the configured threshold. Standard DB profiles attach RocksDB's CompactOnDeletionCollectorFactory while preserving managed resource ownership. ### Caveats - This is opt-in and disabled by default (`0`). Values in `(0, 1]` enable the collector; a restart is required. - A suitable deletion ratio must be determined through empirical experiments with representative workloads. `0.5` in the tests is not a production recommendation. Evaluate read/list latency, compaction I/O, disk utilization, and write stalls before rollout. - The collector's configuration support requires RocksDB 8.11.3 or later. Ozone master already uses RocksDB 10.10.1.1 and provides the required Java bindings. - This is a shared configuration with scope across OM, SCM, and datanode RocksDB databases, rather than an OM-only setting or a table allowlist. Every column family using a standard DB profile is eligible, so consider the blast radius when distributing this setting cluster-wide. - Explicit column family options and RocksDB options files retain precedence and bypass this setting. This includes datanode schema V3 tables with explicitly supplied options; the implementation does not unconditionally cover every column family. - The collector evaluates SST entries, not memtable tombstones or the database-wide deletion ratio. It respects disabled automatic compaction; snapshots may prevent tombstone reclamation. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-12254 ## How was this patch tested? Using JDK 21: - `./hadoop-ozone/dev-support/checks/checkstyle.sh`: passed, zero violations. - `mvn -pl :hdds-server-framework,:hdds-managed-rocksdb test -Dtest=TestDBStoreBuilder,TestDBConfigFromFile,TestManagedRocksDB -Dsurefire.failIfNoSpecifiedTests=false -DskipShade -DskipRecon -DskipDocs`: 25 tests passed. Includes native automatic compaction under SSD/DISK profiles, disabled and below-threshold behavior, invalid ratios, options-file loading, and managed RocksDB reopening. - `mvn -pl :hdds-managed-rocksdb,:hdds-server-framework -Dspotbugs.version=4.9.7.0 -DskipShade -DskipRecon -DskipDocs -Djacoco.skip test-compile spotbugs:spotbugs`: completed successfully. The reports contain 138 findings in existing code; none have a primary source location overlapping changed lines. - The standard `findbugs.sh` invocation was attempted first, but the pinned SpotBugs 3.1.12.2 cannot analyze Java 17 class files (`Unsupported class file major version 61`). The compatible version was supplied only on the command line; no build dependency versions were changed. - `git diff --check`: passed. Generated-by: Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
