smengcl opened a new pull request, #11084: URL: https://github.com/apache/ozone/pull/11084
## What changes were proposed in this pull request? Ozone's RocksDB-backed table iterators currently position themselves at the table or prefix start during construction. Callers that then explicitly seek to another key perform two native positioning operations: 1. Seek to the table or prefix start during iterator construction. 2. Seek to the caller's requested key. The first operation is redundant. A RocksDB seek to the logical beginning is not merely assigning a cursor to the beginning of one file: RocksDB must merge its internal sources and find the first visible key, which can require skipping a large run of uncompacted tombstones. This PR defers initial positioning until the first iterator operation: - `seek()`, `seekToFirst()`, and `seekToLast()` position the iterator directly without an earlier seek. - `hasNext()`, direct `next()`, and `removeFromDB()` preserve the existing behavior by initializing at the table or prefix start on first use. - The behavior is implemented in `RDBStoreAbstractIterator` and applies consistently to the byte-array and `CodecBuffer` iterators. - The native RocksDB iterator is still created immediately, preserving its database view and lifetime. The PR also uses a RocksDB point Get for `RDBTable#getRangeKVs` when all of the following apply: - `startKey` is non-null. - `count` is exactly one. - No key filter is present. - The range contract requires the exact start key to exist. - The start key belongs to the requested prefix. Prefix-boundary, filtered, multi-entry, and other ordered range requests continue using an iterator. The change does not modify any public API, wire format, RocksDB schema, metadata layout, or caller-visible iterator semantics. ### Expected benefit The change eliminates one native RocksDB positioning operation when a caller creates an iterator and explicitly repositions it. The absolute latency reduction equals the cost of the eliminated table-start or prefix seek. Production paths that can benefit include: - `listKeys` and S3 `ListObjects`: OM creates a key-table iterator and seeks to the requested bucket, prefix, or continuation marker. First pages can benefit when the bucket sorts after a tombstone-heavy region, while later pages can benefit when the continuation marker is beyond deleted keys in the bucket. - OzoneFS `listStatus`: FSO listings use directory and file prefix iterators that may seek to a later start key. LEGACY and OBJECT_STORE listings also use a table iterator followed by a start-key seek. - `listOpenFiles`: OM seeks an open-key-table iterator to the requested path or continuation token. - `ListMultipartUploads`: paginated requests seek a bucket-prefix iterator to the key and upload-ID marker. - Key lifecycle scans: resumed scans can seek directly to the saved last-scanned key after suspension, restart, or leadership transfer. - Snapshot and Recon pagination: listings and endpoints using start-key or previous-key markers avoid the same redundant positioning. - Exact single-entry `getRangeKVs` requests avoid iterator creation and positioning entirely when they meet the point-Get conditions. A standalone synthetic benchmark used RocksDB JNI 10.10.1.1, a warm cache, one thread, disabled automatic compaction, and consecutive point tombstones before a live target. The benchmark measures RocksDB positioning and exact-key operations, not end-to-end Ozone RPC latency. When the continuation target was beyond the tombstone band: | Tombstones | Double seek | Single seek | Positioning speedup | |---:|---:|---:|---:| | 10,000 | 819.5 µs | 1.1 µs | approximately 756× | | 100,000 | 8.27 ms | 1.2 µs | approximately 6,850× | | 500,000 | 96.7 ms | 1.5 µs | approximately 66,000× | When both positioning operations crossed the tombstone region, removing one of them produced approximately a 2× positioning improvement. Exact count-one range requests were approximately 3× faster at the method level, saving roughly 1–2 µs per warm-cache operation. Actual request-level improvement depends on tombstone distribution, compaction state, cache state, result decoding, OM cache merging, storage latency, and RPC processing. Little improvement is expected for compacted tables, low tombstone counts, or callers that begin ordinary iteration without explicitly repositioning the iterator. The standalone `RocksTombstoneBench.java` reproducer can be provided as a separate patch. Its header contains the exact compilation and execution commands and documents the benchmark conditions. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-16250 ## How was this patch tested? Regression tests verify that: - Iterator construction performs no native seek. - `hasNext()` and direct `next()` still initialize at the table or prefix beginning. - Repeated `hasNext()` calls do not repeat initialization. - An explicit seek on a prefixed iterator performs exactly one native seek to the requested key. - `seekToLast()` does not first seek to the beginning. - `removeFromDB()` preserves its implicit-start behavior. - Byte-array and `CodecBuffer` iterators have identical positioning behavior. - Present and absent exact count-one ranges use a point Get without creating an iterator. - Returned keys do not alias the caller's start-key array. - A start key equal to the prefix retains lower-bound range semantics. - Filtered requests continue until a matching key is found. - Multi-entry and out-of-prefix requests retain ordered-range semantics. Validation results: - Targeted `hdds-server-framework` tests: 52 passed with no failures or errors. - Broader `hdds-server-framework` suite excluding `TestDU`: 598 passed and 1 skipped. - `TestDU.testExcludePattern` fails with the same error on clean master on macOS and is unrelated to this change. - Repository-wide checkstyle: all 58 modules passed. - The standalone RocksDB synthetic benchmark compiled and completed successfully under JDK 21. A wall-clock performance assertion is intentionally excluded from the unit suite because it would depend on host timing, cache state, storage, and RocksDB compaction state. The regression tests instead verify the number and order of native positioning operations deterministically. Generated-by: Codex (GPT-5.6 Sol) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
