lucasfang opened a new pull request, #522: URL: https://github.com/apache/paimon-cpp/pull/522
### Purpose Fix two existing main-branch bugs identified in the review of #513: - [Validate byte ranges against file size](https://github.com/apache/paimon-cpp/pull/513#discussion_r4476202458). A cache for a 1 KiB file previously accepted a 32 MiB range starting at offset 512. Validate the entire registration against the known file size before the coalescer splits ranges or cache state changes. The subtraction-based check avoids integer overflow, rejects invalid late registrations atomically, and preserves zero as the unknown-size sentinel. - [Skip payload range queries when caching is disabled](https://github.com/apache/paimon-cpp/pull/513#discussion_r4476202461). With prefetch enabled, caching disabled and `warmup=NONE`, an unnecessary LM payload `PreBufferRange()` failure previously failed an otherwise readable scan. Install the callback only when the shared cache exists. ### Tests Five new regression tests cover out-of-file and overflowing ranges, invalid late registrations, exact EOF boundaries, unknown file sizes, and cache-disabled LM reads under NONE/RAW/DECODED warmup. The LM test makes every range query fail and verifies all filtered keys and payload values, so successful reads require skipping those queries. Before the production fix, the five tests produced the expected three failures and two compatibility passes. After the fix, all five pass. Native Debug validation (`build-rest`, ORC/Parquet/Avro/REST enabled): ```sh cmake --build build-rest --target paimon-common-test paimon-parquet-format-test paimon-orc-format-test -j16 ./build-rest/debug/paimon-common-test ./build-rest/debug/paimon-parquet-format-test ./build-rest/debug/paimon-orc-format-test pre-commit run --files src/paimon/common/utils/read_ahead_cache.cpp src/paimon/common/utils/read_ahead_cache.h src/paimon/common/utils/read_ahead_cache_test.cpp src/paimon/common/reader/prefetch_file_batch_reader_impl.cpp src/paimon/common/reader/late_materializing_file_batch_reader_test.cpp git diff --check ``` - Common: 1,862 passed; two existing parameterized cases skip because their cache is disabled. - Parquet: 240 passed. - ORC: 120 passed. - Formatting, lint and whitespace checks passed for all five changed files. The all-component build was attempted but could not finish because of local Rust dependency/toolchain incompatibilities (Vortex with Rust 1.67, then Lance/ethnum with installed stable Rust 1.97). The successful native build disables those optional Rust formats. Repository-wide `unittest`, integration executables and sanitizer builds were not run. ### API and Format No changes to installed public API, storage format or protocol. Malformed ranges beyond a known file size now return a validation error before allocation/splitting; file size zero retains its existing unknown-size behavior. ### Documentation Updated the internal `ReadAheadCache` constructor and `AddRanges()` comments to describe validation and the unknown-size convention. No new feature or configuration option. ### Generative AI tooling Generated-by: OpenAI Codex (codex-cli 0.166.0) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
