lucasfang opened a new pull request, #513: URL: https://github.com/apache/paimon-cpp/pull/513
### Purpose Linked issue: Closes #512 With batch prefetch disabled, Parquet and ORC previously bypassed the shared read-ahead cache and could not warm the next file. Warmup also stopped when the format or adaptive strategy declined prefetch. This change makes the existing settings independent. The default `EnablePrefetch(false)`, cache-enabled, RAW configuration now caches and warms synchronous reads. RAW only fetches bytes; DECODED prepares one batch in the background before continuing synchronously. Active prefetch keeps its existing parallel decode loop. - Extract shared cache creation, range registration, late-materialization callbacks, deferred errors and I/O metrics into `FileReadAhead`. - Add a synchronous warmup wrapper that joins/discards pending work before schema reset, close or destruction; no prefetch executor or queue settings are required. - Route adaptive fallback through the same wrapper and let an active shared cache replace Parquet's own pre-buffering. - Preserve plain reads for other format plugins and account for the cache's separate stream in the main-reader reuse regression test. ### Tests Local Debug build with tests enabled: ```bash cmake --build build --target paimon-common-test paimon-core-test paimon-parquet-format-test paimon-orc-format-test -j 32 ./build/debug/paimon-common-test ./build/debug/paimon-core-test ./build/debug/paimon-parquet-format-test ./build/debug/paimon-orc-format-test git diff --check ``` All four complete test executables passed: common 1,864 passed / 2 conditional cache-disabled skips; core 2,622 passed; Parquet 240 passed; ORC 120 passed. Total: 4,846 passed, 2 skipped, 0 failures. Regression coverage includes real Parquet/ORC combinations of prefetch/cache/warmup settings, adaptive and format fallback, late materialization with predicates and bitmaps, file row IDs, multiple files, one-batch background preparation, repeated warmup, schema reset, deferred failures and unconsumed close/destruction. Controlled asynchronous streams and completion signals verify warmup behavior without timing-based ordering assertions. Read-only clang-format 20.1.8, cmake-format and whitespace checks passed, as did applicable pre-commit large-file, spelling, Sphinx and C++ lint hooks. The rewriting format/whitespace hooks were replaced by those read-only checks. Repository-wide `unittest`, integration executables and sanitizer builds were not run locally. ### API and Format No public signature, enum value, configuration default, storage format or protocol change. Public API comments now describe independent operation. Behavior changes intentionally: synchronous Parquet/ORC reads now honor the already-enabled read-ahead cache and RAW default. DECODED without prefetch buffers one batch; RAW requires the cache. Cache/I/O metrics remain available independently of batch prefetch, and synchronous warmup reports prefetch disabled. ### Documentation Update the prefetch/warmup guide, I/O metrics guide and API comments. Retain documentation of memory costs, early-stop cleanup, and RAW's synchronous-filesystem limitation. ### Generative AI tooling Generated-by: OpenAI Codex (codex-cli 0.166.0) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
