Gabriel39 opened a new pull request, #66485: URL: https://github.com/apache/doris/pull/66485
## What changed - Probe FileCache with the exact requested range before allowing `MergeRangeFileReader` to coalesce remote reads. A full cache hit now bypasses merged IO. - Expose remote Parquet predicate-column ranges only when their execution stage is reached. Local readers keep the existing eager coalescing plan to avoid extra small reads. - Add a bounded per-reader cache-block buffer for external files. A block is promoted after repeated access and the buffer is capped at 16 MB per reader. - Publish `FileReadBytes` as per-scanner deltas so sibling scanners cannot overwrite each other, and add `ExactCacheHitBytes`, `MergedUsefulBytes`, `MergedGapBytes`, and `FuturePredicatePrefetchBytes` profile counters. This follows the same cache-before-coalescing layering used by StarRocks while keeping the new memory reuse and staged scheduling isolated from local and Doris-internal readers. ## Performance Release build, one pinned CPU, 10 repetitions, CPU-time median. The local cases are negative controls: they do not use the remote/FileCache-only staged path. | Case | Baseline CPU median | Patch CPU median | Delta | Baseline / patch CV | |---|---:|---:|---:|---:| | Compound residual scan | 1.520 ms | 1.522 ms | +0.13% | 7.06% / 4.77% | | Multi-column OR, Page Index off | 9.834 ms | 10.207 ms | +3.79% | 4.26% / 7.25% | | Multi-column OR, Page Index on | 6.663 ms | 6.532 ms | -1.97% | 6.25% / 4.07% | The observed deltas remain within run-to-run variance. Remote behavior is validated deterministically: full cache hits perform zero remote/merged reads, cache misses preserve range coalescing, later predicate ranges remain hidden until activation, and promoted cache blocks remain readable without another cache-file read. ## Validation - Release benchmark target built successfully with `-j128`. - ASAN `doris_be_test` target built successfully with `-j128`. - 13 focused Scanner V2, Parquet scheduler, MergeRange, and FileCache tests passed. - 36 full `BufferedReaderTest` and `FileScannerV2Test` tests passed. - 11 related `CachedRemoteFileReader` tests passed. - `clang-format --dry-run --Werror` and `git diff --check` passed. Depends on #66412. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
