XiaoHongbo-Hope opened a new pull request, #1218: URL: https://github.com/apache/paimon-rust/pull/1218
## What changes Coalesce consecutive missing cache blocks into bounded source range reads, rather than awaiting one request per block. Cached holes split requests; cache-block overfetch is unchanged. Batches are bounded to 32 blocks and 8 MiB; larger individual blocks keep existing behavior. Preserve shared download locks, generation checks, and bounded background publication. No cache format or option changes. ## Validation Added coverage for mixed hits/misses, partial tails, batch bounds, overlapping reads, truncated responses, invalidation, and shared background publication. Formatting and diff checks pass. Current-main focused test build is pending dependency downloads; keeping this PR draft. A preliminary experiment on an earlier implementation reduced single-process cold OSS index planning from 6.26 to 3.50 seconds with 1 MiB blocks, and 15.60 to 3.74 seconds with 256 KiB blocks (medians of three fresh processes per arm). Download volume and output hashes were unchanged. These are not benchmarks of this current-main port or end-to-end training throughput. AI-assisted implementation. Current-head cross-process contention and end-to-end throughput have not been benchmarked. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
