XiaoHongbo-Hope opened a new pull request, #11034: URL: https://github.com/apache/paimon/pull/11034
### Purpose Sampling frames throughout a small remote video can spend more time on repeated range requests than decoding. Add explicit, bounded whole-payload prefetch to `VideoFrameCollator`, exposed to LeRobot through `read.video.prefetch-max-bytes`. The default is **0 (disabled)**. Prefetch admits a payload only when it fits the remaining encoded-byte budget across cached decoders in that collator. Otherwise it preserves range reads. The budget is per video feature and worker; decoded frames, codec buffers and temporary copies are additional, and independent decoder replicas consume it separately. Existing decoder eviction/close releases the buffered payload. Indexed range callbacks use the prefetched bytes as well, rather than fetching the same media again through FileIO. This changes the read policy only; no video format, re-encoding or metadata migration is required. Documentation covers the bandwidth/memory tradeoff and the public collator argument. It does not automatically select a policy based on access density. ### Tests - Real MP4 indexed decoding matches range reads, including reordered and duplicate frames; prefetch performs one media read and repeated batches perform no further media reads. - Added coverage for cached-decoder reuse, aggregate encoded-payload admission, disabled/oversized fallback, budget release on close, truncated payload cleanup, invalid arguments, and LeRobot option propagation. - Related video/collator/LeRobot/TorchCodec suites: **118 passed, 63 skipped, 1 deselected** in the local optional-dependency environment. - The deselected `test_video_batches_include_delta_frames_and_preserve_backends` reproduces unchanged on base `b000c3945f` (combined-call expectation versus per-window batching). - Flake8 with the repository configuration and `git diff --check` pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
