XiaoHongbo-Hope opened a new pull request, #9130: URL: https://github.com/apache/paimon/pull/9130
### Purpose Ray currently reads each inference batch independently. With small batches, nearby ranges from the same Blob file can land in separate tasks, so the existing range coalescing cannot combine them. This adds an opt-in `blob_uri_affinity` mode to `map_with_blobs` that: - clusters descriptors by a deterministic URI hash and offset; - coalesces reads across bounded prefetch windows while preserving the UDF batch size; - keeps the existing no-shuffle path unchanged by default. The documentation calls out the distributed-sort cost and the `prefetch_bytes` memory bound. ### Tests - Related Blob, Ray, and row-id suites: 194 passed. - Focused real-Ray test passes on Ray 2.51 and 2.56. - Interleaved ranges from two Blob files with `batch_size=1`: read groups decrease from 4 to 2 while the UDF batch size remains 1. - Real local Paimon Blob read path is covered. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
