XiaoHongbo-Hope opened a new pull request, #8807: URL: https://github.com/apache/paimon/pull/8807
### Purpose Pipeline split reads in `to_arrow_batch_reader` without materializing the full result. The reader uses bounded per-split queues, preserves input split order, and follows `parallelism` / `read.parallelism`. This benefits multi-split streaming consumers such as Ray, BLOB streaming, and batch-oriented training readers. ### Tests - `pypaimon/tests/reader_parallel_test.py` - `pypaimon/tests/ray_data_test.py` - `DedicatedFormatWriterTest#test_to_arrow_batch_reader` - `BlobParallelismTest#test_to_arrow_batch_reader_blob_parallelism` - `MultimodalTableTest#test_scan_stream_blobs` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
