XiaoHongbo-Hope opened a new pull request, #1475: URL: https://github.com/apache/paimon-rust/pull/1475
## Summary - Read ordered splits through a sliding window of individual split streams, so a newly available slot starts the next split while output order stays unchanged. - Keep at most the configured number of active split streams and one prefetched batch per active split. - Preserve cancellation on LIMIT, reader errors, and dropped output streams. Unordered execution retains its existing grouping behavior. ## Why The prior ordered path grouped contiguous splits per worker. A worker could not begin its next split until its earlier split finished, leaving the later splits unprefetched even when other workers had completed their work. ## Tests - `cargo test -p paimon --lib table::read_execution::tests --locked` (10 passed) - `cargo test -p paimon --test parallel_table_read_test --locked` (5 passed) - `cargo test -p paimon --test data_evolution_compact_test --locked` (12 passed) - `cargo fmt --all -- --check` - `git diff --check` The change was also compared against the prior ordered path using a filtered BLOB descriptor workload with 16, 100, and 1000 clips. The measurements used a separate runtime revision that includes an unmerged CRC compatibility change; this PR contains only the ordered split fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
