XiaoHongbo-Hope opened a new pull request, #1475:
URL: https://github.com/apache/paimon-rust/pull/1475

   ## Summary
   - Read ordered splits through a sliding window of individual split streams, 
so a newly available slot starts the next split while output order stays 
unchanged.
   - Keep at most the configured number of active split streams and one 
prefetched batch per active split.
   - Preserve cancellation on LIMIT, reader errors, and dropped output streams. 
Unordered execution retains its existing grouping behavior.
   
   ## Why
   The prior ordered path grouped contiguous splits per worker. A worker could 
not begin its next split until its earlier split finished, leaving the later 
splits unprefetched even when other workers had completed their work.
   
   ## Tests
   - `cargo test -p paimon --lib table::read_execution::tests --locked` (10 
passed)
   - `cargo test -p paimon --test parallel_table_read_test --locked` (5 passed)
   - `cargo test -p paimon --test data_evolution_compact_test --locked` (12 
passed)
   - `cargo fmt --all -- --check`
   - `git diff --check`
   
   The change was also compared against the prior ordered path using a filtered 
BLOB descriptor workload with 16, 100, and 1000 clips. The measurements used a 
separate runtime revision that includes an unmerged CRC compatibility change; 
this PR contains only the ordered split fix.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to