tzphh opened a new pull request, #10987: URL: https://github.com/apache/paimon/pull/10987
### Purpose Ray 2.60.0 defaults to `shuffle_v2`. Its sorted reduce fails when real update blocks and the sentinel have different Arrow schemas, or when a map with no batches emits a zero-column, zero-row shard. Align routed update blocks with the sentinel schema before grouping. For the affected Ray 2.60.0 groupings, register a callback on a copy of the dataset context. It removes schema-less empty shards before the sorted reducer while keeping typed empty shards. Row-ID reads stay lazy and continue to use the configured shuffle strategy. ### Tests - Ray 2.60.0 with default `shuffle_v2` passed all six previously failing tests and 24 related tests covering lazy reads, grouped writes, and routing. - Ray 2.60.0 with `hash_shuffle` passed the same six tests. - Repository Flake8 and `git diff --check` passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
