tzphh opened a new pull request, #10987:
URL: https://github.com/apache/paimon/pull/10987

   ### Purpose
   
   Ray 2.60.0 defaults to `shuffle_v2`. Its sorted reduce fails when real 
update blocks and the sentinel have different Arrow schemas, or when a map with 
no batches emits a zero-column, zero-row shard. Align routed update blocks with 
the sentinel schema before grouping.
   
   For the affected Ray 2.60.0 groupings, register a callback on a copy of the 
dataset context. It removes schema-less empty shards before the sorted reducer 
while keeping typed empty shards. Row-ID reads stay lazy and continue to use 
the configured shuffle strategy.
   
   ### Tests
   
   - Ray 2.60.0 with default `shuffle_v2` passed all six previously failing 
tests and 24 related tests covering lazy reads, grouped writes, and routing.
   - Ray 2.60.0 with `hash_shuffle` passed the same six tests.
   - Repository Flake8 and `git diff --check` passed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to