QuakeWang opened a new pull request, #8071: URL: https://github.com/apache/paimon/pull/8071
### Purpose Ray writes create one independent Paimon writer per write task. For primary-key tables, this is unsafe unless all rows for each `(partition, bucket)` are routed to one writer. The previous Ray path only guarded HASH_FIXED primary-key tables in default/off mode. Default primary-key tables use dynamic buckets (`bucket = -1`), so they were written as-is. Multiple Ray tasks could then assign buckets and sequence numbers from independent local state, producing duplicate keys with overlapping `_SEQUENCE_NUMBER`. This PR makes Ray primary-key writes fail fast unless the table is HASH_FIXED and `hash_fixed_precluster="map_groups"` is selected. Append-only non-HASH_FIXED tables still pass through unchanged. The PyPaimon Ray docs are updated to match the new behavior. ### Tests - `python3 -m py_compile paimon-python/pypaimon/ray/shuffle.py paimon-python/pypaimon/ray/ray_paimon.py paimon-python/pypaimon/write/table_write.py paimon-python/pypaimon/tests/test_ray_shuffle_helper.py paimon-python/pypaimon/tests/ray_repartition_test.py` - `python3 -m pytest paimon-python/pypaimon/tests/test_ray_shuffle_helper.py -q` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
