XiaoHongbo-Hope opened a new pull request, #9339: URL: https://github.com/apache/paimon/pull/9339
## What changed - Recover Ray self-merge updates when concurrent compaction changes row-ID file boundaries after partial-column files are staged. - Read the staged update files with Ray, route them against the latest complete logical file groups, and retry the commit without rerunning the source scan or user callable. - Keep logical concurrent-update conflicts and unsupported cases fail-closed, and remove superseded staged files after a successful commit. ## Why A compaction between the self-merge read and commit currently raises a Row ID existence conflict and fails the whole job. The existing PyPaimon fallback runs on the driver and defaults to a 256 MiB affected-file limit. This adds the distributed Ray equivalent of Spark PR #9091 for self-merge updates. Recovery remains disabled for deletion-vector tables, schema changes, missing row-ID coverage, and existing-row BLOB or VECTOR staged files. Same-column concurrent updates are still rejected. ## Validation - Ray merge_into tests: 118 passed. - Added an end-to-end regression with two logical file groups, four VARIANT DOUBLE path updates, a topic_schema update, and compaction after staging. The callable result is applied exactly once and superseded files are cleaned. - Added a regression proving that a same-column concurrent update plus compaction still fails instead of being hidden. - flake8 and git diff --check passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
