JingsongLi opened a new pull request, #11287: URL: https://github.com/apache/paimon/pull/11287
### Purpose A Spark Data Evolution centroid build can train on snapshot 1 and publish after a concurrent `CALL sys.rewrite_file_index`. The file-index rewrite emits a DELETE and an ADD for the same physical data file. The source check introduced in #11249 interprets both entries as vector data changes and rejects the build. Match replacement pairs within each COMPACT delta by partition, bucket and file name. Skip a unique pair only when the total bucket count matches and the full data metadata compares equal after removing embedded and external file indexes. Continue checking the other entries in the same snapshot. ### Tests - Added a commit regression covering an embedded-to-external file-index rewrite, row-ID reassignment, and mixed compaction entries. - Ran `ConflictDetectionTest` and `FileStoreCommitTest` on Java 8, plus Spotless and Checkstyle. - Replayed a Spark SQL race with the centroid writer paused before publication. Rewriting the scalar Bloom filter kept the original data path and row values. The same build that previously failed now published successfully. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
