JingsongLi opened a new pull request, #10507: URL: https://github.com/apache/paimon/pull/10507
## Purpose Use native execution for PyPaimon's existing `BatchTableUpdate` and `StreamTableUpdate.delete_by_predicate`. Previously Python still planned partition metadata deletion or collected all matched row IDs before invoking the native row-ID writer. Depends on apache/paimon-rust#1105. Keep this draft until that PR reaches Rust `main`; Native CI continues to build upstream `main`. ## Changes - Delegate the complete predicate-delete operation to Rust core, converting only predicates and commit messages in Python. - Keep native preparation separate from execution: execution, snapshot-load and message-conversion failures cannot fall back into a second Python operation. - Share common native update eligibility. Partition-only deletes also use Rust for append/PK tables without row tracking or deletion vectors, including pending bucket -2 files. - Align Python commit conflict checks with Java: remove the extra partition-level rejection based solely on a message snapshot baseline. Prepared deletes preserve subsequent appends with fresh row IDs; existing file/DV/row-ID checks still reject conflicting replacements and updates. - Add REST tests for batch/stream operation ownership, partition/data/mixed/NULL predicates, column deltas, prior DV contents, scalar indexes, unindexed rows, pending files, races and snapshot-load errors. ## Java semantics `FileStoreCommitImpl.configureRowIdCheckFromMessages` configures row-ID checking; it does not enable `StrictModeChecker`. The stricter checker is constructed only when an explicit last-safe snapshot is configured. `DataEvolutionConflictDetection` ignores fresh row IDs beyond the baseline and checks overlapping historical row/column writes. Python now follows this default behavior. ## Verification Built the paired Rust extension locally and enabled Native planning, reads, writes and commits: - 756 related Native regression tests passed; 2 additional REST snapshot tests passed. - Final predicate-delete suite: 30 passed, including after rebasing onto current master. - Existing Python update suite: 130 passed plus 20 subtests. - Existing commit/conflict suites: 93 passed plus 13 subtests. - Changed-file flake8 and Python 3.6 syntax checks passed. - Independent core and facade/Java-semantics review found no blocking issues; extra batch/stream checks confirmed concurrent existing-row updates still conflict. Prepared files remain preserved on commit failures or unknown outcomes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
