JingsongLi opened a new pull request, #10419: URL: https://github.com/apache/paimon/pull/10419
### Purpose Use native prepared MERGE for the Paimon table source already accepted by PyPaimon's batch and stream update APIs. Native source scanning, key matching, action selection and file staging run in Rust core; Python transports the resolved Table and validates the existing clauses from schema metadata. Also fix Python source normalization overriding an explicitly selected historical snapshot with the latest snapshot. Source reads now use their effective branch and point-in-time options, as in Java. Depends on https://github.com/apache/paimon-rust/pull/1071. This PR is draft until that dependency merges. Native CI continues to install paimon-rust `main`; it is not pinned to this branch. - Enable full-snapshot Parquet append/primary-key sources without requiring source data evolution or row tracking. - Keep incremental sources and non-REST Blob views requiring catalog resolution on the existing Python path before native execution starts. Reuse the reader's Blob view capability check to prevent writing reference bytes as payload. - Validate source scan-mode options before native execution. - Add source-table regressions for batch/stream, selected snapshot, branch isolation, logical PK reads, nested ARRAY/MAP, NULL keys, unused Blob projection, Blob view payloads and invalid scan options. - No public PyPaimon API or dependency changes; no retry or prepared-file deletion on native errors. ### Tests - Rebuilt the local Rust extension with the dependent core implementation. - MERGE suites with native planning/reading/writing enabled: 57 passed, 6 subtests passed. - Related Native regression suites with planning, reading, writing, updating and committing enabled: 1,190 passed, 2 skipped, 27 subtests passed. Counters confirm row-ID, grouped, predicate, upsert, incremental and MERGE update paths all ran. - Changed Python files pass the CI flake8 configuration and Python 3.6 syntax parsing. - Two focused review rounds completed; confirmed Java/Paimon Spark source-read and MERGE semantics and repaired Blob view resolution before staging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
