JingsongLi opened a new pull request, #10428: URL: https://github.com/apache/paimon/pull/10428
### Purpose Run the numeric timestamp-window table sources already supported by PyPaimon through native prepared MERGE. Snapshot/window resolution, source projection and reading, key matching and target staging belong to Rust core. Python passes the resolved table and existing clauses. Depends on https://github.com/apache/paimon-rust/pull/1074. This PR is a draft until that core capability reaches main. Native CI continues to install paimon-rust main; no branch pin, compatibility probe or runtime retry is added. ### Changes - Allow `incremental-between-timestamp` sources in the batch/stream native MERGE selector. Keep existing unsupported source route and partition-type gates. - Extract scan-mode validation so native MERGE can validate metadata without constructing a Python scanner or resolving source snapshots in Python. - Align existing Python numeric-window behavior with Java: equal bounds and tables without snapshots return empty; reversed bounds fail when snapshots exist; numeric milliseconds use Java Long syntax/range rather than Python-only whitespace, underscores or unbounded integers. - Align AUTO with Java: read physical changelog manifests when a producer is configured, and APPEND deltas otherwise. OVERWRITE remains excluded and the selected ending snapshot supplies plan/split metadata. - Preserve native error propagation. Failed source scans do not retry in Python or stage target files. ### Tests - Add 126 committed MERGE cases for append/PK/data-evolution sources, numeric window boundaries, batch/stream, Python/native controls, selected branch, schema rename/addition with retained field IDs, AUTO changelog, invalid values and missing source history. - Native cases prohibit Python source snapshot resolution, materialization and matching. Source-failure cases assert target files, rows and snapshot identity remain unchanged. - Add focused scanner regressions for empty/equal/reversed windows and physical changelog/end-snapshot identity; update the old equal-window rejection expectation to Java behavior. - Rebuilt the local extension from paimon-rust #1074. Related suites with all Native switches enabled: 1,323 passed, 5 skipped, 85 subtests passed. Native counters confirm row-ID, grouped, predicate, upsert, incremental and MERGE update paths all ran. - Follow-up snapshot-identity assertion: 2 passed with native planning/reading enabled. - Changed files pass CI flake8, Python 3.6 syntax parsing and git diff checks. - Two independent verification rounds completed; no remaining actionable findings. Scope remains existing numeric PyPaimon timestamp windows. No textual timestamp or snapshot/tag incremental API is introduced. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
