JingsongLi opened a new pull request, #8416: URL: https://github.com/apache/paimon/pull/8416
## Summary Adds single-process PyPaimon `TableUpdate.merge_into` support for data-evolution tables. The implementation follows the existing Ray merge clause helpers, supports batch and stream commit lifecycles, and documents the new API. ## Changes - Expose `merge_into` on `BatchTableUpdate` and `StreamTableUpdate`. - Add a local PyArrow-based merge implementation for PyPaimon table, Arrow table, and pandas DataFrame sources. - Support matched updates, not-matched inserts, renamed `ON` keys, conditions, and self-merge by `_ROW_ID`. - Treat target self-assignments like `t.col = t.col` as no-op updates in the shared merge normalization path. - Add merge tests covering inserts, updates, row-id stability, renamed keys, partition-column rejection, duplicate matches, self-merge, and stream commit identifiers. - Update the PyPaimon data-evolution docs and keep stream-mode guidance in a single section. ## Testing - [x] `python -m unittest pypaimon.tests.table_merge_into_test` - [x] `python -m unittest pypaimon.tests.table_update_test.TableUpdateBatchTest.test_update_existing_column` - [x] `python -m unittest pypaimon.tests.table_upsert_by_key_test.TableUpsertByKeyBatchTest.test_mixed_update_and_append` - [x] `python -m py_compile pypaimon/table/data_evolution_merge_into.py pypaimon/ray/data_evolution_merge_into.py pypaimon/write/table_update.py pypaimon/tests/table_merge_into_test.py` - [x] `git diff --check` ## Notes - One condition-clause test is skipped locally when `datafusion` is not installed. - Ray merge tests were not run locally because the current Python environment does not have `ray` installed; the shared helper behavior was covered by the PyPaimon merge tests and a direct normalize smoke check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
