JingsongLi opened a new pull request, #8426: URL: https://github.com/apache/paimon/pull/8426
## Summary This PR adds DELETE support to PyPaimon data-evolution APIs and wires delete branches into table, multimodal, and Ray MERGE INTO flows. It also changes partition-only DELETE to emit file-level `deleted_files` commit messages, including matching index deletes, so the commit path uses overwrite-style conflict detection instead of a partition-filter shortcut. ## Changes - Add `delete_by_predicate` and `delete_by_row_id` support for data-evolution tables using deletion vectors. - Support `WhenMatched.delete()` alongside `WhenMatched.update()` in table MERGE INTO, multimodal MERGE, and Ray MERGE INTO. - Replace the internal partition-delete commit-message shortcut with `deleted_files` manifest deletes and matching `index_deletes`. - Trigger overwrite-style conflict detection for table DELETE entries and deletion-vector index changes, including stale partition detection for partition-only DELETE. - Split Ray merge update/delete branches so row-id actions are validated and committed separately. - Update PyPaimon docs for data evolution, multimodal APIs, and Ray data APIs. ## Testing - [x] `python -m pytest paimon-python/pypaimon/tests/table_update_test.py paimon-python/pypaimon/tests/data_evolution_deletion_vector_test.py paimon-python/pypaimon/tests/table_merge_into_test.py paimon-python/pypaimon/tests/multimodal_table_test.py paimon-python/pypaimon/tests/partition_predicate_test.py paimon-python/pypaimon/tests/overwrite_changes_cache_test.py -q` - [x] `python -m pytest paimon-python/pypaimon/tests/file_store_commit_test.py paimon-python/pypaimon/tests/table_commit_test.py paimon-python/pypaimon/tests/write/changelog_producer_test.py -q` - [x] `PYTHONPATH=paimon-python python -m pytest paimon-python/pypaimon/tests/write/conflict_detection_test.py -q` - [x] `python -m py_compile paimon-python/pypaimon/write/commit_message.py paimon-python/pypaimon/write/file_store_commit.py paimon-python/pypaimon/write/table_update.py paimon-python/pypaimon/write/commit/conflict_detection.py paimon-python/pypaimon/write/commit/overwrite_changes_provider.py paimon-python/pypaimon/ray/data_evolution_merge_into.py paimon-python/pypaimon/ray/data_evolution_merge_join.py paimon-python/pypaimon/ray/data_evolution_merge_transform.py` - [x] `git diff --check` ## Notes Ray e2e tests were not run locally because the environment does not have the `ray` Python package installed (`ModuleNotFoundError: No module named 'ray'`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
