Akash3121 commented on code in PR #11144:
URL: https://github.com/apache/paimon/pull/11144#discussion_r4570267161
##########
paimon-python/pypaimon/write/backend_dispatch.py:
##########
@@ -459,7 +459,13 @@ def commit_table_messages(
"Committing table %s, %d non-empty messages",
owner.table.identifier, len(non_empty_messages)
)
- prepared = owner._prepare_native_commit(non_empty_messages)
+ # Native 0.5.0 does not validate retained row-ID field identities against
+ # the schema used by each publication attempt. Python performs that check
Review Comment:
`commit_table_messages` inspects `message.new_files` here before calling
`owner._prepare_native_commit(non_empty_messages)` (and even when
`commit.native.enabled` is `false`).
This is the direct cause of the 5 failing tests in both `Python / Native
(3.10–3.13)` and `Python / Python Execution`:
-
`native_write_fork_safety_test.py::test_native_preparation_propagates_fork_errors_without_python_mutations[direct-commit]`
-
`native_write_fork_safety_test.py::test_native_preparation_propagates_fork_errors_without_python_mutations[cause-commit]`
-
`native_write_fork_safety_test.py::test_native_preparation_propagates_fork_errors_without_python_mutations[context-commit]`
-
`native_write_fork_safety_test.py::test_ordinary_native_preparation_failures_keep_the_safe_python_fallback[ordinary-error-commit]`
-
`native_write_fork_safety_test.py::test_ordinary_native_preparation_failures_keep_the_safe_python_fallback[missing-runtime-commit]`
In `native_write_fork_safety_test.py` (lines 60 - 68), `operation ==
'commit'` creates `message = Mock()` with `message.is_empty.return_value =
False` on a `data-evolution` table and patches `native_messages_supported`.
Iterating `for file in message.new_files` here raises `TypeError: 'Mock' object
is not iterable` before `_prepare_native_commit` is ever reached.
In addition, retained row-ID column identity validation only applies to
partial-column updates (`file.write_cols is not None`), not every file with
`first_row_id is not None`. Moving this check into `native_messages_supported`
(or checking `file.first_row_id is not None and file.write_cols is not None`
inside `native_messages_supported` / `prepare_native_commit`) keeps commit
message validation in one place and fixes all 5 CI jobs:
```suggestion
prepared = owner._prepare_native_commit(non_empty_messages)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]