Akash3121 commented on code in PR #11048:
URL: https://github.com/apache/paimon/pull/11048#discussion_r4515591837
##########
paimon-python/pypaimon/execution/python/write/commit/row_id_conflict.py:
##########
@@ -35,10 +35,9 @@ def check_row_id_conflicts(table, messages,
latest_snapshot_id, check_from_snaps
baseline = manager.get_snapshot_by_id(check_from_snapshot)
if latest is None or baseline is None:
raise ValueError('Row-ID conflict check snapshots must exist in the
current table')
- current_schema = table.schema_manager.latest()
- if (baseline.schema_id != latest.schema_id
- or current_schema is not None and current_schema.id !=
table.table_schema.id):
- raise ValueError('Row-ID conflict checks require an unchanged, current
table schema')
+ # Logical conflicts compare field IDs resolved in each file's historical
+ # schema, just like ordinary commits. Different schema versions alone do
Review Comment:
Please also validate staged update field identities when no physical rewrite
is required. If an `age` update is staged, then `age` is dropped and recreated
with a new field ID while the underlying file ranges remain unchanged,
`_rewrite_updates` returns at `if not candidates`
(`row_id_conflict_rewriter.py:232-233`) before `_with_latest_schema` or
`_partial_file_write_cols` can reject the dropped field. This change also
removes the ordinary commit’s previous unchanged-schema guard, so the commit
succeeds and reports `{'num_updated': 1`, but reading the table shows the
recreated `age` column is still null for every row - a silent lost update. I
reproduced this on both the exact PR head and a clean merge into current
master. The existing dropped-field test compacts first, so it only exercises
the candidate/rewrite path. Please resolve each staged partial file’s
historical write-field IDs against the latest schema before the no-candidates
return (without forcing a distributed rewr
ite), preserving valid add/rename/type-widen cases while rejecting missing
field IDs.
The key supporting control flow is:
```python
if not candidates:
return None
...
table = _with_latest_schema(table)
_validate_no_logical_conflict(...)
```
The base implementation rejected this race with Row-ID conflict checks
require an unchanged, current table schema ; the PR removes that guard but only
installs field-identity validation on actual rewrite candidates
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]