YannByron commented on code in PR #9484:
URL: https://github.com/apache/paimon/pull/9484#discussion_r3892272148
##########
paimon-python/pypaimon/write/table_update.py:
##########
@@ -180,6 +181,15 @@ def _update_by_arrow_batches_with_row_id(
updater = TableUpdateByRowId(
self.table, self.commit_user, commit_identifier)
updater.update_columns(table, cols)
+ overlapping_first_row_ids = updated_first_row_ids.intersection(
Review Comment:
Good point. Fixed in 4944d1d80: `TableUpdateByRowId.update_columns` now
calculates the current `first_row_id` set and checks it against prior updates
before `_write_by_first_row_id`, so the conflicting batch is never staged. The
outer batch operation still aborts files staged by earlier batches. I also
added a unit test asserting the write method is called only for the first batch.
##########
paimon-python/pypaimon/write/table_update.py:
##########
@@ -180,6 +181,15 @@ def _update_by_arrow_batches_with_row_id(
updater = TableUpdateByRowId(
self.table, self.commit_user, commit_identifier)
updater.update_columns(table, cols)
+ overlapping_first_row_ids = updated_first_row_ids.intersection(
Review Comment:
Agreed. Fixed in 4944d1d80 by tracking `first_row_id` groups per updated
column. Batches that target the same file group with disjoint columns are now
allowed; only overlap on the same column is rejected. Added an end-to-end test
updating `age` and `city` in separate batches within the same file group.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
