JingsongLi commented on code in PR #7873:
URL: https://github.com/apache/paimon/pull/7873#discussion_r3485047083
##########
paimon-python/pypaimon/write/file_store_commit.py:
##########
@@ -189,6 +191,118 @@ def commit(self, commit_messages: List[CommitMessage],
commit_identifier: int):
allow_rollback=allow_rollback,
index_deletes=index_deletes)
+ def commit_compact(self, commit_messages: List[CommitMessage],
commit_identifier: int):
+ """Commit compaction-only messages.
+
+ Each message must carry no data increment; compact_before → DELETE
+ entries, compact_after → ADD entries; snapshot kind = COMPACT. Skips
Review Comment:
This says COMPACT skips row-id assignment, but `commit_compact()` still
calls `_try_commit()` with `commit_kind="COMPACT"`, and `_try_commit()` applies
`_assign_snapshot_id()` / `_assign_row_tracking_meta()` whenever row tracking
is enabled. That means a `compact_after` file with no `first_row_id` is treated
like a new append file, gets a fresh row-id range from
`latest_snapshot.next_row_id`, and the COMPACT snapshot advances `next_row_id`.
Compaction should preserve the logical row ids of the compacted input files,
otherwise row-id based updates/conflict checks can no longer locate those rows
after Python commits the compaction. Please gate the row-tracking assignment to
write/overwrite commits (or explicitly skip it for COMPACT) and add a
row-tracking compact test that verifies `compact_after.first_row_id` and
`next_row_id` are not rewritten.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
