rambleraptor commented on code in PR #3760:
URL: https://github.com/apache/iceberg-python/pull/3760#discussion_r4544282178
##########
pyiceberg/table/update/snapshot.py:
##########
@@ -1261,9 +1265,18 @@ def _commit(self) -> UpdatesAndRequirements:
Tuple of updates and requirements to be committed,
as required by the calling parent apply functions.
"""
- # Remove any protected snapshot IDs from the set to expire, just in
case
+ # Resolved here rather than in older_than() so that snapshots released
by
+ # remove_expired_refs() are expirable no matter which order the two
were called in.
protected_ids = self._get_protected_snapshot_ids()
+ if self._expire_older_than_ms is not None:
+ for snapshot in self._transaction.table_metadata.snapshots:
+ if snapshot.timestamp_ms < self._expire_older_than_ms and
snapshot.snapshot_id not in protected_ids:
+ self._snapshot_ids_to_expire.add(snapshot.snapshot_id)
+
+ # Remove any protected snapshot IDs from the set to expire, just in
case
self._snapshot_ids_to_expire -= protected_ids
+ for ref_name in self._ref_names_to_remove:
+ self._updates += (RemoveSnapshotRefUpdate(ref_name=ref_name),)
Review Comment:
I think we need to also add a `AssertRefSnapshotId` requirement. This helps
ensure that if a ref is expired, it's never checked against what's in the
catalog at commit time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
