Fokko commented on code in PR #4273:
URL: https://github.com/apache/iceberg-python/pull/4273#discussion_r4561321536


##########
pyiceberg/table/update/__init__.py:
##########
@@ -746,11 +746,11 @@ def update_table_metadata(
     for update in updates:
         new_metadata = _apply_table_update(update, new_metadata, context)
 
-    # Update last_updated_ms if it was not updated by update operations
+    # last_updated_ms must not precede the base's, which becomes the latest 
metadata-log entry
     if context.has_changes():
         if metadata_location:
             new_metadata = _update_table_metadata_log(new_metadata, 
metadata_location, base_metadata.last_updated_ms)
-        if base_metadata.last_updated_ms == new_metadata.last_updated_ms:
+        if new_metadata.last_updated_ms <= base_metadata.last_updated_ms:

Review Comment:
   I compared this against Java's TableMetadata.Builder 
(core/src/main/java/org/apache/iceberg/TableMetadata.java). Java never inherits 
the snapshot timestamp into last-updated-ms at all: addSnapshot() does not 
touch it, and build() stamps System.currentTimeMillis() (lines 1559-1561). 
PyIceberg inherits the snapshot timestamp in AddSnapshotUpdate because 
snapshots are staged eagerly (in Java the snapshot is produced at commit time), 
which is the root cause of #4189. This change restores the invariant Java 
enforces at lines 394-403 (last-updated-ms within one minute after the latest 
metadata-log entry), so from Java's perspective the output is accepted again.



##########
tests/table/test_init.py:
##########
@@ -1683,6 +1683,23 @@ def test_update_metadata_log(table_v2: Table) -> None:
     assert len(new_metadata.metadata_log) == 2
 
 
+def test_update_metadata_last_updated_ms_not_before_metadata_log(table_v2: 
Table) -> None:
+    stale_snapshot = Snapshot(
+        snapshot_id=25,
+        parent_snapshot_id=19,
+        sequence_number=200,
+        timestamp_ms=table_v2.metadata.last_updated_ms - 61_000,
+        manifest_list="s3:/a/b/c.avro",
+        summary=Summary(Operation.APPEND),
+        schema_id=3,
+    )
+
+    new_metadata = update_table_metadata(
+        table_v2.metadata, (AddSnapshotUpdate(snapshot=stale_snapshot),), 
False, table_v2.metadata_location
+    )
+    assert new_metadata.last_updated_ms > 
new_metadata.metadata_log[-1].timestamp_ms

Review Comment:
   Nit: nice regression test. The issue's repro also covers the 
SetSnapshotRefUpdate path (the snapshot log gets stamped with the stale 
snapshot timestamp). I probed that separately: Java tolerates it (snapshot-log 
entries are validated with the same one-minute tolerance, and the stamping 
actually matches Java's timeOfChange behavior for added snapshots), so this is 
fine as is. Just mentioning in case you want a variant of this test with a 
SetSnapshotRefUpdate to main for completeness.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to