JingsongLi commented on code in PR #11072:
URL: https://github.com/apache/paimon/pull/11072#discussion_r4546096675


##########
paimon-core/src/main/java/org/apache/paimon/iceberg/IcebergCommitCallback.java:
##########
@@ -364,12 +373,12 @@ private void createMetadata(
                                         String.valueOf(snapshotId));
                     }
                     if (metadataCommitter != null) {
-                        // recommit the pointer (previous version as base, so 
a lagging
-                        // catalog advances) before retiring files it may 
still reference
+                        // Recommit the pointer using the same surviving base 
as the first
+                        // attempt, before retiring files a lagging catalog 
may still reference.
                         Path existingPath = 
pathFactory.toMetadataPath(snapshotId);
-                        Path basePath = pathFactory.toMetadataPath(snapshotId 
- 1);
+                        Path basePath = findBaseMetadataPath(snapshot);

Review Comment:
   [P2] Reuse the persisted metadata's parent when retrying publication
   
   This recomputes the base from whichever metadata files exist now, although 
the existing repair metadata already fixes its parent. I reproduced the 
following sequence without concurrent threads:
   
   1. Publish Iceberg snapshot 2, then fail the metadata write for Paimon 
snapshot 3.
   2. Remove an older manifest while keeping snapshot 3's delta manifest intact.
   3. Run the repair: it writes `v4.metadata.json` with parent 2, but catalog 
publication fails.
   4. Retry the original snapshot 3 callback. It successfully writes 
`v3.metadata.json`; the head guard correctly leaves the catalog at snapshot 2.
   5. Retry `remove_unexisting_manifests`. This branch now passes base 3 
alongside the existing snapshot 4 whose parent is still 2.
   
   The added regression fails with expected base `v2.metadata.json`, actual 
base `v3.metadata.json`. REST `checkBase` consequently rejects the valid 
incremental update and enters recreation/registration. For v3 metadata with a 
nonzero first-row-id, a catalog without `registerTable` cannot finish recovery; 
with registration available, this unnecessarily replaces the catalog table.
   
   Please read the existing metadata first and resolve the base from its 
recorded parent, rather than selecting the newest predecessor again, and cover 
a predecessor callback retry between the failed repair publication and the 
procedure retry.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to