JingsongLi commented on code in PR #11072:
URL: https://github.com/apache/paimon/pull/11072#discussion_r4546096675
##########
paimon-core/src/main/java/org/apache/paimon/iceberg/IcebergCommitCallback.java:
##########
@@ -364,12 +373,12 @@ private void createMetadata(
String.valueOf(snapshotId));
}
if (metadataCommitter != null) {
- // recommit the pointer (previous version as base, so
a lagging
- // catalog advances) before retiring files it may
still reference
+ // Recommit the pointer using the same surviving base
as the first
+ // attempt, before retiring files a lagging catalog
may still reference.
Path existingPath =
pathFactory.toMetadataPath(snapshotId);
- Path basePath = pathFactory.toMetadataPath(snapshotId
- 1);
+ Path basePath = findBaseMetadataPath(snapshot);
Review Comment:
[P2] Reuse the persisted metadata's parent when retrying publication
This recomputes the base from whichever metadata files exist now, although
the existing repair metadata already fixes its parent. I reproduced the
following sequence without concurrent threads:
1. Publish Iceberg snapshot 2, then fail the metadata write for Paimon
snapshot 3.
2. Remove an older manifest while keeping snapshot 3's delta manifest intact.
3. Run the repair: it writes `v4.metadata.json` with parent 2, but catalog
publication fails.
4. Retry the original snapshot 3 callback. It successfully writes
`v3.metadata.json`; the head guard correctly leaves the catalog at snapshot 2.
5. Retry `remove_unexisting_manifests`. This branch now passes base 3
alongside the existing snapshot 4 whose parent is still 2.
The added regression fails with expected base `v2.metadata.json`, actual
base `v3.metadata.json`. REST `checkBase` consequently rejects the valid
incremental update and enters recreation/registration. For v3 metadata with a
nonzero first-row-id, a catalog without `registerTable` cannot finish recovery;
with registration available, this unnecessarily replaces the catalog table.
Please read the existing metadata first and resolve the base from its
recorded parent, rather than selecting the newest predecessor again, and cover
a predecessor callback retry between the failed repair publication and the
procedure retry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]