junmuz commented on code in PR #11072:
URL: https://github.com/apache/paimon/pull/11072#discussion_r4540387001


##########
paimon-core/src/main/java/org/apache/paimon/iceberg/IcebergCommitCallback.java:
##########
@@ -1053,13 +1071,31 @@ private void createMetadataWithBase(
                         .filter(meta -> meta.content() == 
IcebergManifestFileMeta.Content.DELETES)
                         .collect(Collectors.toList());
 
-        Map<String, Pair<BinaryRow, DataFileMeta>> removedFiles = new 
LinkedHashMap<>();
+        // Manifest replacement can change the base files without recording a 
delta. Detect the
+        // empty-delta overwrite on retries as well, when the original 
callback context is gone.
+        boolean fullSnapshot =
+                fileChangesCollector == null
+                        || (snapshot.commitKind() == 
Snapshot.CommitKind.OVERWRITE
+                                && table.store()
+                                        .manifestListFactory()
+                                        .create()
+                                        .readDeltaManifests(snapshot)
+                                        .isEmpty());
+        List<DataSplit> snapshotSplits =
+                fullSnapshot
+                        ? 
table.newSnapshotReader().withSnapshot(snapshotId).read().dataSplits()
+                                .stream()
+                                .filter(DataSplit::rawConvertible)

Review Comment:
   Addressed. Repair reconciliation now reads live manifest entries and 
preserves previously published files by physical path, including files in 
splits requiring merging. Added a regression for the Lmax + L0 overlap case 
that verifies Iceberg retains all four previously visible rows. The test passes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to