raunaqmorarka commented on code in PR #18074:
URL: https://github.com/apache/iceberg/pull/18074#discussion_r4478151640
##########
core/src/main/java/org/apache/iceberg/SnapshotProducer.java:
##########
@@ -610,13 +615,36 @@ private void notifyListeners() {
}
protected void cleanAll() {
- for (String manifestList : manifestLists) {
+ for (String manifestList : manifestsByManifestList.keySet()) {
deleteFile(manifestList);
}
- manifestLists.clear();
+ manifestsByManifestList.clear();
cleanUncommitted(EMPTY_SET);
}
+ /**
+ * Returns the manifest paths of the committed snapshot without reading its
manifest list when
Review Comment:
Done, replaced with inline comments.
##########
core/src/main/java/org/apache/iceberg/SnapshotProducer.java:
##########
@@ -610,13 +615,36 @@ private void notifyListeners() {
}
protected void cleanAll() {
- for (String manifestList : manifestLists) {
+ for (String manifestList : manifestsByManifestList.keySet()) {
deleteFile(manifestList);
}
- manifestLists.clear();
+ manifestsByManifestList.clear();
cleanUncommitted(EMPTY_SET);
}
+ /**
+ * Returns the manifest paths of the committed snapshot without reading its
manifest list when
+ * this producer wrote that manifest list.
+ */
+ private Set<String> committedManifestPaths(Snapshot saved) {
+ Set<String> paths = writtenManifestPaths(saved.manifestListLocation());
+ if (paths != null) {
Review Comment:
Done.
##########
core/src/test/java/org/apache/iceberg/TestSnapshotProducer.java:
##########
@@ -403,4 +408,74 @@ public void monotonicityIsScopedToTargetBranch() {
.isEqualTo(mainNewTs)
.isLessThan(branchTs);
}
+
+ @TestTemplate
+ public void commitDoesNotReadCommittedManifestList() {
Review Comment:
Added `fastForwardCherryPickReadsCommittedManifestList`, which stages a
delete because cherry-picking an append reads the manifest list itself before
commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
