aokolnychyi commented on a change in pull request #2865:
URL: https://github.com/apache/iceberg/pull/2865#discussion_r677033560
##########
File path: core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java
##########
@@ -297,6 +283,39 @@ protected void validateAddedDataFiles(TableMetadata base,
Long startingSnapshotI
}
}
+ /**
+ * Validates that no new delete files that must be applied to the given data
files have been added to the table since
+ * a starting snapshot.
+ *
+ * @param base table metadata to validate
+ * @param startingSnapshotId id of the snapshot current at the start of the
operation
+ * @param dataFiles data files to validate have no new row deletes
+ */
+ protected void validateNoNewDeletesForDataFiles(TableMetadata base, Long
startingSnapshotId,
+ Iterable<DataFile>
dataFiles) {
+ // if there is no current table state, no files have been added
+ if (base.currentSnapshot() == null) {
+ return;
+ }
+
+ Pair<List<ManifestFile>, Set<Long>> history =
+ validationHistory(base, startingSnapshotId,
VALIDATE_REPLACED_DATA_FILES_OPERATIONS, ManifestContent.DELETES);
+ List<ManifestFile> deleteManifests = history.first();
+
+ long startingSequenceNumber = startingSnapshotId == null ? 0 :
base.snapshot(startingSnapshotId).sequenceNumber();
+ DeleteFileIndex deletes = DeleteFileIndex.builderFor(ops.io(),
deleteManifests)
+ .afterSequenceNumber(startingSequenceNumber)
+ .specsById(ops.current().specsById())
+ .build();
+
+ for (DataFile dataFile : dataFiles) {
Review comment:
Oh, that would be way more important than skipping this loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
