aokolnychyi commented on a change in pull request #2865:
URL: https://github.com/apache/iceberg/pull/2865#discussion_r677010865
##########
File path: core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java
##########
@@ -297,6 +283,39 @@ protected void validateAddedDataFiles(TableMetadata base,
Long startingSnapshotI
}
}
+ /**
+ * Validates that no new delete files that must be applied to the given data
files have been added to the table since
+ * a starting snapshot.
+ *
+ * @param base table metadata to validate
+ * @param startingSnapshotId id of the snapshot current at the start of the
operation
+ * @param dataFiles data files to validate have no new row deletes
+ */
+ protected void validateNoNewDeletesForDataFiles(TableMetadata base, Long
startingSnapshotId,
+ Iterable<DataFile>
dataFiles) {
+ // if there is no current table state, no files have been added
+ if (base.currentSnapshot() == null) {
+ return;
+ }
+
+ Pair<List<ManifestFile>, Set<Long>> history =
+ validationHistory(base, startingSnapshotId,
VALIDATE_REPLACED_DATA_FILES_OPERATIONS, ManifestContent.DELETES);
+ List<ManifestFile> deleteManifests = history.first();
+
+ long startingSequenceNumber = startingSnapshotId == null ? 0 :
base.snapshot(startingSnapshotId).sequenceNumber();
+ DeleteFileIndex deletes = DeleteFileIndex.builderFor(ops.io(),
deleteManifests)
+ .afterSequenceNumber(startingSequenceNumber)
+ .specsById(ops.current().specsById())
+ .build();
+
+ for (DataFile dataFile : dataFiles) {
Review comment:
nit: Do we have to iterate through data files if the index is empty and
we did not find any delete manifests? This logic will be triggered on top of
various operations including copy-on-write MERGE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
