geruh commented on code in PR #15006:
URL: https://github.com/apache/iceberg/pull/15006#discussion_r2679207571
##########
core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java:
##########
@@ -269,10 +278,28 @@ private void addInternal(DeleteFile file) {
DeleteFileSet deleteFiles =
newDeleteFilesBySpec.computeIfAbsent(spec.specId(), ignored ->
DeleteFileSet.create());
if (deleteFiles.add(file)) {
- addedFilesSummary.addedFile(spec, file);
hasNewDeleteFiles = true;
if (ContentFileUtil.isDV(file)) {
- newDVRefs.add(file.referencedDataFile());
+ if (!newDVRefs.add(file.referencedDataFile())) {
+ DeleteFileSet duplicateDVs =
+ duplicateDVsForDataFile.computeIfAbsent(
+ file.referencedDataFile(),
+ referencedFile -> {
+ // Find the first DV that was added with referenced file.
There must be one if
+ // we hit this case
+ return DeleteFileSet.of(
+ List.of(
+ deleteFiles.stream()
+ .filter(
Review Comment:
Should we add a safe guard to filter against the dv's here to avoid NPE
against`referencedDataFile` check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
