Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/18383 )
Change subject: IMPALA-11023: Raise error when delete file is found in an Iceberg table ...................................................................... Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/18383/4/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java File fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java: http://gerrit.cloudera.org:8080/#/c/18383/4/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java@122 PS4, Line 122: dataFile > Instead of invoking planFiles() multiple times (once in hasDeleteFile(), an The DeleteFile objects are stored separately in the FileScanTask and are not added to the DataFile list currently. https://github.com/apache/iceberg/blob/62a53fed6fad24616aea7170d254e529602fabf1/api/src/main/java/org/apache/iceberg/FileScanTask.java#L41 https://github.com/apache/impala/blob/7b235eebd5dda9074e2b7724e6b290f49c1bb8ce/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L533 There is a shared ancestor called ContentFile, but I do not think that we should merge them, because the DeleteFiles are linked to the DataFiles. The other option I could think of was to return the FileScanTask objects in the getIcebergDataFiles(), but in that case the caller should handle the DataFile collection. -- To view, visit http://gerrit.cloudera.org:8080/18383 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e9cbf2424b27157883d551f73e728ab4ec6d21e Gerrit-Change-Number: 18383 Gerrit-PatchSet: 4 Gerrit-Owner: Tamas Mate <[email protected]> Gerrit-Reviewer: Gergely Fürnstáhl <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tamas Mate <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Fri, 08 Apr 2022 08:41:00 +0000 Gerrit-HasComments: Yes
