Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19419 )
Change subject: IMPALA-11826: Avoid calling planFiles() on Iceberg V2 tables when there are no predicates ...................................................................... IMPALA-11826: Avoid calling planFiles() on Iceberg V2 tables when there are no predicates Similar to IMPALA-11591 but this Jira extends it to V2 tables. With this patch we group data files into two categories in IcebergContentFileStore: * data files without deletes * data files with deletes With this information we can avoid calling planFiles() when planning the scans of Iceberg tables. We can just set the lists of the file descriptors based on IcebergContentFileStore then invoke the regular planning methods. iceberg-v2-tables.test had to be updated a bit because now we are calculating the lengths of the file paths based on Impala's file descriptor objects + table location, and not based on data file information in Iceberg metadata (which has the file system prefix stripped) Testing: * executed existing tests * Updated plan tests Change-Id: Ia46bd2dce248a9e096fc1c0bd914fc3fa4686fb0 Reviewed-on: http://gerrit.cloudera.org:8080/19419 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M common/thrift/CatalogObjects.thrift M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java A fe/src/main/java/org/apache/impala/catalog/iceberg/GroupedContentFiles.java M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test 9 files changed, 277 insertions(+), 172 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/19419 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia46bd2dce248a9e096fc1c0bd914fc3fa4686fb0 Gerrit-Change-Number: 19419 Gerrit-PatchSet: 5 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Gergely Fürnstáhl <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tamas Mate <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
