Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/18531 )
Change subject: IMPALA-10453: Support file pruning via runtime filters on Iceberg ...................................................................... Patch Set 4: (2 comments) Thank you for the reviews Csaba/Zoltan. Updated the change, modified the filter logic a bit, it is now skipping the filters that would check slots that were not written during template tuple creation. http://gerrit.cloudera.org:8080/#/c/18531/4/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java File fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java: http://gerrit.cloudera.org:8080/#/c/18531/4/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java@268 PS4, Line 268: partField.getTransformType() == TIcebergPartitionTransformType.IDENTITY) > We cannot create template tuple from the other partition transforms. E.g. i Thank you for the additional explanation. Yeah, that makes sense, I was thinking about checking the result of the CreateTemplateTuple but keeping the logic in FE seemed more descriptive. At this point it makes more sense to handle in the BE side. http://gerrit.cloudera.org:8080/#/c/18531/5/tests/query_test/test_iceberg.py File tests/query_test/test_iceberg.py: http://gerrit.cloudera.org:8080/#/c/18531/5/tests/query_test/test_iceberg.py@617 PS5, Line 617: self.execute_query("SET RUNTIME_FILTER_MODE=OFF") > Why are we disabling it in this case? Would it make the tests meaningless? During an earlier verify job the runtime filter rejected a file in this test, this test checks the number of rowgroups read, which could become flaky without disabling it here. -- To view, visit http://gerrit.cloudera.org:8080/18531 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7762e1238bdf236b85d2728881a402a2bb41f36a Gerrit-Change-Number: 18531 Gerrit-PatchSet: 4 Gerrit-Owner: Tamas Mate <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Gergely Fürnstáhl <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tamas Mate <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Mon, 18 Jul 2022 16:00:39 +0000 Gerrit-HasComments: Yes
