Gergely Fürnstáhl has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/19850 )
Change subject: WIP IMPALA-11619: Improve Iceberg V2 reads with a custom Iceberg Position Delete operator ...................................................................... WIP IMPALA-11619: Improve Iceberg V2 reads with a custom Iceberg Position Delete operator Implemented new exec node to optimize iceberg v2 read performance. IcebergDeleteNode and IcebergDeleteBuild classes are based on PartitionedHashJoin counterparts. The actual "join" part of the node is optimized, while others are kept very similarly, to be able to intergrate features of PartitionedHashJoin if needed (partitioning, spilling). Still TODO: - Clean/slice up the join part for easier digestion. - Double check code comments Change-Id: I024a61573c83bda5584f243c879d9ff39dd2dcfa --- M be/src/codegen/impala-ir.cc M be/src/exec/CMakeLists.txt M be/src/exec/data-sink.cc M be/src/exec/data-sink.h M be/src/exec/exec-node.cc M be/src/exec/hash-table.h A be/src/exec/iceberg-delete-builder.cc A be/src/exec/iceberg-delete-builder.h A be/src/exec/iceberg-delete-node.cc A be/src/exec/iceberg-delete-node.h A be/src/exec/iceberg-delete-node.inline.h M be/src/exec/join-builder.h M be/src/runtime/query-state.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/DataSinks.thrift M common/thrift/ImpalaService.thrift M common/thrift/PlanNodes.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/analysis/JoinOperator.java M fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java M fe/src/main/java/org/apache/impala/planner/HashJoinNode.java A fe/src/main/java/org/apache/impala/planner/IcebergDeleteNode.java M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java M fe/src/main/java/org/apache/impala/planner/JoinBuildSink.java M fe/src/main/java/org/apache/impala/planner/JoinNode.java M fe/src/main/java/org/apache/impala/planner/NestedLoopJoinNode.java M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M tests/query_test/test_iceberg.py 30 files changed, 2,052 insertions(+), 126 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/19850/9 -- To view, visit http://gerrit.cloudera.org:8080/19850 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I024a61573c83bda5584f243c879d9ff39dd2dcfa Gerrit-Change-Number: 19850 Gerrit-PatchSet: 9 Gerrit-Owner: Gergely Fürnstáhl <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
