Gergely Fürnstáhl has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19850
Change subject: WIP IMPALA-11619: Improve Iceberg V2 reads with a custom Iceberg Position Delete operator ...................................................................... WIP IMPALA-11619: Improve Iceberg V2 reads with a custom Iceberg Position Delete operator Implemented new exec node to optimize iceberg v2 read performance. IcebergDeleteNode and IcebergDeleteBuild classes are based on PartitionedHashJoin counterparts. The actual "join" part of the node is optimized, while others are kept very similarly, to be able to intergrate features of PartitionedHashJoin if needed (partitioning, spilling). Still TODO: - Add fe/thrift changes to the patchset (now its the original version copied from PHJ) - Clean/slice up the join part for easier digestion. - Double check code comments Change-Id: I024a61573c83bda5584f243c879d9ff39dd2dcfa --- M be/src/codegen/impala-ir.cc M be/src/exec/CMakeLists.txt M be/src/exec/data-sink.cc M be/src/exec/data-sink.h M be/src/exec/exec-node.cc M be/src/exec/hash-table.h A be/src/exec/iceberg-delete-builder.cc A be/src/exec/iceberg-delete-builder.h A be/src/exec/iceberg-delete-node.cc A be/src/exec/iceberg-delete-node.h A be/src/exec/iceberg-delete-node.inline.h M be/src/exec/join-builder.h M be/src/runtime/query-state.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/DataSinks.thrift M common/thrift/ImpalaService.thrift M common/thrift/PlanNodes.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java A fe/src/main/java/org/apache/impala/planner/IcebergDeleteNode.java M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java M fe/src/main/java/org/apache/impala/planner/JoinBuildSink.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M tests/query_test/test_iceberg.py 27 files changed, 2,140 insertions(+), 114 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/19850/4 -- To view, visit http://gerrit.cloudera.org:8080/19850 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I024a61573c83bda5584f243c879d9ff39dd2dcfa Gerrit-Change-Number: 19850 Gerrit-PatchSet: 4 Gerrit-Owner: Gergely Fürnstáhl <[email protected]>
