Balazs Hevele has uploaded this change for review. ( http://gerrit.cloudera.org:8080/25017
Change subject: PROTOTYPE: lazy join materialization ...................................................................... PROTOTYPE: lazy join materialization Extend late materialization to broadcast join's probe side: -from scan node, directly call into the join node, to directly use the hash table for filtering -removed runtime filters in this case This doesn't seem to help much performance-wise compared to runtime-filters, even when they don't work that well, except for: -when whole pages get skipped, saving IO and decompression -the skipped columns are expensive to decode -or almost all rows are discarded, in long runs Assisted-by: Claude Opus 5.5 Change-Id: Id404477010f24003ec46c4471423e3a5859a89f1 --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/hash-table.h M be/src/exec/hash-table.inline.h M be/src/exec/hdfs-columnar-scanner-ir.cc M be/src/exec/hdfs-columnar-scanner.cc M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scanner.cc M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-builder.h M be/src/exec/partitioned-hash-join-node-ir.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/partitioned-hash-join-node.h M be/src/exec/scan-node.cc M be/src/exec/scan-node.h M be/src/runtime/fragment-instance-state.cc M be/src/runtime/fragment-instance-state.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/PlanNodes.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java 25 files changed, 419 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/25017/1 -- To view, visit http://gerrit.cloudera.org:8080/25017 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id404477010f24003ec46c4471423e3a5859a89f1 Gerrit-Change-Number: 25017 Gerrit-PatchSet: 1 Gerrit-Owner: Balazs Hevele <[email protected]>
