Henry Robinson has uploaded a new patch set (#2). Change subject: IMPALA-3077: Enable runtime filters when PHJ spills ......................................................................
IMPALA-3077: Enable runtime filters when PHJ spills This patch changes when runtime filters are produced in the partitioned hash-join node to allow filters to be produced even when the PHJ spills. Filters are now produced during the level0 processing of the PHJ's build-side input in ProcessBuildBatchWithFilters(). Since this function is codegen'ed, so now is filter production. We use constant-propagation via constant argument injection to disable filter production at no cost when it is not needed (including in level1+ repartitioning). I inspected the IR to confirm that the constant propagation works as expected. This change also allows us to send filters earlier during build-side processing. This patch also fixes a bug where re-partitioning beyond level0 would not use the codegen'ed implementation of ProcessBuildBatch(). A new test is added to test_runtime_row_filters, for Parquet only, which spills and confirms that filtering still occurs. Change-Id: I59a2d9ee03ccea6b674392584e4c7f272233571e --- M be/src/exec/partitioned-hash-join-node-ir.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/partitioned-hash-join-node.h M testdata/workloads/functional-query/queries/QueryTest/runtime_row_filters.test 4 files changed, 75 insertions(+), 61 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/83/2783/2 -- To view, visit http://gerrit.cloudera.org:8080/2783 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I59a2d9ee03ccea6b674392584e4c7f272233571e Gerrit-PatchSet: 2 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Henry Robinson <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
