Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11852 )
Change subject: IMPALA-3652: Fix resource transfer in subplans with limits ...................................................................... IMPALA-3652: Fix resource transfer in subplans with limits Impala assumes that when Reset() is called on an ExecNode, all of the memory returned from that node by GetNext() has been attached to the output RowBatch. In a query with a LIMIT on the subplan, such that some nodes don't reach 'eos', this may not be the case. The solution is to have Reset() take a RowBatch that any such memory can be attached to. I examined all ExecNodes for resources being transferred on 'eos' and added transferring of those resources in Resst(). Testing: - Added e2e tests that repro the issue for hash and nested loop joins. Change-Id: I3968a379fcbb5d30fcec304995d3e44933dbbc77 Reviewed-on: http://gerrit.cloudera.org:8080/11852 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/exec/aggregation-node-base.cc M be/src/exec/aggregation-node-base.h M be/src/exec/aggregator.h M be/src/exec/analytic-eval-node.cc M be/src/exec/analytic-eval-node.h M be/src/exec/blocking-join-node.cc M be/src/exec/blocking-join-node.h M be/src/exec/cardinality-check-node.cc M be/src/exec/cardinality-check-node.h M be/src/exec/data-source-scan-node.cc M be/src/exec/data-source-scan-node.h M be/src/exec/exchange-node.cc M be/src/exec/exchange-node.h M be/src/exec/exec-node.cc M be/src/exec/exec-node.h M be/src/exec/grouping-aggregator.cc M be/src/exec/grouping-aggregator.h M be/src/exec/hbase-scan-node.cc M be/src/exec/hbase-scan-node.h M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/nested-loop-join-node.cc M be/src/exec/nested-loop-join-node.h M be/src/exec/non-grouping-aggregator.h M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-builder.h M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/partitioned-hash-join-node.h M be/src/exec/select-node.cc M be/src/exec/select-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/exec/streaming-aggregation-node.cc M be/src/exec/streaming-aggregation-node.h M be/src/exec/subplan-node.cc M be/src/exec/subplan-node.h M be/src/exec/topn-node.cc M be/src/exec/topn-node.h M be/src/exec/union-node.cc M be/src/exec/union-node.h M be/src/exec/unnest-node.cc M be/src/exec/unnest-node.h A testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch-limit.test M tests/query_test/test_nested_types.py 46 files changed, 149 insertions(+), 86 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I3968a379fcbb5d30fcec304995d3e44933dbbc77 Gerrit-Change-Number: 11852 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Marshall <thomasmarsh...@cmu.edu> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Thomas Marshall <thomasmarsh...@cmu.edu> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
