IMPALA-5419: Check for cancellation when building hash tables When inserting build rows into the hash table for each partition in PhjBuilder::Partition::BuildHashTable(), we didn't check for cancellation. This may lead to orphaned fragments running for excessive amount of time and consuming a lot of memory after the query has been cancelled.
This change fixes the problem by checking for cancellation per row batch in the loop inside PhjBuilder::Partition::BuildHashTable(). Manually verified with the following query that cancelling the query right after all build rows have been returned by the scan node will not cause memory usage to continue going up. select /* +straight_join */ count(*) from tpch20_parquet.lineitem t1 join /* +broadcast */ tpch20_parquet.lineitem t2 on t1.l_orderkey = t2.l_orderkey Change-Id: I8047f532f55dc0118f7a843c91275f752c8a190d Reviewed-on: http://gerrit.cloudera.org:8080/7047 Reviewed-by: Sailesh Mukil <[email protected]> Reviewed-by: Dan Hecht <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/cde19ab8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/cde19ab8 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/cde19ab8 Branch: refs/heads/master Commit: cde19ab8c7801436070ce0438e28d5042265dfd1 Parents: f3fdd4d Author: Michael Ho <[email protected]> Authored: Thu Jun 1 15:49:45 2017 -0700 Committer: Impala Public Jenkins <[email protected]> Committed: Fri Jun 2 10:03:11 2017 +0000 ---------------------------------------------------------------------- be/src/exec/partitioned-hash-join-builder.cc | 1 + 1 file changed, 1 insertion(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/cde19ab8/be/src/exec/partitioned-hash-join-builder.cc ---------------------------------------------------------------------- diff --git a/be/src/exec/partitioned-hash-join-builder.cc b/be/src/exec/partitioned-hash-join-builder.cc index dcff33d..57a690c 100644 --- a/be/src/exec/partitioned-hash-join-builder.cc +++ b/be/src/exec/partitioned-hash-join-builder.cc @@ -686,6 +686,7 @@ Status PhjBuilder::Partition::BuildHashTable(bool* built) { } else { if (UNLIKELY(!InsertBatch(prefetch_mode, ctx, &batch, indices))) goto not_built; } + RETURN_IF_CANCELLED(state); RETURN_IF_ERROR(state->GetQueryStatus()); // Free any local allocations made while inserting. ExprContext::FreeLocalAllocations(parent_->expr_ctxs_to_free_);
