Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-5419: Check for cancellation when building hash tables ......................................................................
IMPALA-5419: Check for cancellation when building hash tables When inserting build rows into the hash table for each partition in PhjBuilder::Partition::BuildHashTable(), we didn't check for cancellation. This may lead to orphaned fragments running for excessive amount of time and consuming a lot of memory after the query has been cancelled. This change fixes the problem by checking for cancellation per row batch in the loop inside PhjBuilder::Partition::BuildHashTable(). Manually verified with the following query that cancelling the query right after all build rows have been returned by the scan node will not cause memory usage to continue going up. select /* +straight_join */ count(*) from tpch20_parquet.lineitem t1 join /* +broadcast */ tpch20_parquet.lineitem t2 on t1.l_orderkey = t2.l_orderkey Change-Id: I8047f532f55dc0118f7a843c91275f752c8a190d Reviewed-on: http://gerrit.cloudera.org:8080/7047 Reviewed-by: Sailesh Mukil <[email protected]> Reviewed-by: Dan Hecht <[email protected]> Tested-by: Impala Public Jenkins --- M be/src/exec/partitioned-hash-join-builder.cc 1 file changed, 1 insertion(+), 0 deletions(-) Approvals: Impala Public Jenkins: Verified Sailesh Mukil: Looks good to me, but someone else must approve Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/7047 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I8047f532f55dc0118f7a843c91275f752c8a190d Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Sailesh Mukil <[email protected]>
