Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-561: Allow multiple callbacks in a thread resource pool. ......................................................................
IMPALA-561: Allow multiple callbacks in a thread resource pool. Previously, thread resource manager only supports a single callback for each resource pool. The callback is invoked when a thread token is available. This mostly works as scan node is the only consumer and there is usually one scan node in a plan fragment. As shown in IMPALA-3064 and IMPALA-561, it's possible to generate a plan fragment with more than one scan nodes. In which case, one of the scan nodes may be running with single thread and in debug builds, a DCHECK will be hit. This change fixes the problem by allowing more than one callbacks in a given resource pool. The thread resource manager will go through all the registered callbacks in round robin fashion. This change also adds a missing thread token release call in HdfsScanNode::ThreadTokenAvailableCb(). Change-Id: Iddfff1feef0b59d407994ad3bc560166acbfa623 Reviewed-on: http://gerrit.cloudera.org:8080/2430 Reviewed-by: Dan Hecht <[email protected]> Tested-by: Internal Jenkins --- M be/src/exec/hdfs-scan-node.cc M be/src/exec/hdfs-scan-node.h M be/src/runtime/thread-resource-mgr-test.cc M be/src/runtime/thread-resource-mgr.cc M be/src/runtime/thread-resource-mgr.h M testdata/workloads/functional-query/queries/QueryTest/single-node-nlj.test 6 files changed, 162 insertions(+), 36 deletions(-) Approvals: Internal Jenkins: Verified Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/2430 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iddfff1feef0b59d407994ad3bc560166acbfa623 Gerrit-PatchSet: 6 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Michael Ho <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]>
