Henry Robinson has uploaded a new patch set (#2). Change subject: IMPALA-3106: Don't allow scanner threads to spin before ranges are issued ......................................................................
IMPALA-3106: Don't allow scanner threads to spin before ranges are issued Scanner threads are started in HdfsScanNode::Open(), before the actual scan ranges are issued in GetNext(). If there is a delay between these methods, the scanner threads will spin on DiskIoMgr::GetNextRange() which returns immediately thinking that all the scan ranges are finished (rather than not yet issued). This patch works around the issue by forcing scanner threads to wait until the initial ranges are issued. That wait is limited to 20ms, after which the thread will wake up to check if it should yield based on contention for thread tokens; otherwise it waits again. Fixing this drops CPU usage to nearly 0 during a large delay between GetNext() and Open() (e.g. by artificially delaying runtime filter delivery), where before nearly one CPU per scanner thread was consumed. Change-Id: I064de1ae037b578c70d65503895a920461af877a --- M be/src/exec/hdfs-scan-node.cc M be/src/exec/hdfs-scan-node.h M be/src/util/counting-barrier.h 3 files changed, 32 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/79/2379/2 -- To view, visit http://gerrit.cloudera.org:8080/2379 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I064de1ae037b578c70d65503895a920461af877a Gerrit-PatchSet: 2 Gerrit-Project: Impala Gerrit-Branch: cdh5-2.5.0_5.7.0 Gerrit-Owner: Henry Robinson <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]>
