Joe McDonnell has uploaded a new patch set (#2). Change subject: IMPALA-5598: Fix excessive dumping in MemLimitExceeded ......................................................................
IMPALA-5598: Fix excessive dumping in MemLimitExceeded ExecQueryFInstances RPC timeouts in stress tests were traced to excessive dumping in MemLimitExceeded() and LogUsage(). The QueryState::Init() hits the process memory limit, and this causes MemLimitExceeded to dump the process memory tracker. On the stress test, this involves dumping hundreds of queries and all the structures underneath. This is expensive and the contention between threads accessing these structures causes RPC timeouts. This adds the ability to the limit the recursive depth when dumping memory trackers. It also modifies the dumping in MemLimitExceeded() to have the following semantics: 1. The query memory tracker is always logged in full if it exists. 2. The process memory tracker is logged if the query memory tracker doesn't exist or if the process memory limit is being hit. The process memory tracker is limited to dumping only two levels of children. Other uses of LogUsage() are not limited (e.g. /memz and the query memory page on the web UI). A stress test run with the process memory tracker limited to two levels showed no RPC timeouts. Change-Id: Ib151c3894d4c43b508779e6809d77e2f8af89cf2 --- M be/src/exec/exec-node.cc M be/src/exec/hash-table-test.cc M be/src/runtime/initial-reservations.cc M be/src/runtime/mem-tracker.cc M be/src/runtime/mem-tracker.h M be/src/runtime/runtime-state.cc M be/src/service/impala-http-handler.cc M be/src/util/default-path-handlers.cc 8 files changed, 61 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/7597/2 -- To view, visit http://gerrit.cloudera.org:8080/7597 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib151c3894d4c43b508779e6809d77e2f8af89cf2 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Matthew Jacobs <m...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
