Hello Adar Dembo,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/6915
to review the following change.
Change subject: process_memory: go back to non-incremental tracking
......................................................................
process_memory: go back to non-incremental tracking
The incremental tracking has more overhead than expected, even with our
fancy striped counters, etc. In tpch_real_world benchmarks, the
LongAdder::IncrementBy() and GetAllocatedSize() calls in the hooks
were taking a significant amount of CPU (in the top 10 consumers).
Here's yet another approach: back to non-incremental tracking, but with
an optimization that, if we've calculated the memory usage in the last
50ms, or if another thread is already in the process of calculating, we
don't re-calculate it.
This has some minor risk of "overshooting" the memory limit, but since
our limiting is already probabilistic and not 100% thorough anyway, I
think that's acceptable.
Benchmarked using process_memory-test:
Before:
I0517 16:46:09.596946 7419 process_memory-test.cc:68] Performed 2.18441e+06
iters/sec
After:
I0517 16:45:50.497973 7051 process_memory-test.cc:68] Performed 1.22264e+08
iters/sec
(55x speedup)
I also ran tpch_real_world and graphed CPU consumed per inserted row and
found it to be noticeably better with this optimized version. I
spot-checked perf results and don't see any tcmalloc stat-related
methods in the high consumers list anymore. I also checked in this
workload that the heap usage was properly limited (indistinguishable
from the before-patch implementation)
Change-Id: I8823028de3ea260f1450d9bf34af2dc5a794b206
---
M src/kudu/tserver/tablet_server-stress-test.cc
M src/kudu/util/process_memory.cc
2 files changed, 23 insertions(+), 41 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/15/6915/1
--
To view, visit http://gerrit.cloudera.org:8080/6915
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8823028de3ea260f1450d9bf34af2dc5a794b206
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
