Hello Jean-Daniel Cryans, Adar Dembo,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/6793
to review the following change.
Change subject: process_memory: incrementally track total process consumption
......................................................................
process_memory: incrementally track total process consumption
This switches CurrentConsumption() from calling into TCMalloc to instead
using its own tracking, based on installing new/delete hooks into
tcmalloc. The hooks incrementally update a global LongAdder for
consumption, and we just query that object when we need to know the
current value.
This is meant to address a performance regression seen on YCSB where the
tablet server spends a lot of time in tcmalloc::ThreadCache::GetThreadStats
trying to compute memory usage.
I wrote a small micro-benchmark which starts 200 threads which allocate
memory and call CurrentConsumption(). Without the patch, it only
performed about 650K iters/sec. With the patch, it performs around 5M
iters/sec. With no tracking at all (no hooks and no consumption calls)
the performance is around 25M iterations/second. So, we are definitely
paying some cost for the hooks, but this should at least be an
improvement. If it doesn't win back all the performance, we can look for
an even better approach.
On the side, this changes process_memory to use GoogleOnce instead of
std::once. This sped up the microbenchmark significantly (almost 2x),
apparently due to better inlining.
Change-Id: I3293110b7eda7f083bf400e165be3d782c441c49
---
M src/kudu/util/CMakeLists.txt
A src/kudu/util/process_memory-test.cc
M src/kudu/util/process_memory.cc
3 files changed, 131 insertions(+), 25 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/93/6793/1
--
To view, visit http://gerrit.cloudera.org:8080/6793
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3293110b7eda7f083bf400e165be3d782c441c49
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Jean-Daniel Cryans <[email protected]>
