Todd Lipcon has submitted this change and it was merged. Change subject: process_memory: incrementally track total process consumption ......................................................................
process_memory: incrementally track total process consumption This switches CurrentConsumption() from calling into TCMalloc to instead using its own tracking, based on installing new/delete hooks into tcmalloc. The hooks incrementally update a global LongAdder for consumption, and we just query that object when we need to know the current value. This is meant to address a performance regression seen on YCSB where the tablet server spends a lot of time in tcmalloc::ThreadCache::GetThreadStats trying to compute memory usage. I wrote a small micro-benchmark which starts 200 threads which allocate memory and call CurrentConsumption(). Without the patch, it only performed about 650K iters/sec. With the patch, it performs around 5M iters/sec. With no tracking at all (no hooks and no consumption calls) the performance is around 25M iterations/second. So, we are definitely paying some cost for the hooks, but this should at least be an improvement. If it doesn't win back all the performance, we can look for an even better approach. On the side, this changes process_memory to use GoogleOnce instead of std::once. This sped up the microbenchmark significantly (almost 2x), apparently due to better inlining. Change-Id: I3293110b7eda7f083bf400e165be3d782c441c49 Reviewed-on: http://gerrit.cloudera.org:8080/6793 Reviewed-by: Adar Dembo <[email protected]> Tested-by: Kudu Jenkins --- M src/kudu/tserver/tablet_server-stress-test.cc M src/kudu/util/CMakeLists.txt A src/kudu/util/process_memory-test.cc M src/kudu/util/process_memory.cc M src/kudu/util/process_memory.h 5 files changed, 151 insertions(+), 26 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/6793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3293110b7eda7f083bf400e165be3d782c441c49 Gerrit-PatchSet: 3 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Jean-Daniel Cryans <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <[email protected]>
