Hello Dan Burkert,

I'd like you to do a code review. Please visit


to review the following change.

Change subject: thirdparty: patch tcmalloc to improve AllocLarge performance

thirdparty: patch tcmalloc to improve AllocLarge performance

This pulls in an upstream tcmalloc pull request[1] which addresses O(n)
behavior in the large allocation path. Without this patch, large
allocations (>=1MB) do a linear scan of all large spans of free heap
to find the best fit, which is very expensive especially as the heap
grows more fragmented over time.

I tested this using YCSB workload C (100% random read) on an in-memory dataset
(10M rows on 6 nodes). As noted in KUDU-1465, scanners currently always
allocate 1MB buffers as a starting point even if the scan will only return a
small amount of data. These 1M allocations aggravate the AllocLarge path
and without this patch, AllocLarge was the top CPU consumer.

Prior to this fix, the workload managed about 20k reads/second. With the
fix, the workload averaged around 85k reads/second.

We should still fix KUDU-1465 to allocate smaller buffers, since small
allocations will always be faster (and less wasteful), but with this tcmalloc
fix, the performance issue is much less pronounced. The fix will also likely
help any other places that we might be making large allocations.

[1] https://github.com/gperftools/gperftools/pull/960

Change-Id: I4abffbd1cb02f99ebda1628d98e5c342ccb7f0f9
M thirdparty/download-thirdparty.sh
2 files changed, 496 insertions(+), 1 deletion(-)

  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/92/9392/1
To view, visit http://gerrit.cloudera.org:8080/9392
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4abffbd1cb02f99ebda1628d98e5c342ccb7f0f9
Gerrit-Change-Number: 9392
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Dan Burkert <d...@cloudera.com>

Reply via email to