Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9752 )
Change subject: KUDU-972. Cache should track the memory overhead ...................................................................... KUDU-972. Cache should track the memory overhead This changes the cache 'charge' mechanism to use malloc_usable_size on its actual allocations rather than tracking only the cached _values_. This addresses two sources of inaccuracy: 1) we weren't counting the cache key and LRUHandle overhead, which is apparently 104 bytes per object. 2) we weren't accounting for tcmalloc internal fragmentation, which can be significant for allocations in our typical block size range. For example, a 4097-byte allocation will use 4608 bytes (12% overhead). In an insert-heavy workload most of hte cache is used for bloom blocks which are ~4kb, so the above is a very common case. Change-Id: I6cc69004dccb1cc372fdedbac321154e46a92263 Reviewed-on: http://gerrit.cloudera.org:8080/9752 Reviewed-by: Adar Dembo <[email protected]> Tested-by: Kudu Jenkins --- M src/kudu/cfile/block_cache-test.cc M src/kudu/cfile/block_cache.cc M src/kudu/cfile/block_cache.h M src/kudu/util/cache.cc M src/kudu/util/cache.h M src/kudu/util/nvm_cache.cc 6 files changed, 60 insertions(+), 15 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/9752 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I6cc69004dccb1cc372fdedbac321154e46a92263 Gerrit-Change-Number: 9752 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <[email protected]>
