Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21076 )
Change subject: KUDU-3557 Simple way to change maximum thread cache size of tcmalloc. ...................................................................... Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/21076/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21076/3//COMMIT_MSG@12 PS3, Line 12: but : thread cache does not need to release the memory to central free : list all the time if max_total_thread_cache_bytes is larger. I guess it's all originated from the way how the GC process manages the per-thread max_size cache's limit. By design, this results in situations when memory is often 'stolen' from a thread's cache during garbage collection by another thread, and it then 'steals' it back during the next run of the GC process if sees its max_size is too low. That's documented at https://gperftools.github.io/gperftools/tcmalloc.html. Maybe, providing a link to the docs page in this description could be useful to people who want to understand what's going on: ... Each time the cache runs a garbage collection, it will also try to grow its max_size. If the sum of the thread cache sizes is less than --tcmalloc_max_total_thread_cache_bytes, max_size grows easily. If not, thread cache 1 will try to steal from thread cache 2 (picked round-robin) by decreasing thread cache 2's max_size. In this way, threads that are more active will steal memory from other threads more often than they are have memory stolen from themselves. Mostly idle threads end up with small caches and active threads end up with big caches. Note that this stealing can cause the sum of the thread cache sizes to be greater than --tcmalloc_max_total_thread_cache_bytes until thread cache 2 deallocates some memory to trigger a garbage collection ... -- To view, visit http://gerrit.cloudera.org:8080/21076 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3cb8c6ed6a8f24c63258ae65f8b841fe74b75308 Gerrit-Change-Number: 21076 Gerrit-PatchSet: 3 Gerrit-Owner: Song Jiacheng <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Tue, 27 Feb 2024 19:41:35 +0000 Gerrit-HasComments: Yes
