[
https://issues.apache.org/jira/browse/IMPALA-13479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-13479.
------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> Patch gperftools to allow max_total_thread_cache_bytes to exceed 1GB
> --------------------------------------------------------------------
>
> Key: IMPALA-13479
> URL: https://issues.apache.org/jira/browse/IMPALA-13479
> Project: IMPALA
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: Impala 4.5.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Critical
> Fix For: Impala 5.0.0
>
>
> gperftools limits max_total_thread_cache_bytes to 1GB here:
> [https://github.com/gperftools/gperftools/blob/gperftools-2.10/src/thread_cache.cc#L520-L523]
> {noformat}
> void ThreadCache::set_overall_thread_cache_size(size_t new_size) {
> // Clip the value to a reasonable range
> if (new_size < kMinThreadCacheSize) new_size = kMinThreadCacheSize;
> if (new_size > (1<<30)) new_size = (1<<30); // Limit to 1GB{noformat}
> I confirmed that setting --tcmalloc_max_total_thread_cache_bytes=2147483648
> still results in a 1GB limit.
> Sometimes, we would want a higher limit for systems with a large amount of
> memory and CPUs. For example, some systems now have 1TB of memory and 96
> CPUs. With high concurrency, there is high contention on tcmalloc locks on
> central data structures. Increasing the total thread cache size could avoid
> this, and a value higher than 1GB is still a small part of system memory.
> We can patch our toolchain gperftools to allow a higher value (and notify
> gperftools community).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
