This is an automated email from the ASF dual-hosted git repository.
bmahler pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
The following commit(s) were added to refs/heads/master by this push:
new 2836057c8 Reduced the number of jemalloc arenas.
2836057c8 is described below
commit 2836057c8881a62139b5db1f175d3d027bf2e645
Author: Ilya Pronin <[email protected]>
AuthorDate: Wed Sep 5 17:58:43 2018 -0700
Reduced the number of jemalloc arenas.
It seems that the default high number of allocation arenas (nproc*4)
combined with the high number of libprocess worker threads, which are
mostly "dormant", causes the allocator request a lot of memory from the
system resulting in high process RSS
(https://jira.twitter.biz/browse/AURORA-12085).
Most of jemalloc parameters, including the number of arenas, cannot be
changed in runtime. So the way we configure it is a bit hacky - through
the global string literal. The other way is using MALLOC_CONF
environment variable, which can still be used as an override.
The new number of arenas was selected pretty much arbitrarily. It helped
reduce the agent's RSS (better than 8, same as 2) and existing
benchmarks didn't reveal any performance hits caused by it.
Review: https://reviewboard.twitter.biz/r/931992
---
src/slave/main.cpp | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/src/slave/main.cpp b/src/slave/main.cpp
index 84b813cf3..7eeeaae65 100644
--- a/src/slave/main.cpp
+++ b/src/slave/main.cpp
@@ -134,6 +134,18 @@ using std::string;
using std::vector;
+#ifdef ENABLE_JEMALLOC_ALLOCATOR
+// Reduce the number of arenas jemalloc uses. On multi-core systems
+// jemalloc creates multiple allocation arenas to reduce lock
+// contention between threads using a single arena. The default high
+// number of arenas (nproc*4) causes high RSS for the agent use case.
+// The new number was selected pretty much arbitrarily. It helped
+// reduce the agent's RSS (better than 8, same as 2) and existing
+// benchmarks didn't reveal any performance hits caused by it.
+const char* malloc_conf = "narenas:4";
+#endif
+
+
#ifdef __linux__
// Move the slave into its own cgroup for each of the specified
// subsystems.
