I'm for making jemalloc default as well with adding an opt-out option to CMake and autotools build scripts.
On Sat, Aug 19, 2017 at 3:23 AM, Benjamin Mahler <bmah...@apache.org> wrote: > This will be a big win Benno, thanks for driving it! > > Nice to see that the heap profiling overhead is really low, I'd love to be > able to just hit an endpoint on the master or agent and get a memory > profile. > > I'm a +1 for making it the default, however, I seem to recall hearing that > there were some issues with JNI? > > Ben > > On Fri, Aug 18, 2017 at 3:49 AM, Benno Evers <bev...@mesosphere.com> > wrote: > > > Hi all, > > > > I would like to propose bundling jemalloc as a new dependency > > under `3rdparty/`, and to link Mesos against this new memory > > allocator by default. > > > > > > # Motivation > > > > The Mesos master and agent binaries are, ideally, very long-running > > processes. This makes them susceptible to memory issues, because > > even small leaks have a chance to build up over time to the point > > where they become problematic. > > > > We have seen several such issues on our internal Mesos installations, > > for example https://issues.apache.org/jira/browse/MESOS-7748 > > or https://issues.apache.org/jira/browse/MESOS-7800. > > > > I imagine any organization running Mesos for an extended period > > of time has had its share of similar issues, so I expect this > > proposal to be useful for the whole community. > > > > > > # Why jemalloc? > > > > Given that memory issues tend to be most visible after a given > > process has been running for a long time, it would be great to > > have the option to enable heap tracking and profiling at runtime, > > without having to restart the process. (This ability could then > > be connected to a Mesos endpoint, similar to how we can adjust > > the log level at runtime) > > > > The two production-quality memory allocators that have this > > ability currently seem to be tcmalloc and jemalloc. Of these, > > jemalloc does produce in our experience better and more > > detailed statistics. > > > > > > # What is the impact on users who do not need this feature? > > > > Naturally, not every single user of Mesos will have a need > > for this feature. To ensure these users would not experience serious > > performance regressions as a result of this change, we conducted > > a preliminary set of benchmarks whose results are collected > > under https://issues.apache.org/jira/browse/MESOS-7876 > > > > It turns out that we could probably even expect a small speedup (1% - 5%) > > as a nice side-effect of this change. > > > > Users who compile Mesos themselves would of course have the option > > to disable jemalloc at configuration time or replace it with their > > memory allocator of choice. > > > > > > > > I'm looking forward to hear any thoughts and comments. > > > > > > Thanks, > > -- > > Benno Evers > > Software Engineer, Mesosphere > > >
