Large string processing + concurrency + caching/memoization really brings out the worst in glibc malloc :< --- Documentation/public-inbox-tuning.pod | 5 ++++- examples/[email protected] | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/Documentation/public-inbox-tuning.pod b/Documentation/public-inbox-tuning.pod index 73246144..7d0690b4 100644 --- a/Documentation/public-inbox-tuning.pod +++ b/Documentation/public-inbox-tuning.pod @@ -165,8 +165,11 @@ capacity planning. Bursts of small object allocations late in process life contribute to fragmentation of the heap due to arenas (slabs) used internally by Perl. -jemalloc (tested as an LD_PRELOAD on GNU/Linux) appears to reduce +jemalloc (tested as an LD_PRELOAD on GNU/Linux) reduces overall fragmentation compared to glibc malloc in long-lived processes. +glibc malloc users may try setting C<MALLOC_MMAP_THRESHOLD_> to a lower +value (e.g. 131072) but that may require increasing the +C<sys.vm.max_map_count> sysctl. =head2 Other OS tuning knobs diff --git a/examples/[email protected] b/examples/[email protected] index 83d2e995..7437f086 100644 --- a/examples/[email protected] +++ b/examples/[email protected] @@ -12,8 +12,11 @@ Wants = public-inbox-netd.socket After = public-inbox-netd.socket [Service] -# An LD_PRELOAD for libjemalloc can be added here. It currently seems +# An LD_PRELOAD for libjemalloc can be added here. It is # more resistant to fragmentation in long-lived daemons than glibc. +# If you're unable to use jemalloc, setting MALLOC_MMAP_THRESHOLD_ +# to a lower value (e.g. 131072) but that may also require increasing +# the sys.vm.max_map_count sysctl. Environment = PI_CONFIG=/home/pi/.public-inbox/config \ PATH=/usr/local/bin:/usr/bin:/bin \ TZ=UTC \
