This is a known and unfortunate issue with the latest versions of jemalloc and certain allocation patterns. You need to downgrade to jemalloc 3.6.
https://github.com/varnishcache/varnish-cache/issues/3511#issuecomment-771592238 -- Reza Naghibi VP of Technology Varnish Software On Wed, May 5, 2021 at 5:35 AM Marco Dickert - evolver group < [email protected]> wrote: > Hi Geoff, > > thanks for your answer! > > On 2021-04-29 18:00:39, Geoff Simmons wrote: > > You can also turn off THP system-wide (for all processes): > > $ echo never > /sys/kernel/mm/transparent_hugepage/enabled > > That might be your only option if the jemalloc version is too old. > > We diabled huge pages in the kernel, but this didn't solve the problem, > the RAM > consumption was unaffected. > > However, we found that the "transient storage" of varnish may be part of > our > problem. At least, we could mitigate this behaviour by limiting the > transient > storage in the start parameters (last option): > > ------- > DAEMON_OPTS="-a :6081 \ > -T :6082 \ > -f /etc/varnish/default.vcl \ > -p ping_interval=6 -p cli_timeout=10 -p pipe_timeout=600 \ > -p listen_depth=4096 -p thread_pool_min=200 -p > thread_pool_max=500 \ > -p workspace_client=128k -p nuke_limit=1000 \ > -S /etc/varnish/secret \ > -s malloc,6G \ > -s Transient=malloc,3G" > ------- > > Now varnish uses less RAM, and the varnishstats confirm that our limits > should > work: > > ------- > SMA.s0.g_bytes 6.00G -107.61K . 6.00G 6.00G > 6.00G > SMA.s0.g_space 132.70K 107.61K . 136.67K 137.35K > 137.35K > SMA.Transient.g_bytes 1.55G 1022.19 . 1.55G 1.55G > 1.55G > SMA.Transient.g_space 1.45G -1022.19 . 1.45G 1.45G > 1.45G > ------- > > However, varnish, in total, uses up to 12GB RAM instead of only 6GB > (cache) + > 3GB (transient). I tried to find a value in the varnishstat output which > might > indicate how this additional RAM is used, but didn't find anything useful > yet. > > So two questions: > > 1) What might cause varnish to consume considerably more RAM than (cache + > transient > storage)? > > 2) What objects exactly are stored in the transient storage? The > documentation > mentions "shortlived" objects [1] (the "shortlived" parameter is 200 in > our > varnish, which seems to be the varnish debian package default, since we > didn't set this explicitly), but I am not sure if that is limited to > cacheable objects or or not. Also I don't know how to determine which > requests lead to excessive usage of the transient storage. > > Thanks in advance for any further input. > > Cheers, > Marco > > [1] > https://varnish-cache.org/docs/trunk/users-guide/storage-backends.html#transient-storage > > -- > Mit freundlichen Grüßen > Marco Dickert > > Administration und Technik > evolver services GmbH > > Fon +49 / (0)3 71 / 4 00 03 78 24 > Fax +49 / (0)3 71 / 4 00 03 79 > > E-Mail [email protected] > Web https://www.evolver.de > > Sitz der Gesellschaft: Chemnitz > Handelsregister: Amtsgericht Chemnitz, HRB 22649 > Geschäftsführer: Torsten Gramann und Mathias Möckel > _______________________________________________ > varnish-misc mailing list > [email protected] > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >
_______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
