Re: Varnish over memory allocation

2020-05-25 Thread Alex Wakefield
Hey Leon, $ cat /proc/meminfo MemTotal: 32940300 kB MemFree: 309836 kB MemAvailable:2083696 kB Buffers: 127768 kB Cached: 1999212 kB SwapCached:0 kB Active: 30551008 kB Inactive:1561588 kB Active(anon): 30067316 kB Inactive(anon):

Re: Varnish over memory allocation

2020-05-25 Thread info+varnish
Am Montag, den 25.05.2020, 22:38 +1000 schrieb Alex Wakefield: > Hey Dridi, > > Thanks for the reply. I originally thought that it might've been > transient storage as well however checking that shows that only 287MB > of storage has been allocated from it. Its allocated and released > 1.06TB

Re: Varnish over memory allocation

2020-05-25 Thread Alex Wakefield
Hey Dridi, Thanks for the reply. I originally thought that it might've been transient storage as well however checking that shows that only 287MB of storage has been allocated from it. Its allocated and released 1.06TB though, is that perhaps the cause of the issue? SMA.s0.g_bytes shows the

Re: Varnish over memory allocation

2020-05-25 Thread Dridi Boukelmoune
On Mon, May 25, 2020 at 8:07 AM Alex Wakefield wrote: > > Whoops, knew I forgot to specify something! > > We're using malloc. Command line switch is specifically `-s malloc,24GB` The -s option only specifies the storage size (HTTP responses with some metadata). The rest of Varnish's memory

Re: Varnish over memory allocation

2020-05-25 Thread Cosimo Streppone
On Mon, May 25, 2020, at 10:06, Alex Wakefield wrote: > > We're using malloc. Command line switch is specifically `-s malloc,24GB` > I think this is what you mean? Yes. Unfortunately (or rather fortunately) I never had problems like those with malloc storage. -- Cosimo

Re: Varnish over memory allocation

2020-05-25 Thread Alex Wakefield
Whoops, knew I forgot to specify something! We're using malloc. Command line switch is specifically `-s malloc,24GB` I think this is what you mean? Cheers, Alex On Mon, 25 May 2020, at 6:03 PM, Cosimo Streppone wrote: > On Mon, May 25, 2020, at 08:26, Alex Wakefield wrote: > > > > These

Re: Varnish over memory allocation

2020-05-25 Thread Cosimo Streppone
On Mon, May 25, 2020, at 08:26, Alex Wakefield wrote: > > These instances live on 32GB VMs (Ubuntu 18.04) but are tuned to only > use 24GB of memory to allow enough overhead for fragmentation and other > processes on the machine. If left alone they grow until OOM killer > kicks in and kills

Varnish over memory allocation

2020-05-25 Thread Alex Wakefield
Hi all, I currently have multiple Varnish 6.0.5 (varnish-6.0.5 revision 3065ccaacc4bb537fb976a524bd808db42c5fe40) instances deployed to production with Varnish modules (0.15.0) installed that consistently go over their memory allocation by several gigabytes. These instances live on 32GB VMs