This is expected behavior with "-sfile" -- Varnish will use all available RAM to cache its much larger on-disk cache component. If you can get away with memory-only caching, you can use "-smalloc" to utilize a specific amount of RAM.
If you do want to use "-sfile" but are having issue with swap or memory allocation, increasing /proc/sys/vm/min_free_kbytes (or vm.min_free_kbytes via sysctl) to something like 131072 will stabilize performance /in my experience/. Hope it helps, -- kb On Thu, Apr 7, 2011 at 03:38, Jean-Francois Laurens < [email protected]> wrote: > Hi there ! > > I recently updated varnish from version 2.0.6 to 2.1.5 on a centos 5.4 > server. > > After pretty much a day running 2.1.5, the system went out of memory. The > server has 8G of memory and we specified a file of 50Gb as varnish cache > file. > > I changed some options yesterday following Kristian Lyngstol > recommendations ( * > http://kristianlyng.wordpress.com/2009/10/19/high-end-varnish-tuning/)*and > did a restart so the system is now running. > > I decreased the size of the cache file to 40Gb to. > > Here is a top output where you can see that varnish is consuming pretty > much all the memory available: > top -cbn 1 > top - 11:54:24 up 625 days, 19:51, 1 user, load average: 0.52, 0.41, 0.36 > Tasks: 110 total, 1 running, 109 sleeping, 0 stopped, 0 zombie > Cpu(s): 0.1%us, 0.2%sy, 0.0%ni, 99.1%id, 0.5%wa, 0.0%hi, 0.1%si, > 0.0%st > Mem: 8177764k total, 8132712k used, 45052k free, 13100k buffers > Swap: 4096564k total, 263324k used, 3833240k free, 7337568k cached > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > 3733 varnish 15 0 44.6g 7.1g 6.7g S 2.0 91.4 9:56.07 > /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f > /etc/varnish/default.vcl -T 0.0.0.0:6082 -t 120 -w 100,3000,120 -u varnish > -g varn > .... > 3732 root 15 0 106m 720 492 S 0.0 0.0 0:00.72 > /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f > /etc/varnish/default.vcl -T 0.0.0.0:6082 -t 120 -w 100,3000,120 -u varnish > -g varn > > Regularly it’s consuming more and more swap aswell: > From 15mn ago: > free -m > total used free shared buffers cached > Mem: 7986 7941 44 0 12 7163 > -/+ buffers/cache: 765 7220 > Swap: 4000 258 3742 > > From 5 mn ago: > free -m > total used free shared buffers cached > Mem: 7986 7940 45 0 15 7163 > -/+ buffers/cache: 761 7224 > Swap: 4000 260 3739 > > From 1mn ago: > free -m > total used free shared buffers cached > Mem: 7986 7940 45 0 17 7160 > -/+ buffers/cache: 763 7222 > Swap: 4000 262 3737 > > I understood that varnish relies on the system itself for memory > allocation. > Could we say that the bigger you file cache size is the more varnish need > memory ? > Or Am I facing an issue with my vcl that could have this side effect or ?? > > In addition to this I've seen the percentage hit decreasing from 65% > average with 2.0.6 to 40% with 2.1.5. > Apart from changing obj to beresp in the vcl_fetch, I had to specify > return(deliver) or return(pass) instead of only deliver or pass. > Would it be a reason for this ? > Would you be able to give me pointers for me to dig a bit more ? > > I Attached a plot from cacti wich would show the difference between 2.0.6 > and 2.1.5 (the gap is due to the time to update the server, the trafic was > sent to an other server ...) > > > Thanks in advance for your advices ! > > Jean-Francois Laurens > Ingénieur Système Unix > Resources et Développement > Secteur Backend > *RTS - Radio Télévision Suisse > *Quai Ernest-Ansermet 20 > Case postale 234 > CH - 1211 Genève 8 > T +41 (0)58 236 81 63 > > > _______________________________________________ > varnish-misc mailing list > [email protected] > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >
_______________________________________________ varnish-misc mailing list [email protected] http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
