Hi, sorry to reply so late, here are the results of my tests : 0 -- - system has 32GB of RAM, 8GB of SWAP and 3TB of available storage mounted on /var/cache/varnish - allocation : "file,/var/cache/varnish/limited100GB.bin,100GB" - 4.2 million of objects max in cache - 30GB of RAM used and 1.4GB of swap - varnishd child process not killed any more - 200k requests per minute before starting eating SWAP, 95k after eating some SWAP - oldest objects are replaced by newer ones, IO average is 53% on HDD
Today, I ran some other tests after a memory upgrade from 32GB to 64GB, here are the results : 1 -- - system has 64GB of RAM, 8GB of SWAP and 3TB of available storage mounted on /var/cache/varnish - allocation : "file,/var/cache/varnish/limited200GB.bin,200GB" - 8.8 million of objects max in cache - 56GB of RAM used and 1.9GB of swap - varnishd child process not killed any more - starting at 250k requests per minute for 10 minutes then 200k before starting eating SWAP, 145k after eating some SWAP - oldest objects are replaced by newer ones, IO average is 53% on HDD 2 -- - system has 64GB of RAM, 50MB of SWAP and 3TB of available storage mounted on /var/cache/varnish - allocation : "file,/var/cache/varnish/limited300GB.bin,300GB" - 6.8 million of objects max in cache before varnishd killed - 60GB of RAM used and 500MB of swap - varnishd child process was killed because running out of swap space - starting at 250k requests per minute for 10 minutes then 200k before starting eating SWAP, 95k after eating some SWAP - oldest objects are replaced by newer ones, IO average is 53% on HDD Does anybody have a large platform running Varnish ? I would like to raise 200 millions of objects but it seems it needs a very large amount of RAM or I have a problem with memory leaks using Varnish 3.0.5 under FreeBSD 10... Regards, Denis > De: "Paul Armstrong" <[email protected]> > Envoyé: Mercredi 6 Août 2014 17:20:38 > > Look at varnishstat and see if the temporary storage is large as this > can also be restricted (but isn't by default). For my workloads I > typically have malloc set to only 25-50% of RAM in order to avoid > running out. > > Paul > > On 6 Aug 2014, at 5:19, [email protected] wrote: > > >> I run a similar configuration except instead of file I use > >> > >> malloc,26G > >> > >> Have you tried that? does it react the same? > > > > Hi, > > I tried that and the system became unstable after 3 minutes. I had > > to reboot because of <defunct> processes appears after that. No > > hardware failure nor specific system log. After the reboot, I made > > another injection test with "file" storage. > > > > My conclusion is that Varnish really eats 1GB per 1 million objects > > but it also consume 22GB RAM for the rest of its activity !!! > > What the hell is it doing with 22GB of RAM ? 80MB for SHM logs, > > 100MB for threads ok but what about the rest ? > > > > The platform I use runs at 2k requests per second for one injection > > job. One thing I can do is limiting "file" storage to 100GB to be > > sure the system won't use SWAP. I am currently running this test. > > > > Regards, > > Denis _______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
