On 27/09/11 12:43, Aurélien Lemaire wrote:
Le 16/09/2011 08:55, Jorge Nerín a écrit :
On Thu, Sep 15, 2011 at 11:29, Aurélien Lemaire <[email protected]
<mailto:[email protected]>> wrote:
Good day folks,
First of all, varnish is an outstanding piece of software that my company
and i are addicted to. So thanks to all the coders.
Here is my problem :
I allocated varnish 1G of RAM on a website that can have more than 2 Go
of possible cacheable objects . Not to worry though as any proxy-cache
system should smartly nuke old objects to make place to new one to live
peacefully within its allocated RAM. And that's where Varnish behave
unexpectedly : each time it need to nuke SOME objects : it nukes ALMOST
ALL of them (often ~80% of my 35k objects) which is quite aggressive ;
thus i lost almost all my cache....IRK !
3 Munin graphs attached to see the problem clearly : big drop each time a
nuking happens.
To make sure my pbr is about varnish nuking system : i increased from 1G
to 3G(more than the max possible 2G cacheable objects) on another varnish
of this platefom (this website is delivered by multiple front/varnish
server all stricly similar and independant) and this issue disappeared
(no more nuking : no lost of ~80%of my objects)
Here is my env :
Debian 5.0.8 64 bits on 2.6.32-5-openvz-amd64 kernel
Varnish 2.1.3 SVN 5049:5055(debian package 2.1.3-8)
200 varnish 's worker threads running constantly (no issue on workers)
30req/s average with 60/s in peak
Daemon run as such :
/usr/sbin/varnishd -P /var/run/varnishd.pid -a :80 -T localhost:6082 -S
/etc/varnish/secret -f /etc/varnish/serverx.vcl -w 100,1024 -s
file,/var/lib/varnish/serverx/varnish_storage.bin,3G
Here a quick varnishstat -1 :
Is it normal varnish behaviour ? sounds like a bug to me.
Am i missing some tuning (lru_interval) to soften the nuking algo ?
Do you need more info ?
helps appreciated here ;-)
Regards, Aurelien Lemaire
It could be someone downloading a large file (like a ~700Mb iso file) and
varnish nuking objects to make room for this file (even if its configured to
not cache it).
Try to get a varnishlog trace of the moment the nuking begins.
--
Jorge Nerín
<[email protected] <mailto:[email protected]>>
This website does not content any files bigger than a couple of Mb.
Will see what i can do to get the varnishlog during the nuking.
--
aurelien lemaire
Providing a reduced-to-the-minimum varnishlog on this stressed prod env won't be
easy for me moreover i doubt that nuking activity is reported by varnishlog . So
meanwhile :
My "agressive nuking" problem seems to be known and often-encountered :
- http://www.gossamer-threads.com/lists/varnish/dev/9631
- http://www.mail-archive.com/[email protected]/msg03684.html
- Quite some thread without any outcome found by google search where all the
people just fix it by increasign the the "-s file" size which only postpone the
problem
On this first link Artur ( a varnish dev
http://www.gossamer-threads.com/lists/engine?user=137;list=varnish ) shortly
admit "/... There are problems with the fragmentation of the store.../" on the
1st of april 2009.
On the second link there is all the varnishlog and backtrace infos needed to
open the bug report to get it fixed
Thus is it possible to open this bug report officially as "to be fixed" ?
At the same time i'll try to swith to "-s malloc " as suggested by Artur and
keep you all informed.
Cheers,
--
Aurélien Lemaire
_______________________________________________
varnish-misc mailing list
[email protected]
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
