Re: Question about threads
On Wed, 18 Jun 2008 13:24:52 -0700 Michael S. Fischer [EMAIL PROTECTED] wrote: On Wed, Jun 18, 2008 at 4:51 AM, Rafael Umann [EMAIL PROTECTED] wrote: If it is a 32bits system, probably the problem is that your stack size is 10Mb. So 238 * 10mb = ~2gb I decreased my stack size to 512Kb. Using 1gb storage files i can now open almost 1900 threads using all the 2gb that 32bits can alloc. So, my Varnish makes 2 hits/second serving clients! What is your request:connection ratio? Best regards, --Michael Unfortunately now i dont have servers doing 2 hits/second, and thats why i dont have stats for you. I have 6 servers runing this service now, each one doing 5500 hits/sec with 10% CPU usage, and this infrastructure scales to 2 hits/sec each server. For test purpose only i let just 2 servers with all the traffic, so i saw this limit i told you. In this 2 hits/sec im seeing two bottle necks: 1. The 32bits arch (cant open threads and the storage file is too small), so im moving into 64bits. 2. The cpu usage of the listener process with 2 hits/sec is getting to 100% in one CPU (we also made a patch to improve CPU usage, and it is avaliable in this last trunk version.. without this patch varnish was doing 8000 hits/sec). But anyway, i spoke to Poul, and he told me that the listener process will be changed soon (i really hope so, because i would love to use more than 30% of my cpu!). Anyway, varnish is doing a great job for me!!! Some stats taken now with 5500 hits/sec: # netstat -nap|grep :80 |grep SYN_REC |wc -l 166 # netstat -nap|grep :80 |grep ESTAB |wc -l 17139 # netstat -nap|grep :80 |grep FIN |wc -l 1454 # netstat -nap|grep :80 |grep TIME_W |wc -l 7766 # varnishstat Hitrate ratio: 10 13 13 Hitrate avg: 0.9990 0.9990 0.9990 36189916 476.66 250.71 Client connections accepted 404804204 5494.13 2804.30 Client requests received 391586079 5335.24 2712.74 Cache hits 10975643 157.8976.03 Cache hits for pass 1125709 1.00 7.80 Cache misses 12101360 158.8983.83 Backend connections success 0 0.00 0.00 Backend connections failures 0 0.00 0.00 Backend connections reuses 11139673 141.9077.17 Backend connections recycles 350-1.00 0.00 Backend connections unused 13447 .. N struct srcaddr 11353 .. N active struct srcaddr 30351 .. N struct sess_mem 17391 .. N struct sess 47069 .. N struct object 47080 .. N struct objecthead 97813 .. N struct smf 4185 .. N small free smf 5 .. N large free smf 0 .. N struct vbe_conn 556 .. N worker threads 556 0.00 0.00 N worker threads created 0 0.00 0.00 N worker threads not created 0 0.00 0.00 N worker threads limited 0 0.00 0.00 N queued work requests 556 0.00 0.00 N overflowed work requests 0 0.00 0.00 N dropped work requests 0 .. N expired objects 1079159 .. N LRU nuked objects 0 .. N LRU saved objects 0 .. N objects on deathrow 48 0.00 0.00 HTTP header overflows 0 0.00 0.00 Objects sent with sendfile 135051222 1812.72 935.58 Objects sent with write 36189893 494.65 250.71 Total Sessions 404810596 5491.13 2804.35 Total Requests 0 0.00 0.00 Total pipe 0 0.00 0.00 Total pass 12101371 158.8983.83 Total fetch 93389657948 1269372.98646962.32 Total header bytes 418107385660 5962840.69 2896463.38 Total body bytes 541249189.9437.50 Session Closed 0 0.00 0.00 Session Pipeline 1623343 0.0011.25 Session Read Ahead 398293975 5409.19 2759.20 Session herd 15780035398212996.82109317.12 SHM records 516154306 6307.55 3575.69 SHM writes 266181533.9818.44 SHM MTX contention 23279696 242.83 161.27 allocator requests 93623 .. outstanding allocations 910299136 .. bytes allocated []s, -- Rafael Umann [EMAIL PROTECTED] Suporte Engenharia 1 Terra Networks Brasil S/A Tel: +55 (51) 3284-4344 ___
Core Dump
I am running #2651 trunk. Can someone on the list send me a link to debugging instructions so I can post here / on trac? Thanks, Max Jun 19 11:16:48 can05 varnishd: Child (781) said Assert error in STV_alloc(), stevedore.c line 71: Jun 19 11:16:48 can05 kernel: pid 781 (varnishd), uid 101: exited on signal 6 Jun 19 11:16:48 can05 varnishd: Child (781) said Condition((st) != NULL) not true. Jun 19 11:16:48 can05 varnishd: Child (2934) said Child starts Jun 19 11:16:48 can05 varnishd: Child (2934) said managed to mmap 1073741824 bytes of 1073741824 Jun 19 11:16:48 can05 varnishd: Child (2934) said Ready Jun 19 11:16:49 can05 kernel: pid 780 (varnishd), uid 0: exited on signal 6 (core dumped) ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Question about threads
On Thu, Jun 19, 2008 at 5:37 AM, Rafael Umann [EMAIL PROTECTED] wrote: What is your request:connection ratio? Unfortunately now i dont have servers doing 2 hits/second, and thats why i dont have stats for you. Actually, it's right there in your varnishstat output: 36189916 476.66 250.71 Client connections accepted 404804204 5494.13 2804.30 Client requests received Your request:connection ratio is 10:1! This is a very good situation to be in. varnish doesn't have to spawn nearly as many threads as it would if the ratio were lower, as is common at many other sites. I have 6 servers runing this service now, each one doing 5500 hits/sec with 10% CPU usage, and this infrastructure scales to 2 hits/sec each server. It's probably inaccurate to assume that things will scale linearly :) Best regards, --Michael ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Question about threads
On Thu, 19 Jun 2008 08:05:21 -0700 Michael S. Fischer [EMAIL PROTECTED] wrote: On Thu, Jun 19, 2008 at 5:37 AM, Rafael Umann [EMAIL PROTECTED] wrote: What is your request:connection ratio? Unfortunately now i dont have servers doing 2 hits/second, and thats why i dont have stats for you. Actually, it's right there in your varnishstat output: 36189916 476.66 250.71 Client connections accepted 404804204 5494.13 2804.30 Client requests received Your request:connection ratio is 10:1! This is a very good situation to be in. varnish doesn't have to spawn nearly as many threads as it would if the ratio were lower, as is common at many other sites. OK, what i meant is that i dont have the stats for the amount of hits/second that i told u. Anyway, with this amount of traffic, with this timeouts that i set, it will probablly be the same. I have 6 servers runing this service now, each one doing 5500 hits/sec with 10% CPU usage, and this infrastructure scales to 2 hits/sec each server. It's probably inaccurate to assume that things will scale linearly :) Its not with Varnish :) actually i was surprised with this, but it scales almost linearly to the 2 hits/sec (ok, ok, with a litle bit of response time decrease), but it dont go any further. Best regards, --Michael E-mail verificado pelo Terra Anti-Spam. Para classificar como spam ou não spam, visite http://mail.terra.com.br/cgi-bin/reportspam.cgi?+_d=SCY1ODAyNDQ3I3Blcm0hdGVycmEmMSwxMjEzODg3OTIzLjk1ODU0MC4xNjgyMC5nYW5hbm9xdWUudGVycmEuY29tLDU0Mzc= Verifique periodicamente a pasta Spam para garantir que apenas mensagens indesejadas sejam classificadas como Spam. Esta mensagem foi verificada pelo E-mail Protegido Terra. Atualizado em 19/06/2008 -- Rafael Umann [EMAIL PROTECTED] Suporte Engenharia 1 Terra Networks Brasil S/A Tel: 55 (51) 3284-4344 ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Hostname in error pages
Hello, With multiple servers behind a load balancer, how do I add the hostname to the error page (i.e. 503 service unavailable) so we can identify the system giving the error? Thanks, Max ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
