Hi Rubén,
Apparently we made a mistake a while ago by not confirming the varnish 3.0 defaults compared to 2.1. So we were happily still running with a sess_workspace of 16k, instead of 64k. This worked fine with our previous config, but since we added our sharding vmod, 16k is not enough anymore. So, the fixed config with 64k as sess_workspace has been running on one of our varnish servers for almost a week now. And it's been working fine. Even when a Justin Bieber link went viral yesterday, which put our systems to the real test. Not to mention this happened at the same time it's already peak-hour. And Varnish (of course;) happily served thousands of requests per second. So yes, I can now confirm: increasing sess_workspace fixed the assertion errors in my case. regards, Enno van Amerongen _____ From: Rubén Romero [mailto:[email protected]] To: Enno van Amerongen [mailto:[email protected]] Sent: Sun, 14 Apr 2013 12:52:48 +0200 Subject: Re: increase sess_workspace to fix assert error? Hi Enno, Increasing it will likely help. Not sure to how much, but do give it as much as you can and let us know how it went :-) Best, - Rubén Romero Varnish Software Den 2. apr. 2013 21.18 skrev "Enno van Amerongen" <[email protected]> følgende: Dear List, After deployment of a vmod we've written (that enables us to select a director by id in url), we've been experiencing assert errors. Below are the stack traces, but to not waste your time, here's what I think it boils down to: { overflow#012 id = "sess",#012 {s,f,r,e} = {0x7fed76034c78,+16384,(nil),+16384},#012 } 4k as session workspace has worked fine for us for years, but our vmod sharding adds hundreds of directors (as opposed to 4), while the number of backends stayed the same (about 25). So, my question is, do I just need to increase sess_workspace to fix this? If not, what else ;) Regards, Enno van Amerongen Both stacktraces Apr 2 18:43:32 obfuscated /var/www/varnish[12750]: Child (20498) Panic message: Assert error in vh_get_var_alloc(), vmod_var.c line 77:#012 Condition((v) != 0) not true.#012errno = 32 (Broken pipe)#012thread = (cache-worker)#012ident = Linux,2.6.32-5-amd64,x86_64,-smalloc,-smalloc,-hcritbit,epoll#012Backtrace:#012 0x439199: pan_backtrace+19#012 0x43946e: pan_ic+1ad#012 0x7fe137efe347: _end+7fe13786898f#012 0x7fe137efe862: _end+7fe137868eaa#012 0x7fecc5b93e73: _end+7fecc54fe4bb#012 0x7fecc5b939ea: _end+7fecc54fe032#012 0x7fecc5ba24f1: _end+7fecc550cb39#012 0x4417bd: VCL_recv_method+6d#012 0x41d8f5: cnt_recv+262#012 0x41e68d: CNT_Session+5cb#012sp = 0x7fed81293008 {#012 fd = 2849, id = 2849, xid = 918440385,#012 client = 10.1.1.1 51854,#012 step = STP_RECV,#012 handling = deliver,#012 err_code = 200, err_reason = (null),#012 restarts = 0, esi_level = 0#012 flags = #012 bodystatus = 4#012 ws = 0x7fed81293080 { overflow#012 id = "sess",#012 {s,f,r,e} = {0x7fed81293c78,+16384,(nil),+16384},#012 },#012 http[req] = {#012 ws = 0x7fed81293080[sess]#012 "GET",#012 "/f/146057654_2.jpg",#012 "HTTP/1.1",#012 "User-Agent: Opera/9.80 (Windows NT 6.1; WOW64) Presto/2.12.388 Version/12.14",#012 "Host: img.obfuscated.com",#012 "Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1",#012 "Accept-Language: fr-FR,fr;q=0.9,en;q=0.8",#012 "Referer: http://obfuscated",#012 "Connection: Keep-Alive",#012 },#012 worker = 0x7feece7efaa0 {#012 ws = 0x7feece7efcd8 { #012 id = "wrk",#012 {s,f,r,e} = {0x7feece7dd9f0,+16,(nil),+65536},#012 },#012 },#012 vcl = {#012 srcname = {#012 "input",#012 "Default",#012 "/obfuscated/img_backends.vcl",#012 "/obfuscated/img_directors.vcl",#012 So we removed the vmod_var from our configuration as it wasn't really required anyway. Then 13 minutes later, Varnish crashed again, now with a different, but similar assert error: Apr 2 19:07:22 obfuscated /var/www/varnish[12750]: Child (18818) Panic message: Assert error in VRT_int_string(), cache_vrt.c line 324:#012 Condition((p = WS_Alloc(sp->http->ws, size)) != 0) not true.#012thread = (cache-worker)#012ident = Linux,2.6.32-5-amd64,x86_64,-smalloc,-smalloc,-hcritbit,epoll#012Backtrace:#012 0x439199: pan_backtrace+19#012 0x43946e: pan_ic+1ad#012 0x4435f7: VRT_int_string+88#012 0x7fe686b9ce1f: _end+7fe686507467#012 0x7fe686b9c99a: _end+7fe686506fe2#012 0x7fe686ba7424: _end+7fe686511a6c#012 0x4417bd: VCL_recv_method+6d#012 0x41d8f5: cnt_recv+262#012 0x41e68d: CNT_Session+5cb#012 0x43aa0a: wrk_thread_real+8cc#012sp = 0x7fed76034008 {#012 fd = 2357, id = 2357, xid = 2009537706,#012 client = 80.239.243.196 59852,#012 step = STP_RECV,#012 handling = deliver,#012 err_code = 200, err_reason = (null),#012 restarts = 0, esi_level = 0#012 flags = #012 bodystatus = 4#012 ws = 0x7fed76034080 { overflow#012 id = "sess",#012 {s,f,r,e} = {0x7fed76034c78,+16384,(nil),+16384},#012 },#012 http[req] = {#012 ws = 0x7fed76034080[sess]#012 "GET",#012 "/f/148682988.jpg",#012 "HTTP/1.1",#012 "User-Agent: Opera/9.80 (Android; Opera Mini/7.5.32193/29.3134; U; nl) Presto/2.8.119 Version/11.10",#012 "Host: img.obfuscated.com",#012 "Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1",#012 "Accept-Language: nl,en;q=0.9",#012 "Referer: obfuscated",#012 "Connection: Keep-Alive",#012 "X-OperaMini-Phone: Android #",#012 "X-OperaMini-Phone-UA: Mozilla/5.0 (Linux; U; Android 2.2.2; nl-be; Stockholm Build/HuaweiU8180) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1",#012 "x-forwarded-for: obfuscated",#012 "Device-Stock-UA: Mozilla/5.0 (Linux; U; Android 2.2.2; nl-be; Stockholm Build/HuaweiU8180) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1",#012 "X _______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
_______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
