Hi, This looks like https://www.varnish-cache.org/trac/ticket/1539.
What varnish version are you running? On Wed, Nov 19, 2014 at 11:48 AM, Anand Shah <[email protected]> wrote: > > > Thank you Per for replying back. > > > Under desperately low memory conditions, the out-of-memory (OOM) killer > kicks in and picks up child process of varnish that holds up the virtual > memory space (in my case 524785412 kB .i.e 500 GB) and kills it dropping > all objects present in the vm space. I am not able to associate the child > crash that happens and swapping that place is doing everything wrong here. > Can you please re-consider the concerns and have someone look on the errors > that are written while breaking. > > > > Nov 19 15:06:36 cdn abrt[11775]: Saved core dump of pid 10079 > (/usr/sbin/varnishd) to /var/spool/abrt/ccpp-2014-11-19-15:06:33-10079 > (381931520 bytes) > Nov 19 15:06:36 cdn abrtd: Directory 'ccpp-2014-11-19-15:06:33-10079' > creation detected > Nov 19 15:06:36 cdn abrtd: Package 'varnish' isn't signed with proper key > Nov 19 15:06:36 cdn abrtd: 'post-create' on > '/var/spool/abrt/ccpp-2014-11-19-15:06:33-10079' exited with 1 > Nov 19 15:06:36 cdn abrtd: Deleting problem directory > '/var/spool/abrt/ccpp-2014-11-19-15:06:33-10079' > Nov 19 15:06:36 cdn varnishd[11921]: Child (10079) not responding to CLI, > killing it. > Nov 19 15:06:37 cdn varnishd[11921]: Child (10079) died signal=6 (core > dumped) > > Nov 19 15:06:37 cdn varnishd[11921]: Child (10079) Panic > message:#012Assert error in cnt_lookup(), cache/cache_req_fsm.c line > 411:#012 Condition((oc->exp_flags & (1<<7)) == 0) not true.#012thread = > (cache-worker)#012ident = > Linux,2.6.32-431.el6.x86_64,x86_64,-sfile,-smalloc,-hclassic,epoll#012Backtrace:#012 > 0x43b03d: /usr/sbin/varnishd() [0x43b03d]#012 0x43b34d: > /usr/sbin/varnishd() [0x43b34d]#012 0x440723: /usr/sbin/varnishd() > [0x440723]#012 0x44288d: /usr/sbin/varnishd(CNT_Request+0x441) > [0x44288d]#012 0x4339c0: /usr/sbin/varnishd(HTTP1_Session+0x429) > [0x4339c0]#012 0x4458cb: /usr/sbin/varnishd() [0x4458cb]#012 0x43dfb5: > /usr/sbin/varnishd(Pool_Work_Thread+0x4cb) [0x43dfb5]#012 0x456198: > /usr/sbin/varnishd() [0x456198]#012 0x4562c1: > /usr/sbin/varnishd(WRK_thread+0x27) [0x4562c1]#012 0x7f4027fc49d1: > /lib64/libpthread.so.0(+0x79d1) [0x7f4027fc49d1]#012req = 0x7ec30d0e8020 > {#012 sp = 0x7ec30e4257e0, vxid = 1074847970, step = R_STP_LOOKUP,#012 > req_body = R_BODY_NONE,#012 restarts = 0, esi_level = 0#012 sp = > 0x7ec30e4257e0 {#012 fd = 75, vxid = 1106140,#012 client = > 122.176.68.88 56100,#012 step = S_STP_WORKING,#012 },#012 worker = > 0x7f4017709bf0 {#012 ws = 0x7f4017709e08 {#012 id = "wrk",#012 > {s,f,r,e} = {0x7f40177093d0,0x7f40177093d0,(nil),+2048},#012 },#012 > VCL::method = 0x0,#012 VCL::return = deliver,#012 },#012 ws = > 0x7ec30d0e81b8 {#012 id = "req",#012 {s,f,r,e} = > {0x7ec30d0ea010,+872,(nil),+57360},#012 },#012 http[req] = {#012 ws = > 0x7ec30d0e81b8[req]#012 "GET",#012 > "/thumbnail1/3a8bbd2bb61f4f3f.jpg",#012 "HTTP/1.1",#012 > "Connection: keep-alive",#012 "Cache-Control: max-age=0",#012 > "Accept: image/webp,*/*;q=0.8",#012 "If-Modified-Since: Mon, 08 Sep > 2014 03:15:48 GMT",#012 "User-Agent: Mozilla/5.0 (Windows NT 5.1) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 > Safari/537.36",#012 "Referer: http://money.rediff.com/",#012 > "Accept-Language: en-US,en;q=0.8",#012 "X-Forwarded-For: 122.176 > > > > Regards, > Anand > > > > From: Per Buer <[email protected]> > Sent: Tue, 18 Nov 2014 23:10:50 > To: Anand_Shah <[email protected]> > Subject: Re: [email protected]: Swap flushes all objects in > cache > > > Anand, > > On Tue, Nov 18, 2014 at 2:01 PM, Anand Shah <[email protected]> wrote: >> >> I have noticed this everytime the memory offload happens. This is not a >> crash or segfault; coz the service does not go down and it keeps in running >> which is also evident in the graphs shared earlier. I can see the same >> behaviour everytime when operating system swaps. Can you confirm if this is >> a ideal behaviour with varnish? > > > Please keep user questions to the -misc list. > > This is most like misconfiguration causing the OOM killer to kill of the > child process. It doesn't keep running. Look at the accepted sessions in > the graph. It dives when this happens. > > > -- > *Per Buer* > CTO | Varnish Software AS > Cell: +47 95839117 > We Make Websites Fly! > www.varnish-software.com > [image: Register now] > <http://info.varnish-software.com/varnish-summits-autumn-2014-registration> > > _______________________________________________ > varnish-dev mailing list > [email protected] > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev >
_______________________________________________ varnish-dev mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
