Hi Mark,
Thanks for your reply. I agree about increasing the limit/or that I have a descriptor leak. What worries me is the infinite loop. I'll have a look in my code and in libevents code. If I find anything intresting or manage to reproduce it, I'll update. Thanks, Tomer. Sent from my LG Mobile ------ Original message------ From: Mark Ellzey Date: Sat, Jun 6, 2015 01:33 To: [email protected]; Subject:Re: [Libevent-users] Infinite Loop in libevent. On Fri, Jun 05, 2015 at 08:46:35PM +0000, Tomer Heber wrote: > Hi, > > > I've been using libevent (2.0.21) for some time now. I have encountered a > very rare and strange issue on one of our production environments. > > The machine was at 100% cpu, and had been caught on some kind of infinite > loop. Using GDB I traced it to libevent. > > > Just to clarify, I verified with tcpdump that the machine was not under a > DDOS attack. > > > strace showed a constant call to "gettimeofday" (or one of it's equivalent), > and a failure due to too many open file descriptors. > > > I was able to obtain the following from gdb > > #0 0x00007fd495cbf2d3 in vfprintf () from /lib64/libc.so.6 > > #1 0x00007fd495d7aaa0 in __vsnprintf_chk () from /lib64/libc.so.6 > > #2 0x00007fd4969d71c2 in vsnprintf (buf=0x7fd48abfc468 ": Too many open > files", buflen=1000, format=<value optimized out>, ap=<value optimized out>) > at /usr/include/bits/stdio2.h:78 > > #3 evutil_vsnprintf (buf=0x7fd48abfc468 ": Too many open files", > buflen=1000, format=<value optimized out>, ap=<value optimized out>) at > evutil.c:1577 > > #4 0x00007fd4969d7263 in evutil_snprintf (buf=<value optimized out>, > buflen=<value optimized out>, format=<value optimized out>) at evutil.c:1554 > > #5 0x00007fd4969d623d in _warn_helper (severity=2, errstr=0x7fd495dd0d70 > "Too many open files", fmt=<value optimized out>, ap=<value optimized out>) > at log.c:183 > > #6 0x00007fd4969d6634 in event_sock_warn (sock=<value optimized out>, > fmt=0x7fd4969ee50b "Error from accept() call") at log.c:124 > > #7 0x00007fd4969d365c in listener_read_cb (fd=41, what=<value optimized > out>, p=0x7fd460004dc0) at listener.c:441 > > #8 0x00007fd4969c8f8c in event_process_active_single_queue (base=0x1046480, > flags=0) at event.c:1350 > > #9 event_process_active (base=0x1046480, flags=0) at event.c:1420 > > #10 event_base_loop (base=0x1046480, flags=0) at event.c:1621 > > > Since it was a production environment there was little more I could in do. In > the end GDB crashed the process (or the OS?). > > > Any ideas? > > > 10x, > > Tomer. > "#5 0x00007fd4969d623d in _warn_helper (severity=2, errstr=0x7fd495dd0d70 "Too many open files", fmt=<value optimized out>, ap=<value optimized out>) at log.c:183 " You either you're not closing up sockets when you're done with them, or you need to increase your max open fd limit. (ulimit -n). *********************************************************************** To unsubscribe, send an e-mail to [email protected] with unsubscribe libevent-users in the body.
