Ranier, we're still seeing these crashes of rsyslogd even in v8.7.0... I will try to run this under Valgrind, but here is another gdb trace.
RW ________________________________________ From: [email protected] <[email protected]> on behalf of Rainer Gerhards <[email protected]> Sent: Wednesday, October 29, 2014 12:40 PM To: rsyslog-users Subject: Re: [rsyslog] rsyslogd crashes (segv, signal 11) intermittantly on strmSetWCntr (stream.c:1918) oops again, I meant $ valgrind rsyslogd -n ...options.... The -n makes it run in the foreground. You can ctl-c out of it when done. Rainer 2014-10-29 17:39 GMT+01:00 Rainer Gerhards <[email protected]>: > sorry, I thought I had answered that. > > You can simply install the symbols via > > $ yum install rsyslog-debuginfo > > from our repository. > > Then all you need to do is stop rsyslog and interactively call it via > > $ valgrind rsyslogd ...options.... > > That should do the trick. > > Thanks, > Rainer > > 2014-10-27 20:01 GMT+01:00 Raymond Wu <[email protected]>: > >> This will require rebuilding the program with -g, correct? >> We can try this but probably can't do it today. >> Would increasing the polling interval help? >> >> RW >> ________________________________________ >> From: [email protected] < >> [email protected]> on behalf of Rainer Gerhards < >> [email protected]> >> Sent: Monday, October 27, 2014 2:35 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslogd crashes (segv, signal 11) intermittantly >> on strmSetWCntr (stream.c:1918) >> >> Could you try under valgrind? That would be very helpful... >> >> Sent from phone, thus brief. >> Am 27.10.2014 19:11 schrieb "Raymond Wu" <[email protected]>: >> >> > OK. I ran gdb on a few more cores produced from similar crashes... >> > Seems to be the same every time (stream.c:1918). >> > ________________________________________ >> > From: [email protected] < >> [email protected]> >> > on behalf of Rainer Gerhards <[email protected]> >> > Sent: Monday, October 27, 2014 1:26 PM >> > To: rsyslog-users >> > Subject: Re: [rsyslog] rsyslogd crashes (segv, signal 11) intermittantly >> > on strmSetWCntr (stream.c:1918) >> > >> > 2014-10-27 17:33 GMT+01:00 Raymond Wu <[email protected]>: >> > >> > > Thanks for the quick reply, Rainer. >> > > Is there a $ version of mode=polling directive? >> > > We're using the old style configs... >> > > >> > > >> > ah, sorry, I overlooked that. so this can't be the issue here. Old-style >> > always uses polling. >> > >> > Rainer >> > >> > > $ModLoad imfile >> > > $InputFileName /var/log/tsq/beakley.log >> > > $InputFileTag beakley.log_app: >> > > $InputFileStateFile stat-beakley.log >> > > $InputFileSeverity error >> > > $InputFileFacility local7 >> > > $InputRunFileMonitor >> > > $InputFilePollingInterval 1 >> > > >> > > Ray >> > > ________________________________________ >> > > From: [email protected] < >> > [email protected]> >> > > on behalf of Rainer Gerhards <[email protected]> >> > > Sent: Monday, October 27, 2014 11:57 AM >> > > To: rsyslog-users >> > > Subject: Re: [rsyslog] rsyslogd crashes (segv, signal 11) >> intermittantly >> > > on strmSetWCntr (stream.c:1918) >> > > >> > > This strongly smells like being caused by this bug: >> > > >> > > https://github.com/rsyslog/rsyslog/issues/135 >> > > >> > > Rainer >> > > >> > > 2014-10-27 16:53 GMT+01:00 Raymond Wu <[email protected]>: >> > > >> > > > >> > > > Hello, I'm running rsyslog-8.4.2 (Adiscon package) on CentOS 6.5. >> The >> > > > rsyslogd daemon crashes intermittently on a set of our systems.These >> > > hosts >> > > > run apps which write their own log files, which are fed to rsyslog >> via >> > > the >> > > > IMFILE feature. I've patched the OS (CentOS 6.5) on these hosts with >> > all >> > > > updates as of a few days ago. Often the daemon won't start, >> post-crash, >> > > > unless (1) it's run under strace, or (2) I remove (some of) the >> > > > /var/log/srvrfwd-*.queue.* files. Perhaps some bug parsing/flushing >> > queue >> > > > files. I can post the rsyslog configuration if necessary, but there >> > are a >> > > > crapload of small include .conf files (around 40, to IMFILE various >> > > > application logs) but nothing really fancy is being done. Here is >> the >> > > > package info: >> > > > >> > > > >> > > > $ rpm -qvi rsyslog >> > > > >> > > > Name : rsyslog Relocations: (not >> > relocatable) >> > > > Version : 8.4.2 Vendor: (none) >> > > > Release : 1.el6 Build Date: Thu 02 Oct >> 2014 >> > > > 03:25:33 AM EDT >> > > > Install Date: Mon 20 Oct 2014 04:59:59 PM EDT Build Host: >> > > > vmrpm.adiscon.com >> > > > Group : System Environment/Daemons Source RPM: >> > > > rsyslog-8.4.2-1.el6.src.rpm >> > > > Size : 2186241 License: (GPLv3+ and >> ASL >> > > > 2.0) >> > > > Signature : RSA/SHA1, Thu 02 Oct 2014 03:28:29 AM EDT, Key ID >> > > > e0f233b3e00b8985 >> > > > URL : http://www.rsyslog.com/ >> > > > Summary : Enhanced system logging and kernel message trapping >> > daemon >> > > > Description : >> > > > Rsyslog is an enhanced, multi-threaded syslog daemon. It supports >> > MySQL, >> > > > syslog/TCP, RFC 3195, permitted sender lists, filtering on any >> message >> > > > part, >> > > > and fine grain output format control. It is compatible with stock >> > > sysklogd >> > > > and can be used as a drop-in replacement. Rsyslog is simple to set >> up, >> > > with >> > > > advanced features suitable for enterprise-class, >> encryption-protected >> > > > syslog >> > > > relay chains. >> > > > >> > > > >> > > > Attached is the suggested GDB dump from a core file that was >> produced >> > by >> > > > the crash. >> > > > >> > > > >> > > > Any ideas on how to fix this, are appreciated. Thanks! >> > > > >> > > > >> > > > Ray >> > > > >> > > > _______________________________________________ >> > > > rsyslog mailing list >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > http://www.rsyslog.com/professional-services/ >> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > myriad >> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> > > > DON'T LIKE THAT. >> > > > >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T LIKE THAT. >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T LIKE THAT. >> > > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > DON'T LIKE THAT. >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > DON'T LIKE THAT. >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[root@tsqweb01 /]# gdb /sbin/rsyslogd GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /sbin/rsyslogd...Reading symbols from /usr/lib/debug/sbin/rsyslogd.debug...done. done. (gdb) core /core.5278 [New Thread 5380] [New Thread 5278] [New Thread 5280] [New Thread 5285] [New Thread 5289] [New Thread 5293] [New Thread 5283] [New Thread 5282] [New Thread 5294] [New Thread 5287] [New Thread 5281] [New Thread 5295] [New Thread 5292] [New Thread 5288] [New Thread 5284] [New Thread 5286] [New Thread 5290] [New Thread 5383] Missing separate debuginfo for Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/a1/4ed96540aab1af58066fb2fd5fc9510745015c Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libz.so.1 Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /usr/lib64/libestr.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libestr.so.0 Reading symbols from /lib64/libjson-c.so.2...done. Loaded symbols for /lib64/libjson-c.so.2 Reading symbols from /lib64/libuuid.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libuuid.so.1 Reading symbols from /usr/lib64/liblogging-stdlog.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/liblogging-stdlog.so.0 Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libgcc_s.so.1 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/rsyslog/lmnet.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/lmnet.so.debug...done. done. Loaded symbols for /lib64/rsyslog/lmnet.so Reading symbols from /lib64/rsyslog/imuxsock.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/imuxsock.so.debug...done. done. Loaded symbols for /lib64/rsyslog/imuxsock.so Reading symbols from /lib64/rsyslog/imklog.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/imklog.so.debug...done. done. Loaded symbols for /lib64/rsyslog/imklog.so Reading symbols from /lib64/rsyslog/immark.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/immark.so.debug...done. done. Loaded symbols for /lib64/rsyslog/immark.so Reading symbols from /lib64/rsyslog/imudp.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/imudp.so.debug...done. done. Loaded symbols for /lib64/rsyslog/imudp.so Reading symbols from /lib64/rsyslog/imtcp.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/imtcp.so.debug...done. done. Loaded symbols for /lib64/rsyslog/imtcp.so Reading symbols from /lib64/rsyslog/lmnetstrms.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/lmnetstrms.so.debug...done. done. Loaded symbols for /lib64/rsyslog/lmnetstrms.so Reading symbols from /lib64/rsyslog/lmtcpsrv.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/lmtcpsrv.so.debug...done. done. Loaded symbols for /lib64/rsyslog/lmtcpsrv.so Reading symbols from /lib64/rsyslog/imfile.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/imfile.so.debug...done. done. Loaded symbols for /lib64/rsyslog/imfile.so Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libnss_files.so.2 Reading symbols from /lib64/rsyslog/lmtcpclt.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/lmtcpclt.so.debug...done. done. Loaded symbols for /lib64/rsyslog/lmtcpclt.so Reading symbols from /lib64/rsyslog/lmnsd_ptcp.so...Reading symbols from /usr/lib/debug/lib64/rsyslog/lmnsd_ptcp.so.debug...done. done. Loaded symbols for /lib64/rsyslog/lmnsd_ptcp.so Core was generated by `/sbin/rsyslogd -i /var/run/syslogd.pid'. Program terminated with signal 11, Segmentation fault. #0 strmSetWCntr (pThis=0x0, pWCnt=0x7f9976bfcc18) at stream.c:1922 1922 pThis->pUsrWCntr = pWCnt; Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.149.el6.x86_64 json-c-0.11-11.el6.x86_64 libestr-0.1.10-1.el6.x86_64 libgcc-4.4.7-11.el6.x86_64 liblogging-1.0.5-1.el6.x86_64 libuuid-2.17.2-12.18.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) info thread 18 Thread 0x7f99859fa700 (LWP 5383) 0x00007f999ea3d737 in unlink () from /lib64/libc.so.6 17 Thread 0x7f99871fd700 (LWP 5290) 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 16 Thread 0x7f9987fff700 (LWP 5286) 0x00007f999fb3a264 in __lll_lock_wait () from /lib64/libpthread.so.0 15 Thread 0x7f99957fb700 (LWP 5284) 0x00007f999ea49fd3 in epoll_wait () from /lib64/libc.so.6 14 Thread 0x7f999c710700 (LWP 5288) 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 13 Thread 0x7f9986dfc700 (LWP 5292) 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 12 Thread 0x7f99775fe700 (LWP 5295) 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 11 Thread 0x7f99975fe700 (LWP 5281) 0x00007f999fb3a75d in read () from /lib64/libpthread.so.0 10 Thread 0x7f999d111700 (LWP 5287) 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 9 Thread 0x7f9977fff700 (LWP 5294) 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 8 Thread 0x7f9996bfd700 (LWP 5282) 0x00007f999ea42453 in select () from /lib64/libc.so.6 7 Thread 0x7f99961fc700 (LWP 5283) 0x00007f999ea49fd3 in epoll_wait () from /lib64/libc.so.6 6 Thread 0x7f99863fb700 (LWP 5293) 0x00007f999fb3a264 in __lll_lock_wait () from /lib64/libpthread.so.0 5 Thread 0x7f99875fe700 (LWP 5289) 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 4 Thread 0x7f9994dfa700 (LWP 5285) 0x00007f999ea42453 in select () from /lib64/libc.so.6 3 Thread 0x7f9997fff700 (LWP 5280) 0x00007f999ea42453 in select () from /lib64/libc.so.6 2 Thread 0x7f99a016d740 (LWP 5278) 0x00007f999ea42453 in select () from /lib64/libc.so.6 * 1 Thread 0x7f9976bfd700 (LWP 5380) strmSetWCntr (pThis=0x0, pWCnt=0x7f9976bfcc18) at stream.c:1922 (gdb) thread apply all bt full Thread 18 (Thread 0x7f99859fa700 (LWP 5383)): #0 0x00007f999ea3d737 in unlink () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f99a01b4194 in strmCloseFile (pThis=0x7f99a1193cf0) at stream.c:446 currOffs = <value optimized out> iRet = RS_RET_OK #2 0x00007f99a01b46c3 in strmDestruct (ppThis=0x7f99a1191650) at stream.c:922 pThis = 0x7f99a1193cf0 i = <value optimized out> #3 0x00007f99a01b71f5 in qDestructDisk (pThis=0x7f99a11913d0) at queue.c:918 No locals. #4 0x00007f99a01bbf76 in queueSwitchToEmergencyMode (pThis=0x7f99a11913d0, pWti=<value optimized out>) at queue.c:711 No locals. #5 ConsumerReg (pThis=0x7f99a11913d0, pWti=<value optimized out>) at queue.c:1880 iCancelStateSave = <value optimized out> bNeedReLock = 0 iRet = RS_RET_FILE_NOT_FOUND #6 0x00007f99a01b64b2 in wtiWorker (pThis=0x7f99a1195170) at wti.c:334 __clframe = {__cancel_routine = 0x7f99a01b6280 <wtiWorkerCancelCleanup>, __cancel_arg = 0x7f99a1195170, __do_it = 1, __cancel_type = 0} pWtp = 0x7f99a1195020 pAction = <value optimized out> bInactivityTOOccured = 0 localRet = <value optimized out> terminateRet = RS_RET_TERMINATE_WHEN_IDLE wrkrInfo = <value optimized out> iCancelStateSave = <value optimized out> i = <value optimized out> j = <value optimized out> k = <value optimized out> #7 0x00007f99a01b6002 in wtpWorker (arg=0x7f99a1195170) at wtp.c:389 __clframe = {__cancel_routine = 0x7f99a01b60b0 <wtpWrkrExecCancelCleanup>, __cancel_arg = 0x7f99a1195170, __do_it = 1, __cancel_type = 0} pThis = 0x7f99a1195020 ---Type <return> to continue, or q <return> to quit--- sigSet = {__val = {1048576, 0 <repeats 15 times>}} pszDbgHdr = <value optimized out> thrdName = "rs:action 2 queue[DA]:R\000\000\000\000\000\000\000\000" #8 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #9 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 17 (Thread 0x7f99871fd700 (LWP 5290)): #0 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f999d935b5f in wrkr (myself=0x7f999db38cb8) at tcpsrv.c:638 me = 0x7f999db38cb8 #2 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 16 (Thread 0x7f9987fff700 (LWP 5286)): #0 0x00007f999fb3a264 in __lll_lock_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f999fb35508 in _L_lock_854 () from /lib64/libpthread.so.0 No symbol table info available. #2 0x00007f999fb353d7 in pthread_mutex_lock () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f99a01bb07e in qqueueEnqMsg (pThis=0x7f99a1125b20, flowCtlType=eFLOWCTL_NO_DELAY, pMsg=<value optimized out>) at queue.c:2855 iRet = RS_RET_OK iCancelStateSave = <value optimized out> #4 0x00007f99a01c4a1c in doSubmitToActionQ (pAction=0x7f99a11257a0, pWti=0x7f99a11a43e0, pMsg=<value optimized out>) at ../action.c:1403 ttNow = {timeType = -32 '\340', month = 67 'C', day = 26 '\032', hour = -95 '\241', minute = -103 '\231', second = 127 '\177', secfracPrecision = 0 '\000', OffsetMinute = 0 '\000', OffsetHour = 64 '@', OffsetMode = -125 '\203', year = 30728, secfrac = 32665} iRet = RS_RET_OK #5 0x00007f99a01bca3d in execAct (root=<value optimized out>, pMsg=0x7f9978088680, pWti=0x7f99a11a43e0) at ruleset.c:197 ---Type <return> to continue, or q <return> to quit--- iRet = RS_RET_OK #6 scriptExec (root=<value optimized out>, pMsg=0x7f9978088680, pWti=0x7f99a11a43e0) at ruleset.c:449 stmt = 0x7f99a1107d70 iRet = RS_RET_OK #7 0x00007f99a01bd0bc in processBatch (pBatch=0x7f99a11a4410, pWti=0x7f99a11a43e0) at ruleset.c:503 i = <value optimized out> pMsg = 0x7f9978088680 pRuleset = <value optimized out> #8 0x00007f99a01cc1ec in msgConsumer (notNeeded=<value optimized out>, pBatch=0x7f99a11a4410, pWti=0x7f99a11a43e0) at rsyslogd.c:576 i = <value optimized out> #9 0x00007f99a01bbf03 in ConsumerReg (pThis=0x7f99a11a3ee0, pWti=0x7f99a11a43e0) at queue.c:1897 iCancelStateSave = <value optimized out> bNeedReLock = 1 iRet = RS_RET_OK #10 0x00007f99a01b64b2 in wtiWorker (pThis=0x7f99a11a43e0) at wti.c:334 __clframe = {__cancel_routine = 0x7f99a01b6280 <wtiWorkerCancelCleanup>, __cancel_arg = 0x7f99a11a43e0, __do_it = 1, __cancel_type = 0} pWtp = 0x7f99a11a4290 pAction = <value optimized out> bInactivityTOOccured = 0 localRet = <value optimized out> terminateRet = RS_RET_OK wrkrInfo = <value optimized out> iCancelStateSave = <value optimized out> i = <value optimized out> j = <value optimized out> k = <value optimized out> #11 0x00007f99a01b6002 in wtpWorker (arg=0x7f99a11a43e0) at wtp.c:389 __clframe = {__cancel_routine = 0x7f99a01b60b0 <wtpWrkrExecCancelCleanup>, __cancel_arg = 0x7f99a11a43e0, __do_it = 1, __cancel_type = 0} pThis = 0x7f99a11a4290 sigSet = {__val = {1048576, 0 <repeats 15 times>}} pszDbgHdr = <value optimized out> thrdName = "rs:main Q:Reg", '\000' <repeats 18 times> #12 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 ---Type <return> to continue, or q <return> to quit--- No symbol table info available. #13 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 15 (Thread 0x7f99957fb700 (LWP 5284)): #0 0x00007f999ea49fd3 in epoll_wait () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f999d1151e6 in Wait (pNsdpoll=0x7f99800024e0, timeout=-1, numEntries=0x7f99957fa92c, workset=0x7f99957fa110) at nsdpoll_ptcp.c:242 pThis = 0x7f99800024e0 pOurEvt = <value optimized out> event = {{events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}} <repeats 50 times>, {events = 0, data = {ptr = 0x2800000000, fd = 0, u32 = 0, u64 = 171798691840}}, {events = 48, data = {ptr = 0x7f99957fa080, fd = -1786797952, u32 = 2508169344, u64 = 140297614893184}}, { events = 2508169152, data = {ptr = 0x7f99, fd = 32665, u32 = 32665, u64 = 32665}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, { events = 2683732694, data = {ptr = 0x7f99, fd = 32665, u32 = 32665, u64 = 32665}}, {events = 0, data = {ptr = 0x7f99957f9df0, fd = -1786798608, u32 = 2508168688, u64 = 140297614892528}}, {events = 0, data = {ptr = 0x957f9df000000000, fd = 0, u32 = 0, u64 = 10772502487811227648}}, { events = 32665, data = {ptr = 0xc, fd = 12, u32 = 12, u64 = 12}}, {events = 2685864896, data = {ptr = 0x9fd7b9dd00007f99, fd = 32665, u32 = 32665, u64 = 11517878930862276505}}, {events = 0, data = {ptr = 0x7f999ff68c22, fd = -1611232222, u32 = 2683735074, u64 = 140297790458914}}, { events = 0, data = {ptr = 0x27f5ee700000000, fd = 0, u32 = 0, u64 = 179966856348565504}}, {events = 0, data = {ptr = 0x1d, fd = 29, u32 = 29, u64 = 29}}, {events = 2683695984, data = {ptr = 0x7f99, fd = 32665, u32 = 32665, u64 = 32665}}, {events = 0, data = {ptr = 0x7f99957f9f70, fd = -1786798224, u32 = 2508169072, u64 = 140297614892912}}, {events = 3, data = {ptr = 0x7f99, fd = 32665, u32 = 32665, u64 = 32665}}, { events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 80, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = { ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x300000000, fd = 0, u32 = 0, u64 = 12884901888}}, {events = 48, data = { ptr = 0x7c0000005b, fd = 91, u32 = 91, u64 = 532575944795}}, {events = 119, data = {ptr = 0x10000006e, fd = 110, u32 = 110, u64 = 4294967406}}, { events = 0, data = {ptr = 0x1, fd = 1, u32 = 1, u64 = 1}}, {events = 133856, data = {ptr = 0x8000252000000000, fd = 0, u32 = 0, u64 = 9223412856223956992}}, {events = 32665, data = {ptr = 0x28, fd = 40, u32 = 40, u64 = 40}}, {events = 2147483680, data = { ptr = 0x9ecf0e8000007f99, fd = 32665, u32 = 32665, u64 = 11443381121090355097}}, {events = 32665, data = {ptr = 0x7f999e9db346, fd = -1633832122, u32 = 2661135174, u64 = 140297767859014}}, {events = 2508169360, data = {ptr = 0x100007f99, fd = 32665, u32 = 32665, u64 = 4294999961}}, {events = 0, data = {ptr = 0x7f99a10fdf50, fd = -1592795312, u32 = 2702171984, u64 = 140297808895824}}, { events = 2702233232, data = {ptr = 0x800024e000007f99, fd = 32665, u32 = 32665, u64 = 9223412581346082713}}, {events = 32665, data = { ptr = 0x7f99a04168a0, fd = -1606326112, u32 = 2688641184, u64 = 140297795365024}}, {events = 1, data = {ptr = 0x9d11548c00000000, fd = 0, u32 = 0, u64 = 11317920298830921728}}, {events = 32665, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x7f9980002500, fd = -2147474176, u32 = 2147493120, u64 = 140297254216960}}, {events = 1, data = { ptr = 0xa017db4000000004, fd = 4, u32 = 4, u64 = 11535930038457204740}}, {events = 32665, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, { ---Type <return> to continue, or q <return> to quit--- events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = { ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x7f99957f9f70, fd = -1786798224, u32 = 2508169072, u64 = 140297614892912}}, { events = 0, data = {ptr = 0xffffffff00000000, fd = 0, u32 = 0, u64 = 18446744069414584320}}, {events = 32665, data = {ptr = 0x100000000, fd = 0, u32 = 0, u64 = 4294967296}}, {events = 0, data = {ptr = 0x9ecee44000000000, fd = 0, u32 = 0, u64 = 11443334666724048896}}, {events = 32665, data = {ptr = 0x3, fd = 3, u32 = 3, u64 = 3}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x50, fd = 80, u32 = 80, u64 = 80}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x2, fd = 2, u32 = 2, u64 = 2}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x40, fd = 64, u32 = 64, u64 = 64}}, { events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 2, data = { ptr = 0x5b00000030, fd = 48, u32 = 48, u64 = 390842023984}}, {events = 124, data = {ptr = 0x6e00000077, fd = 119, u32 = 119, u64 = 472446402679}}, {events = 5, data = {ptr = 0x100007f99, fd = 32665, u32 = 32665, u64 = 4294999961}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 112, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x500000000, fd = 0, u32 = 0, u64 = 21474836480}}, {events = 49, data = {ptr = 0x7c0000005b, fd = 91, u32 = 91, u64 = 532575944795}}, {events = 119, data = {ptr = 0x957fa0380000006e, fd = 110, u32 = 110, u64 = 10772504996072128622}}, { events = 32665, data = {ptr = 0x7f9980002420, fd = -2147474400, u32 = 2147492896, u64 = 140297254216736}}, {events = 133936, data = { ptr = 0x800024d000000000, fd = 0, u32 = 0, u64 = 9223412512626573312}}, {events = 32665, data = {ptr = 0x48, fd = 72, u32 = 72, u64 = 72}}, { events = 2147483680, data = {ptr = 0x9ecf0e8000007f99, fd = 32665, u32 = 32665, u64 = 11443381121090355097}}, {events = 32665, data = { ptr = 0x7f999e9db346, fd = -1633832122, u32 = 2661135174, u64 = 140297767859014}}, {events = 2645800041, data = {ptr = 0x8000242000007f99, fd = 32665, u32 = 32665, u64 = 9223411756712361881}}, {events = 32665, data = {ptr = 0x7f99800024e0, fd = -2147474208, u32 = 2147493088, u64 = 140297254216928}}, {events = 2635159115, data = {ptr = 0x957fad3000007f99, fd = 32665, u32 = 32665, u64 = 10772519255363583897}}, { events = 32665, data = {ptr = 0x7f9980002420, fd = -2147474400, u32 = 2147492896, u64 = 140297254216736}}, {events = 2147493088, data = { ptr = 0x9d11571800007f99, fd = 32665, u32 = 32665, u64 = 11317923099149631385}}, {events = 32665, data = {ptr = 0x7f99957fad30, fd = -1786794704, u32 = 2508172592, u64 = 140297614896432}}, {events = 2147492880, data = {ptr = 0x957fa09000007f99, fd = 32665, u32 = 32665, u64 = 10772505374029283225}}, {events = 32665, data = {ptr = 0x7f999db3b282, fd = -1649167742, u32 = 2645799554, u64 = 140297752523394}}, { events = 2664369792, data = {ptr = 0x9e9db46200007f99, fd = 32665, u32 = 32665, u64 = 11429489762336014233}}, {events = 32665, data = { ptr = 0x6c6f7064736e6d6c, fd = 1936616812, u32 = 1936616812, u64 = 7813587455246757228}}, {events = 1953521516, data = { ptr = 0xa10fd2d000007063, fd = 28771, u32 = 28771, u64 = 11605726555552116835}}, {events = 32665, data = {ptr = 0xacfa5503b8746c8c, fd = -1200329588, u32 = 3094637708, u64 = 12464368393169169548}}} nfds = <value optimized out> i = <value optimized out> iRet = RS_RET_OK #2 0x00007f999d93581d in Run (pThis=0x7f99a10fd2d0) at tcpsrv.c:895 iRet = RS_RET_OK i = <value optimized out> ---Type <return> to continue, or q <return> to quit--- workset = {{id = 0, pUsr = 0x0} <repeats 128 times>} numEntries = 128 pPoll = 0x7f9980002410 localRet = <value optimized out> #3 0x00007f99a01c54f3 in thrdStarter (arg=0x7f99a11a68f0) at ../threads.c:212 iRet = RS_RET_OK thrdName = "in:imtcp", '\000' <repeats 23 times> sigSet = {__val = {1048576, 0 <repeats 15 times>}} #4 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 14 (Thread 0x7f999c710700 (LWP 5288)): #0 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f999d935b5f in wrkr (myself=0x7f999db38be8) at tcpsrv.c:638 me = 0x7f999db38be8 #2 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 13 (Thread 0x7f9986dfc700 (LWP 5292)): #0 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f99a01b6525 in doIdleProcessing (pThis=0x7f99a1114040) at wti.c:266 t = {tv_sec = 0, tv_nsec = 0} #2 wtiWorker (pThis=0x7f99a1114040) at wti.c:344 __clframe = {__cancel_routine = 0x7f99a01b6280 <wtiWorkerCancelCleanup>, __cancel_arg = 0x7f99a1114040, __do_it = 1, __cancel_type = 0} pWtp = 0x7f99a1113ef0 pAction = <value optimized out> bInactivityTOOccured = 0 ---Type <return> to continue, or q <return> to quit--- localRet = <value optimized out> terminateRet = <value optimized out> wrkrInfo = <value optimized out> iCancelStateSave = <value optimized out> i = <value optimized out> j = <value optimized out> k = <value optimized out> #3 0x00007f99a01b6002 in wtpWorker (arg=0x7f99a1114040) at wtp.c:389 __clframe = {__cancel_routine = 0x7f99a01b60b0 <wtpWrkrExecCancelCleanup>, __cancel_arg = 0x7f99a1114040, __do_it = 1, __cancel_type = 0} pThis = 0x7f99a1113ef0 sigSet = {__val = {1048576, 0 <repeats 15 times>}} pszDbgHdr = <value optimized out> thrdName = "rs:action 1 queue:Reg\000\000\000\000\000\000\000\000\000\000" #4 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 12 (Thread 0x7f99775fe700 (LWP 5295)): #0 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f99a01b6525 in doIdleProcessing (pThis=0x7f99a119ebe0) at wti.c:266 t = {tv_sec = 0, tv_nsec = 0} #2 wtiWorker (pThis=0x7f99a119ebe0) at wti.c:344 __clframe = {__cancel_routine = 0x7f99a01b6280 <wtiWorkerCancelCleanup>, __cancel_arg = 0x7f99a119ebe0, __do_it = 1, __cancel_type = 0} pWtp = 0x7f99a119ea90 pAction = <value optimized out> bInactivityTOOccured = 0 localRet = <value optimized out> terminateRet = <value optimized out> wrkrInfo = <value optimized out> iCancelStateSave = <value optimized out> i = <value optimized out> ---Type <return> to continue, or q <return> to quit--- j = <value optimized out> k = <value optimized out> #3 0x00007f99a01b6002 in wtpWorker (arg=0x7f99a119ebe0) at wtp.c:389 __clframe = {__cancel_routine = 0x7f99a01b60b0 <wtpWrkrExecCancelCleanup>, __cancel_arg = 0x7f99a119ebe0, __do_it = 1, __cancel_type = 0} pThis = 0x7f99a119ea90 sigSet = {__val = {1048576, 0 <repeats 15 times>}} pszDbgHdr = <value optimized out> thrdName = "rs:action 4 queue:Reg\000\000\000\000\000\000\000\000\000\000" #4 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 11 (Thread 0x7f99975fe700 (LWP 5281)): #0 0x00007f999fb3a75d in read () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f999e3509da in read (pModConf=0x7f99a1100ee0) at /usr/include/bits/unistd.h:45 No locals. #2 readklog (pModConf=0x7f99a1100ee0) at bsd.c:253 p = <value optimized out> q = <value optimized out> len = 0 i = <value optimized out> iMaxLine = 8096 bufRcv = "<6>rs:action 2 que[5380]: segfault at 50 ip 00007f99a01b215c sp 00007f9976bfcbf8 error 6 in rsyslogd[7f99a0181000+8c000]\n", '\000' <repeats 130951 times> errmsg = '\000' <repeats 1888 times>, "f\000\000\000\000\000\000\000^\000\000\000\000\000\000\000\002", '\000' <repeats 23 times>, "@", '\000' <repeats 31 times>, "\002\000\000\000\060\000\000\000[\000\000\000|\000\000\000w\000\000\000n\000\000\000`nA\240\231\177\000\000\000\000\000\000\000\000\000\000wh\263\237\231\177\000\000 \000\000\210\231\177\000\000\300\351_\227\231\177\000\000\000\000\000\000\000\000\000\000\003\000\000\000\000\000\000\000\261\266\235\236\231\177\000" pRcv = 0x7f99975dd530 "<6>rs:action 2 que[5380]: segfault at 50 ip 00007f99a01b215c sp 00007f9976bfcbf8 error 6 in rsyslogd[7f99a0181000+8c000]\n" #3 0x00007f999e350b59 in klogLogKMsg (pModConf=<value optimized out>) at bsd.c:311 No locals. ---Type <return> to continue, or q <return> to quit--- #4 0x00007f999e3500b3 in runInput (pThrd=0x7f99a11a62f0) at imklog.c:283 iRet = RS_RET_OK #5 0x00007f99a01c54f3 in thrdStarter (arg=0x7f99a11a62f0) at ../threads.c:212 iRet = RS_RET_OK thrdName = "in:imklog", '\000' <repeats 22 times> sigSet = {__val = {1048576, 0 <repeats 15 times>}} #6 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #7 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 10 (Thread 0x7f999d111700 (LWP 5287)): #0 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f999d935b5f in wrkr (myself=0x7f999db38b80) at tcpsrv.c:638 me = 0x7f999db38b80 #2 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 9 (Thread 0x7f9977fff700 (LWP 5294)): #0 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f99a01b6525 in doIdleProcessing (pThis=0x7f99a1195d80) at wti.c:266 t = {tv_sec = 0, tv_nsec = 0} #2 wtiWorker (pThis=0x7f99a1195d80) at wti.c:344 __clframe = {__cancel_routine = 0x7f99a01b6280 <wtiWorkerCancelCleanup>, __cancel_arg = 0x7f99a1195d80, __do_it = 1, __cancel_type = 0} pWtp = 0x7f99a1195c30 pAction = <value optimized out> bInactivityTOOccured = 0 localRet = <value optimized out> terminateRet = <value optimized out> ---Type <return> to continue, or q <return> to quit--- wrkrInfo = <value optimized out> iCancelStateSave = <value optimized out> i = <value optimized out> j = <value optimized out> k = <value optimized out> #3 0x00007f99a01b6002 in wtpWorker (arg=0x7f99a1195d80) at wtp.c:389 __clframe = {__cancel_routine = 0x7f99a01b60b0 <wtpWrkrExecCancelCleanup>, __cancel_arg = 0x7f99a1195d80, __do_it = 1, __cancel_type = 0} pThis = 0x7f99a1195c30 sigSet = {__val = {1048576, 0 <repeats 15 times>}} pszDbgHdr = <value optimized out> thrdName = "rs:action 3 queue:Reg\000\000\000\000\000\000\000\000\000\000" #4 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 8 (Thread 0x7f9996bfd700 (LWP 5282)): #0 0x00007f999ea42453 in select () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f99a01aa333 in srSleep (iSeconds=<value optimized out>, iuSeconds=<value optimized out>) at srutils.c:494 tvSelectTimeout = {tv_sec = 670, tv_usec = 59730} #2 0x00007f999e14c0b2 in runInput (pThrd=<value optimized out>) at immark.c:190 No locals. #3 0x00007f99a01c54f3 in thrdStarter (arg=0x7f99a11a64f0) at ../threads.c:212 iRet = RS_RET_OK thrdName = "in:immark", '\000' <repeats 22 times> sigSet = {__val = {1048576, 0 <repeats 15 times>}} #4 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 7 (Thread 0x7f99961fc700 (LWP 5283)): ---Type <return> to continue, or q <return> to quit--- #0 0x00007f999ea49fd3 in epoll_wait () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f999df4648a in rcvMainLoop (pWrkr=0x7f999e1496e0) at imudp.c:786 iRet = RS_RET_OK nfds = <value optimized out> efd = 9 i = <value optimized out> frominetPrev = {ss_family = 0, __ss_align = 0, __ss_padding = '\000' <repeats 111 times>} bIsPermitted = 0 udpEPollEvt = 0x7f99980026e0 currEvt = {{events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x7f99961fba60, fd = -1776305568, u32 = 2518661728, u64 = 140297625385568}}, { events = 2518661712, data = {ptr = 0x800007f99, fd = 32665, u32 = 32665, u64 = 34359771033}}, {events = 0, data = {ptr = 0x7f99961fbbd0, fd = -1776305200, u32 = 2518662096, u64 = 140297625385936}}, {events = 2650042706, data = {ptr = 0x9e9a516700007f99, fd = 32665, u32 = 32665, u64 = 11428536507229568921}}, {events = 32665, data = {ptr = 0x7f999df47153, fd = -1644924589, u32 = 2650042707, u64 = 140297756766547}}} errStr = '\000' <repeats 311 times>, "0", '\000' <repeats 40 times>, "(\000\000\000\060\000\000\000\260\274\037\226\231\177\000\000\360\273\037\226\231\177", '\000' <repeats 130 times>, "Ö\366\237\231\177\000\000\000\000\000\000\000\000\000\000\220\272\037\226\231\177\000\000\000\000\000\000\000\000\000\000\220\272\037\226\231\177\000\000\f\000\000\000\000\000\000\000\300\v\027\240\231\177\000\000\207\360\226|\000\000\000\000`\272\037\226\231\177\000\000\300\274\037\226\231\177\000\000P\273\037\226\231\177\000\000\001\000\000\000\000\000\000\000л\037\226\231\177\000\000\037\000\000\000\000\000\000\000Iq\364\235\231\177\000\000\300\274\037\226\231\177\000\000\020\f\246\236\231\177\000\000\001\200\255\373\000\000\000\000\300\274\037\226\231\177\000\000\300\274\037\226\231\177\000\000\300\274\037\226\231\177\000\000\300\274\037\226\231\177\000\000ɼ\037\226\231\177\000\000\337\274\037\226\231\177\000\000\300\274\037\226\231\177\000\000\337\274\037\226\231\177", '\000' <repeats 42 times>, "i\b\000\000\004", '\000' <repeats 27 times>... lstn = <value optimized out> nLstn = <value optimized out> #2 0x00007f999df46c33 in wrkr (myself=0x7f999e1496e0) at imudp.c:1175 pWrkr = 0x7f999e1496e0 thrdName = "imudp(w0)\000\000\000\000\000\000\000\340\226\024\236\231\177\000\000\000\000@\000\000\000\000" #3 0x00007f999df46d6a in runInput (pThrd=0x7f99a11a66f0) at imudp.c:1200 i = <value optimized out> wrkrThrdAttr = {__size = '\000' <repeats 17 times>, "\020", '\000' <repeats 16 times>, "@", '\000' <repeats 20 times>, __align = 0} #4 0x00007f99a01c54f3 in thrdStarter (arg=0x7f99a11a66f0) at ../threads.c:212 iRet = RS_RET_OK ---Type <return> to continue, or q <return> to quit--- thrdName = "in:imudp", '\000' <repeats 23 times> sigSet = {__val = {1048576, 0 <repeats 15 times>}} #5 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #6 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 6 (Thread 0x7f99863fb700 (LWP 5293)): #0 0x00007f999fb3a264 in __lll_lock_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f999fb35508 in _L_lock_854 () from /lib64/libpthread.so.0 No symbol table info available. #2 0x00007f999fb353d7 in pthread_mutex_lock () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f99a01bbeae in ConsumerReg (pThis=0x7f99a1125b20, pWti=<value optimized out>) at queue.c:1918 iCancelStateSave = 1 bNeedReLock = 1 iRet = RS_RET_OK #4 0x00007f99a01b64b2 in wtiWorker (pThis=0x7f99a1190930) at wti.c:334 __clframe = {__cancel_routine = 0x7f99a01b6280 <wtiWorkerCancelCleanup>, __cancel_arg = 0x7f99a1190930, __do_it = 1, __cancel_type = 0} pWtp = 0x7f99a11907e0 pAction = <value optimized out> bInactivityTOOccured = 0 localRet = <value optimized out> terminateRet = RS_RET_OK wrkrInfo = <value optimized out> iCancelStateSave = <value optimized out> i = <value optimized out> j = <value optimized out> k = <value optimized out> #5 0x00007f99a01b6002 in wtpWorker (arg=0x7f99a1190930) at wtp.c:389 __clframe = {__cancel_routine = 0x7f99a01b60b0 <wtpWrkrExecCancelCleanup>, __cancel_arg = 0x7f99a1190930, __do_it = 1, __cancel_type = 0} pThis = 0x7f99a11907e0 ---Type <return> to continue, or q <return> to quit--- sigSet = {__val = {1048576, 0 <repeats 15 times>}} pszDbgHdr = <value optimized out> thrdName = "rs:action 2 queue:Reg\000\000\000\000\000\000\000\000\000\000" #6 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #7 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 5 (Thread 0x7f99875fe700 (LWP 5289)): #0 0x00007f999fb375bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f999d935b5f in wrkr (myself=0x7f999db38c50) at tcpsrv.c:638 me = 0x7f999db38c50 #2 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 4 (Thread 0x7f9994dfa700 (LWP 5285)): #0 0x00007f999ea42453 in select () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f99a01aa333 in srSleep (iSeconds=<value optimized out>, iuSeconds=<value optimized out>) at srutils.c:494 tvSelectTimeout = {tv_sec = 9, tv_usec = 959218} #2 0x00007f999d72df26 in doPolling (pThrd=<value optimized out>) at imfile.c:1159 bHadFileData = 0 #3 runInput (pThrd=<value optimized out>) at imfile.c:1728 iRet = RS_RET_OK #4 0x00007f99a01c54f3 in thrdStarter (arg=0x7f99a1196d30) at ../threads.c:212 iRet = RS_RET_OK thrdName = "in:imfile", '\000' <repeats 22 times> sigSet = {__val = {1048576, 0 <repeats 15 times>}} #5 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. ---Type <return> to continue, or q <return> to quit--- #6 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 3 (Thread 0x7f9997fff700 (LWP 5280)): #0 0x00007f999ea42453 in select () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f999e558313 in runInput (pThrd=<value optimized out>) at imuxsock.c:1408 iRet = RS_RET_OK maxfds = <value optimized out> nfds = <value optimized out> i = <value optimized out> fd = <value optimized out> pReadfds = 0x7f99900008c0 #2 0x00007f99a01c54f3 in thrdStarter (arg=0x7f99a11a60f0) at ../threads.c:212 iRet = RS_RET_OK thrdName = "in:imuxsock", '\000' <repeats 20 times> sigSet = {__val = {1048576, 0 <repeats 15 times>}} #3 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. Thread 2 (Thread 0x7f99a016d740 (LWP 5278)): #0 0x00007f999ea42453 in select () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f99a01ce4ca in mainloop (argc=<value optimized out>, argv=<value optimized out>) at rsyslogd.c:1494 tvSelectTimeout = {tv_sec = 73, tv_usec = 159336} #2 main (argc=<value optimized out>, argv=<value optimized out>) at rsyslogd.c:1627 No locals. Thread 1 (Thread 0x7f9976bfd700 (LWP 5380)): #0 strmSetWCntr (pThis=0x0, pWCnt=0x7f9976bfcc18) at stream.c:1922 No locals. ---Type <return> to continue, or q <return> to quit--- #1 0x00007f99a01b7d78 in qAddDisk (pThis=0x7f99a11913d0, pMsg=0x7f9978298990) at queue.c:930 iRet = RS_RET_OK nWriteCount = <value optimized out> #2 0x00007f99a01b81e6 in qqueueAdd (pThis=0x7f99a11913d0, flowCtlType=<value optimized out>, pMsg=0x7f9978298990) at queue.c:1034 iRet = RS_RET_OK #3 doEnqSingleObj (pThis=0x7f99a11913d0, flowCtlType=<value optimized out>, pMsg=0x7f9978298990) at queue.c:2771 iRet = <value optimized out> err = <value optimized out> t = {tv_sec = 140295106723840, tv_nsec = 140297809514680} #4 0x00007f99a01bb08d in qqueueEnqMsg (pThis=0x7f99a11913d0, flowCtlType=eFLOWCTL_NO_DELAY, pMsg=<value optimized out>) at queue.c:2858 iRet = RS_RET_OK iCancelStateSave = <value optimized out> #5 0x00007f99a01bbd06 in ConsumerDA (pThis=0x7f99a1125b20, pWti=0x7f99a1190f20) at queue.c:1955 i = <value optimized out> iCancelStateSave = <value optimized out> bNeedReLock = 1 iRet = <value optimized out> #6 0x00007f99a01b64b2 in wtiWorker (pThis=0x7f99a1190f20) at wti.c:334 __clframe = {__cancel_routine = 0x7f99a01b6280 <wtiWorkerCancelCleanup>, __cancel_arg = 0x7f99a1190f20, __do_it = 1, __cancel_type = 0} pWtp = 0x7f99a1190dd0 pAction = <value optimized out> bInactivityTOOccured = 0 localRet = <value optimized out> terminateRet = RS_RET_OK wrkrInfo = <value optimized out> iCancelStateSave = <value optimized out> i = <value optimized out> j = <value optimized out> k = <value optimized out> #7 0x00007f99a01b6002 in wtpWorker (arg=0x7f99a1190f20) at wtp.c:389 __clframe = {__cancel_routine = 0x7f99a01b60b0 <wtpWrkrExecCancelCleanup>, __cancel_arg = 0x7f99a1190f20, __do_it = 1, __cancel_type = 0} pThis = 0x7f99a1190dd0 sigSet = {__val = {1048576, 0 <repeats 15 times>}} ---Type <return> to continue, or q <return> to quit--- pszDbgHdr = <value optimized out> thrdName = "rs:action 2 queue:DAwpo\000\000\000\000\000\000\000\000" #8 0x00007f999fb339d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #9 0x00007f999ea499dd in clone () from /lib64/libc.so.6 No symbol table info available. (gdb) q
_______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
