Thanks, I will try this at the next crash. ________________________________________ From: [email protected] <[email protected]> on behalf of Rainer Gerhards <[email protected]> Sent: Wednesday, October 29, 2014 12:40 PM To: rsyslog-users Subject: Re: [rsyslog] rsyslogd crashes (segv, signal 11) intermittantly on strmSetWCntr (stream.c:1918)
oops again, I meant $ valgrind rsyslogd -n ...options.... The -n makes it run in the foreground. You can ctl-c out of it when done. Rainer 2014-10-29 17:39 GMT+01:00 Rainer Gerhards <[email protected]>: > sorry, I thought I had answered that. > > You can simply install the symbols via > > $ yum install rsyslog-debuginfo > > from our repository. > > Then all you need to do is stop rsyslog and interactively call it via > > $ valgrind rsyslogd ...options.... > > That should do the trick. > > Thanks, > Rainer > > 2014-10-27 20:01 GMT+01:00 Raymond Wu <[email protected]>: > >> This will require rebuilding the program with -g, correct? >> We can try this but probably can't do it today. >> Would increasing the polling interval help? >> >> RW >> ________________________________________ >> From: [email protected] < >> [email protected]> on behalf of Rainer Gerhards < >> [email protected]> >> Sent: Monday, October 27, 2014 2:35 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslogd crashes (segv, signal 11) intermittantly >> on strmSetWCntr (stream.c:1918) >> >> Could you try under valgrind? That would be very helpful... >> >> Sent from phone, thus brief. >> Am 27.10.2014 19:11 schrieb "Raymond Wu" <[email protected]>: >> >> > OK. I ran gdb on a few more cores produced from similar crashes... >> > Seems to be the same every time (stream.c:1918). >> > ________________________________________ >> > From: [email protected] < >> [email protected]> >> > on behalf of Rainer Gerhards <[email protected]> >> > Sent: Monday, October 27, 2014 1:26 PM >> > To: rsyslog-users >> > Subject: Re: [rsyslog] rsyslogd crashes (segv, signal 11) intermittantly >> > on strmSetWCntr (stream.c:1918) >> > >> > 2014-10-27 17:33 GMT+01:00 Raymond Wu <[email protected]>: >> > >> > > Thanks for the quick reply, Rainer. >> > > Is there a $ version of mode=polling directive? >> > > We're using the old style configs... >> > > >> > > >> > ah, sorry, I overlooked that. so this can't be the issue here. Old-style >> > always uses polling. >> > >> > Rainer >> > >> > > $ModLoad imfile >> > > $InputFileName /var/log/tsq/beakley.log >> > > $InputFileTag beakley.log_app: >> > > $InputFileStateFile stat-beakley.log >> > > $InputFileSeverity error >> > > $InputFileFacility local7 >> > > $InputRunFileMonitor >> > > $InputFilePollingInterval 1 >> > > >> > > Ray >> > > ________________________________________ >> > > From: [email protected] < >> > [email protected]> >> > > on behalf of Rainer Gerhards <[email protected]> >> > > Sent: Monday, October 27, 2014 11:57 AM >> > > To: rsyslog-users >> > > Subject: Re: [rsyslog] rsyslogd crashes (segv, signal 11) >> intermittantly >> > > on strmSetWCntr (stream.c:1918) >> > > >> > > This strongly smells like being caused by this bug: >> > > >> > > https://github.com/rsyslog/rsyslog/issues/135 >> > > >> > > Rainer >> > > >> > > 2014-10-27 16:53 GMT+01:00 Raymond Wu <[email protected]>: >> > > >> > > > >> > > > Hello, I'm running rsyslog-8.4.2 (Adiscon package) on CentOS 6.5. >> The >> > > > rsyslogd daemon crashes intermittently on a set of our systems.These >> > > hosts >> > > > run apps which write their own log files, which are fed to rsyslog >> via >> > > the >> > > > IMFILE feature. I've patched the OS (CentOS 6.5) on these hosts with >> > all >> > > > updates as of a few days ago. Often the daemon won't start, >> post-crash, >> > > > unless (1) it's run under strace, or (2) I remove (some of) the >> > > > /var/log/srvrfwd-*.queue.* files. Perhaps some bug parsing/flushing >> > queue >> > > > files. I can post the rsyslog configuration if necessary, but there >> > are a >> > > > crapload of small include .conf files (around 40, to IMFILE various >> > > > application logs) but nothing really fancy is being done. Here is >> the >> > > > package info: >> > > > >> > > > >> > > > $ rpm -qvi rsyslog >> > > > >> > > > Name : rsyslog Relocations: (not >> > relocatable) >> > > > Version : 8.4.2 Vendor: (none) >> > > > Release : 1.el6 Build Date: Thu 02 Oct >> 2014 >> > > > 03:25:33 AM EDT >> > > > Install Date: Mon 20 Oct 2014 04:59:59 PM EDT Build Host: >> > > > vmrpm.adiscon.com >> > > > Group : System Environment/Daemons Source RPM: >> > > > rsyslog-8.4.2-1.el6.src.rpm >> > > > Size : 2186241 License: (GPLv3+ and >> ASL >> > > > 2.0) >> > > > Signature : RSA/SHA1, Thu 02 Oct 2014 03:28:29 AM EDT, Key ID >> > > > e0f233b3e00b8985 >> > > > URL : http://www.rsyslog.com/ >> > > > Summary : Enhanced system logging and kernel message trapping >> > daemon >> > > > Description : >> > > > Rsyslog is an enhanced, multi-threaded syslog daemon. It supports >> > MySQL, >> > > > syslog/TCP, RFC 3195, permitted sender lists, filtering on any >> message >> > > > part, >> > > > and fine grain output format control. It is compatible with stock >> > > sysklogd >> > > > and can be used as a drop-in replacement. Rsyslog is simple to set >> up, >> > > with >> > > > advanced features suitable for enterprise-class, >> encryption-protected >> > > > syslog >> > > > relay chains. >> > > > >> > > > >> > > > Attached is the suggested GDB dump from a core file that was >> produced >> > by >> > > > the crash. >> > > > >> > > > >> > > > Any ideas on how to fix this, are appreciated. Thanks! >> > > > >> > > > >> > > > Ray >> > > > >> > > > _______________________________________________ >> > > > rsyslog mailing list >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > http://www.rsyslog.com/professional-services/ >> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > myriad >> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> > > > DON'T LIKE THAT. >> > > > >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T LIKE THAT. >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T LIKE THAT. >> > > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > DON'T LIKE THAT. >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > DON'T LIKE THAT. >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
