Can you get me a debug log?
"James, Jason" <[email protected]> schrieb: Rainer. I tried enabled asynchronous and still had the same issue this morning. After further investigation I see the following error. Jun 19 05:03:47 servername rsyslogd: db error (1064): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '20120619050346', '20120619050346', 1, 'snmpd[1250]:')' at line 1 It looks as though it's not escaping properly? This thread then hangs and rsyslog shoots up to 100% cpu, it does continue to log to local files. This appears to be a bug? -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Rainer Gerhards Sent: Monday, June 18, 2012 10:34 AM To: rsyslog-users Subject: Re: [rsyslog] Upgrading from 4.2.0 to > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of James, Jason > Sent: Monday, June 18, 2012 5:29 PM > To: 'rsyslog-users' > Subject: Re: [rsyslog] Upgrading from 4.2.0 to > > Logging directly to a centralized MySQL DB over SSL. This setup is > working fine on 15+ servers, added two more on Friday and have had > rsyslogd stop logging on both multiple times. The commonality on both > of the new additions is that we have network monitoring sending snmpd > traffic (lots of UDP traffic). Strace did not show much, unfortunately. > They have both locked up twice since 6/16/2012 though. The daemon will > sit at 35%-65% cpu usage but appears to be doing nothing. > > > ################# > #### MODULES #### > ################# > > $ModLoad imuxsock # provides support for local system logging > $ModLoad imklog # provides kernel logging support (previously done by > rklogd) > #$ModLoad immark # provides --MARK-- message capability > $IMUXSockRateLimitInterval 0 > #$InputFileReadMode 1 #indent > > $KLogPath /proc/kmsg > > # provides UDP syslog reception > #$ModLoad imudp > #$UDPServerRun 514 > > # provides TCP syslog reception > #$ModLoad imtcp > #$InputTCPServerRun 514 > > > ########################### > #### GLOBAL DIRECTIVES #### > ########################### > > # > # Use traditional timestamp format. > # To enable high precision timestamps, comment out the following line. > # > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > # Filter duplicated messages > $RepeatedMsgReduction on > > # > # Set the default permissions for all log files. > # > $FileOwner syslog > $FileGroup adm > $FileCreateMode 0640 > $DirCreateMode 0755 > $Umask 0022 > $PrivDropToUser syslog > $PrivDropToGroup syslog > > # > # Include all config files in /etc/rsyslog.d/ > # > $IncludeConfig /etc/rsyslog.d/*.conf > > > mysql.conf > > $ModLoad ommysql > $OmMySQLConfigFile /etc/mysql/my.cnf > *.* :ommysql:server,dbname,dbusername,password > $WorkDirectory /var/rsyslog/work # default location for work > (spool) $files$ActionQueueType LinkedList # use asynchronous > processing > $ActionQueueFileName srvrfwd # set file name, also enables disk mode > $ActionResumeRetryCount -1 # infinite retries on insert failure > $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts > down. > I guess this is not the complete config, but if it is, you run ommysql in direct mode. That means mysql will be written sync, and if the traffic is too slow for mysql, it will delay processing in the main queue. When the main queue hits the default size of 10,000 msgs, inputs (like /dev/log) will block. I *guess* this is your problem. If it doesn't sound so, we can go through debug logs and see what exactly happens. For the async mysql setup, see: http://www.rsyslog.com/doc/rsyslog_high_database_rate.html Rainer > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Rainer Gerhards > Sent: Monday, June 18, 2012 10:13 AM > To: rsyslog-users > Subject: Re: [rsyslog] Upgrading from 4.2.0 to > > > -----Original Message----- > > From: [email protected] [mailto:rsyslog- > > [email protected]] On Behalf Of Roy Smith > > Sent: Monday, June 18, 2012 4:04 PM > > To: [email protected] > > Subject: [rsyslog] Upgrading from 4.2.0 to > > > > We're running rsyslog 4.2.0 on Ubuntu 10.04. Recently, as we've been > > ramping up traffic, we've started experiencing frequent cases of > > rsyslogd getting wedged. Messages stop getting forwarded to our > > central logging host, and applications writing to /dev/log start to > > block in sendto() calls. Restarting rsyslogd gets things going again > > (for a while). We're currently processing about 10 million logged > > lines per day per application server. > > > > It looks like the latest and greatest is 6.2.2. We're thinking of > > installing that to see if it solves our problems. Are there any > > backwards compatibility issues we should be aware of between 6.2.2 > and > > 4.2.0? > > There should be none, but it probably is a good idea to go through the > relevant compatibility docs: > > http://www.rsyslog.com/doc/v5compatibility.html > http://www.rsyslog.com/doc/v6compatibility.html > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards
