Hmmmm... the thing is, my $MainMsgQueueSize is only 5, and I had mysql turned off for 20 minutes... And my message flow rate is up around 1 per second. So I would have expected some queuing to happen. Is there anything else I should look at or try?
On Mon, 12 Sep 2011 11:13:28 +0200 Rainer Gerhards <[email protected]> wrote: > Queue files are only created if necessary. The debug log does not contain any > such situation. > > Rainer > > > -----Original Message----- > > From: [email protected] [mailto:rsyslog- > > [email protected]] On Behalf Of Steve Chupack > > Sent: Sunday, September 11, 2011 11:11 PM > > To: [email protected] > > Subject: Re: [rsyslog] Unable to implement on-disk queuing > > > > Ooops... previous message's attachment was the config, not the debug > > output. Debug attached. > > > > > > > > On Sun, 11 Sep 2011 16:50:57 -0400 > > Steve Chupack <[email protected]> wrote: > > > > > Hi, > > > > > > I have rsyslogd up and running and logging to mysql with loganalyzer > > as a front end. Very cool. > > > > > > However, I can't seem to get on-disk queuing working, which would be > > nice if mysql goes down or a table is locked because I'm purging > > records during a nightly maintenance cron. > > > > > > I never see any queue files created. I dropped the size of the memory > > queue to something ridiculously small, but still no queue files even > > after 20 minutes. > > > > > > I followed the instructions given here: > > http://www.rsyslog.com/doc/rsyslog_high_database_rate.html along with a > > couple of other tutorials on rsyslogd and queuing. > > > > > > I'm quite sure I'm just completely missing something obvious. Details > > below... And thanks to anyone who might be able to point out what I'm > > doing wrong. > > > > > > -Steve > > > > > > > > > > > > MY CONFIG, please, no flames re the horrendously complex rules ;-) > > > > > > # if you experience problems, check > > > # http://www.rsyslog.com/troubleshoot for assistance > > > > > > # rsyslog v3: load input modules > > > # If you do not load inputs, nothing happens! > > > # You may need to set the module load path if modules are not found. > > > > > > $ModLoad immark # provides --MARK-- message capability > > > $ModLoad imuxsock # provides support for local system logging (e.g. > > via logger command) > > > $ModLoad imklog # kernel logging (formerly provided by rklogd) > > > $ModLoad ommysql > > > > > > # SC 2010.11.11: configure disk caching in case mysql is unavailable > > > > > > $MainMsgQueueSize 5 > > > > > > $WorkDirectory /var/log/rsyslogq # default location for work (spool) > > files > > > > > > $ActionQueueType LinkedList # use asynchronous processing > > > $ActionQueueFileName dbq # set file name, also enables disk mode > > > $ActionResumeRetryCount -1 # infinite retries on insert failure > > > > > > # send snmpd INFO messages to the dustbin > > > > > > if $syslogseverity-text == 'info' and $syslogtag contains 'snmpd' > > then ~ > > > > > > $template vtfw,"insert into vtfw (Message, Facility, FromHost, > > Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values > > ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, > > '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, > > '%syslogtag%')",SQL > > > > > > if ($fromhost-ip contains '10.128.255') then > > :ommysql:localhost,Syslog,root,mysqldb44;vtfw > > > & ~ > > > #if ($fromhost-ip contains '10.128.255') then ~ > > > > > > $template vt1hs1_switches,"insert into vt1hs1_switches (Message, > > Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, > > InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', > > %syslogpriority%, '%timereported:::date-mysql%', > > '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL > > > > > > if $fromhost-ip contains '10.128.0' or $hostname contains_i 'core01' > > or $hostname contains_i 'core02' then > > :ommysql:localhost,Syslog,root,mysqldb44;vt1hs1_switches > > > #if ($fromhost-ip contains '10.128.0' or $source=='vt1hs1-dc216- > > core01' or $source=='vt1hs1-dc216-core02') then /var/log/switches > > > & ~ > > > #if $fromhost-ip contains '10.128.0' or $hostname contains_i 'core01' > > or $hostname contains_i 'core02' then ~ > > > > > > $template vt1hs1_wifi,"insert into vt1hs1_wifi (Message, Facility, > > FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, > > SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', > > %syslogpriority%, '%timereported:::date-mysql%', > > '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL > > > > > > if $fromhost-ip contains '10.128.244' then > > :ommysql:localhost,Syslog,root,mysqldb44;vt1hs1_wifi > > > & ~ > > > #if $fromhost-ip contains '10.128.244' then ~ > > > > > > $template vt1hs1_vsphere,"insert into vt1hs1_vsphere (Message, > > Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, > > InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', > > %syslogpriority%, '%timereported:::date-mysql%', > > '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL > > > > > > if ($fromhost-ip == '10.128.24.40' or $fromhost-ip == '10.128.24.44' > > or $fromhost-ip == '10.128.24.48' or $fromhost-ip == '10.128.24.52' or > > $fromhost-ip == '10.128.24.72' or $fromhost-ip == '10.128.24.76' or > > $fromhost-ip == '10.128.24.80' or $fromhost-ip == '10.128.24.84') and > > ($syslogseverity-text == 'warning' or $syslogseverity-text == 'warn' or > > $syslogseverity-text == 'err' or $syslogseverity-text == 'error' or > > $syslogseverity-text == 'crit' or $syslogseverity-text == 'alert' or > > $syslogseverity-text == 'emergency' or $syslogseverity-text == 'panic') > > then :ommysql:localhost,Syslog,root,mysqldb44;vt1hs1_vsphere > > > & ~ > > > > > > if ($fromhost-ip == '10.128.24.40' or $fromhost-ip == '10.128.24.44' > > or $fromhost-ip == '10.128.24.48' or $fromhost-ip == '10.128.24.52' or > > $fromhost-ip == '10.128.24.72' or $fromhost-ip == '10.128.24.76' or > > $fromhost-ip == '10.128.24.80' or $fromhost-ip == '10.128.24.84') and > > ($syslogseverity-text == 'notice') then ~ > > > > > > > > > #*.* :ommysql:localhost,Syslog,root,mysqldb44 > > > *.notice :ommysql:localhost,Syslog,root,mysqldb44 > > > > > > # Log all kernel messages to the console. > > > # Logging much else clutters up the screen. > > > #kern.* /dev/console > > > > > > # Log anything (except mail) of level info or higher. > > > # Don't log private authentication messages! > > > *.info;mail.none;authpriv.none;cron.none - > > /var/log/messages > > > > > > # The authpriv file has restricted access. > > > authpriv.* > > /var/log/secure > > > > > > # Log all the mail messages in one place. > > > mail.* - > > /var/log/maillog > > > > > > > > > # Log cron stuff > > > cron.* - > > /var/log/cron > > > > > > # Everybody gets emergency messages > > > *.emerg * > > > > > > # Save news errors of level crit and higher in a special file. > > > uucp,news.crit - > > /var/log/spooler > > > > > > # Save boot messages also to boot.log > > > local7.* > > /var/log/boot.log > > > > > > # Remote Logging (we use TCP for reliable delivery) > > > # An on-disk queue is created for this action. If the remote host is > > > # down, messages are spooled to disk and sent when it is up again. > > > #$WorkDirectory /rsyslog/spool # where to place spool files > > > #$ActionQueueFileName uniqName # unique name prefix for spool files > > > #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > > possible) > > > #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown > > > #$ActionQueueType LinkedList # run asynchronously > > > #$ActionResumeRetryCount -1 # infinite retries if host is down > > > # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > > > #*.* @@remote-host:514 > > > > > > > > > # ######### Receiving Messages from Remote Hosts ########## > > > # TCP Syslog Server: > > > # provides TCP syslog reception and GSS-API (if compiled to support > > it) > > > $ModLoad imtcp.so # load module > > > $InputTCPServerRun 1470 # start up TCP listener at port 514 > > > > > > # UDP Syslog Server: > > > $ModLoad imudp.so # provides UDP syslog reception > > > $UDPServerRun 514 # start a UDP syslog server at standard port 514 > > > > > > > > > SOME DEBUG OUTPUT: > > > > > > (see attached) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Steve Chupack | IT Systems Administrator > > > V: 877.327.8422 x 1242 > > > [email protected] | www.dealer.com > > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
