David- I do have an ActionQueue before my TCP destination, but do also have retries set to -1 as you pointed out. Here is a more complete set of rules:
---------------------- RULES # Log anything except authpriv, cron, syslog, of level info or higher. *.info;local0.none;authpriv.none;cron.none;syslog.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log cron stuff cron.* /var/log/cron # Keep syslog separate syslog.* /var/log/syslog # Keep mail separate mail.* /var/log/maillog # Everybody gets emergency messages *.emerg;local0.none * # Save news errors of level crit and higher in a special file. uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log $ActionQueueType LinkedList # run asynchronously $ActionQueueFileName mgmtfwd # unique name prefix for spool files $ActionResumeRetryCount -1 # infinite retries if host is down $ActionQueueSaveOnShutdown on # save messages to disk on shutdown $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) >>driver mode settings<< *.info;local0.* @@SOME_IP:SOME_PORT -------------------------------------- Hopefully a larger context of the rules may help. I DO see the startup log printed to /var/log/syslog, as you mentioned, yet it seems all subsequent logs don't get processed to their destinations. Any ideas? Along the lines of the ActionResumeRetryCount, I am under the impression that setting it to -1 means that the TCP socket will try infinitely to send to the remote host, but will use the ActionQueueFileName to save the data if it cannot connect/send. Does it have other implications with regard to writing to the ActionQueueFileName and not merely connecting/sending? Logan Rodrian ________________________________________ From: [email protected] [[email protected]] on behalf of David Lang [[email protected]] Sent: Friday, August 02, 2013 11:50 To: rsyslog-users Subject: EXT :Re: [rsyslog] Local persistence issue when no TCP connection On Fri, 2 Aug 2013, Rodrian, Logan P (IS) wrote: > I am running RHEL 6.3 with included rsyslog-5.8.10-2 rpm. > > I am having an issue upon system boot, whereby logs are not being written to > /var/log/messages when the (later) action to send to another host via TCP > cannot make a connection. > > A snippet from my rules are as follows: > > ... > # Log anything except authpriv, cron, syslog, of level info or higher. > *.info;local0.none;authpriv.none;cron.none;syslog.none /var/log/messages > >>> more rules<< > >>> action and driver mode settings<< > *.info;local0.* @@SOME_IP:SOME_PORT > > ... > It does not matter if I put the remote rule before or after the one directing > to /var/log/messages. > > After boot, I test using 'logger -p security.crit some message', where I > would expect the string 'some message' to appear in /var/log/messages, but it > is not there. > > Using wireshark, I see that the only network traffic upon boot are ARP > messages looking for SOME_IP. > > If I restart the rsyslog service, then I can re-execute the 'logger' command > and then I do see the expected string in /var/log/messages. Looking again at > wireshark, there are still only ARP messages looking for SOME_IP. > > Is there a bug in this version that causes hanging on boot, whereby some > dependent service/item isn't running yet that does exist/is ready when doing > a service restart? The order of services starting are iptables, network, > rsyslog, ... > > Any ideas as to what is causing this/how to fix it? remember that when rsyslog starts up it logs a startup message. If that message is unable to be written to one of your destinations, rsyslog will not process additional logs until and unless it decides to stop writing any logs to that destination. one way to deal with this is to create an action queue for the remote message, that way it won't halt processing of logs for other destinations. If the destination remains unavailable, eventually the action queue will fill up and you will be back in the same situation. The other thing you will need to look at is the retry/discard settings, telling rsyslog how many times to retry sending the log to an output before deciding that it's always going to fail, and what to do in that situation. It's common for people to set retries = -1, which tells rsyslog to never give up, but without seeing your entire config, this is just a guess. David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
