ah, maybe I misunderstood it. So it is good to getting this straigt. Anyhow...
Queue Type for action queues is *per action*. So it really doesn't matter in regard to filters where the queue is defined. So in your sample (a) some config stuff (b) filter to throw away some log messages (c_1) define the queue type (c_2) filter to write log messages to disk Both c_1 and c_2 are the same action. Most importantly, c_1 does not modify anything but the next action immediately following. That means if you have *.* ~ $Action... *.* /logfile logfile should never receive anything, no matter what the queue def is. This is for action queues. For the main queue, it doesn't matter where the typedef is. The queue params are pulled once when the queue is generated and that is AFTER all config statements have been read. I'll try to dig into that use case, but I finally managed to make me very late on my paper deadline. I will probably not be able to do anything but writing this week... Rainer > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of [email protected] > Sent: Monday, February 14, 2011 10:03 AM > To: rsyslog-users > Subject: Re: [rsyslog] Reliable forwarding Not Working > > I could be mistaken here, but as I am currently understanding it (and > without trying to go back through the mailing list archives) it sounds > like what he is doing is > > some config stuff > > filter to throw away some log messages > > define the queue type > > filter to write log messages to disk > > > and he is saying that if he does not define the queue type, everything > works, but once he defines the queue type it ignores the filters that > are > supposed to throw away some log messages. > > I'm saying that I suspect that if he rearranged the config file to be > > some config stuff > > define the queue type > > filter to throw away some log messages > > filter to write log messages to disk > > I would expect it to work better. > > David Lang > > On Mon, 14 Feb 2011, Rainer Gerhards wrote: > > > Mhhh... am I overlooking something? As far as I can see, he first > defines the > > queue parameters and then a (single, as necessary) action. Do you see > more? > > > > Rainer > > > >> -----Original Message----- > >> From: [email protected] [mailto:rsyslog- > >> [email protected]] On Behalf Of [email protected] > >> Sent: Monday, February 14, 2011 9:13 AM > >> To: [email protected]; rsyslog-users > >> Subject: Re: [rsyslog] Reliable forwarding Not Working > >> > >> why don't you define the filters after you define the queue type? > >> > >> given the way that rsyslog interprets the config file, I can easily > see > >> how it could loose the effect of filters if you define a queue type > >> after > >> the filters in the config file. Defining a different queue type > changes > >> somuch of what rsyslog is doing with the messages that I would not > be > >> at > >> all surprised if the process of setting that up lost any previously > >> defined fulter rules. > >> > >> I would also be surprised if that was a order that had ever been > >> tested. > >> > >> David Lang > >> > >> On Sat, 12 Feb 2011, Eric Schoeller wrote: > >> > >>> I've been playing around with this recently too. You certainly need > >> to create > >>> the spool directory. rsyslog won't do it (but like you, I thought > it > >> would). > >>> Once I did that, I quickly saw spool files being created there > while > >>> performing tests similar to yours. > >>> > >>> My issue is still standing, previous filters within the rsyslog > >> configuration > >>> aren't honored when you've enabled disk queuing. Everything gets > sent > >> to the > >>> log server, even messages specifically thrown away earlier with > '~'. > >> I'd be > >>> interested to see if you experience similar behavior. > >>> > >>> -- Eric > >>> > >>> Todd Michael Bushnell wrote: > >>>> I configured reliable forwarding in accordance with instructions > >> here: > >>>> http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html > >>>> > >>>> Version: rsyslog-3.22.1-3.el5_5.1 > >>>> > >>>> Configuration: > >>>> > >>>> # forward to remote host, queueing to local disk if host is down > and > >> memory > >>>> fills up > >>>> # work (spool) files directory > >>>> $WorkDirectory /var/log/rsyslog # start forwarding rule - loghost2 > >>>> # in-memory queue; set for asynchronous processing (?) > >>>> $ActionQueueType LinkedList > >>>> # failover queue filename; also enables disk mode > >>>> $ActionQueueFileName failqueue-loghost2 > >>>> # infinite retries on insert failure > >>>> $ActionResumeRetryCount -1 > >>>> # save in-memory data if rsyslog shuts down > >>>> $ActionQueueSaveOnShutdown on > >>>> # remote logging of everything > >>>> *.* @@loghost2:5140 > >>>> > >>>> I wanted to test its functionality before going into production. > >>>> > >>>> First, I used iptables to block access to the syslog port on the > >> central > >>>> syslog (syslog-ng) server, simulating a down syslog server: > >>>> # on loghost2 > >>>> /sbin/iptables -I INPUT -p tcp --destination-port 5140 -j REJECT > >>>> --reject-with icmp-admin-prohibited > >>>> > >>>> I then ran logger through a loop to start creating a pile of > >> messages on > >>>> the rsyslog client: > >>>> for i in {1..1000000}; do logger -t tmbtest -p local1.info "this > is > >> a test > >>>> $i"; done > >>>> > >>>> I ran this loop twice in an effort to sufficiently fill up memory > >> and > >>>> initiate dump to disk. While this loop was running I verified > that > >> memory > >>>> consumption for the rsylogd process on the client was increasing. > >> It > >>>> eventually got to this point: > >>>> root 20263 0.2 77.8 2537008 1603712 ? Sl Feb10 6:09 > >>>> /sbin/rsyslogd -c 3 > >>>> > >>>> To be honest, I don't know how much memory it will consume before > >> dumping > >>>> to disk (feel free to school me on this) so I figured I'd keep > going > >> until > >>>> I saw /var/log/rsyslog directory and files created. This never > >> happened > >>>> and my second iteration stopped at about 600k and I saw some > memory > >> fork > >>>> errors (though they dumped only to standard error, not log, so I > >> lost them > >>>> (sorry)). > >>>> > >>>> Dump to disk having failed, I next wanted to see if rsyslog would > at > >> least > >>>> resume forward to remote host when it came back up (dumping > whatever > >> was in > >>>> memory to central syslog server). I restarted iptables on the > >> syslog > >>>> server to restore access to the port, but no logs were forwarded > >> from the > >>>> rsyslog client. > >>>> > >>>> Lastly, I restarted rsyslog, hoping that I would see a dump to > disk > >> but > >>>> this failed as well. > >>>> I'm sure it's something I'm doing incorrectly. Would appreciate > >> some > >>>> guidance. Who knows, maybe I just need to create the > >> /var/log/rsyslog > >>>> directory (assumed rsyslog would create it). While I'm waiting > for > >>>> feedback, I'll probably give that a shot. Thanks. > >>>> Todd > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
