Hi, I can verify that this issue IS still happening on the latest 8.4.0 when installed from RPM's, however, I have built rsyslog from source on Ubuntu 14.04 and run against the same central server with the same config file .. and apparently it's NOT happening.
My (working) source build version was set up with; ./configure --enable-imdiag --enable-relp --enable-guardtime (as opposed to the RPM which looks to include pretty much everything) Before I start to iterate over options, does this ring any bells with anyone ?? tia Gareth. ----- Original Message ----- From: "Rainer Gerhards" <[email protected]> To: "rsyslog-users" <[email protected]> Sent: Tuesday, 9 September, 2014 4:52:29 PM Subject: Re: [rsyslog] Logging to central server / data loss .... n Tue, Sep 9, 2014 at 1:03 PM, Gareth Bult <[email protected]> wrote: > Hi Rainer, > > Many thanks for looking, I appreciate you're busy. > > If it looked trivial I might've tried to patch it, but it "looks" like > it's pulling from the queue and then running the send plugins, so my > initial > impression is that various bits of code need reordering - which is too much > for me. I would guess it needs to be peeking the queue and only de-queueing > once all the output modules have been satisfied .. > > Just quickly: that's not the case. Messages are deleted from the queue only after the ruleset has been processed. However, actions cannot stall the overall process (except if you *really* want this tell rsyslog to actuall stall in that case) so we need to keep track of messages inside the relp stack. As far as I can see, the problem is that some status seems to be either returned wrongly or be processed wrongly, leading to this missing one message. HTH Rainer > It's interesting how things develop, back in "the good-old-days" central > logging > was useful to spotting problems without sshing to lots of boxes, and some > data > loss / the use of udp was quite acceptable. Today however, people seem to > be > using it for collecting 'important' information where 100% accuracy and log > signing are critical .. a paradigm shift in "use-case" really ... > > Regards, > Gareth. > > ----- Original Message ----- > From: "Rainer Gerhards" <[email protected]> > To: "rsyslog-users" <[email protected]> > Sent: Tuesday, 9 September, 2014 11:38:07 AM > Subject: Re: [rsyslog] Logging to central server / data loss .... > > On Tue, Sep 9, 2014 at 10:42 AM, Gareth Bult <[email protected]> wrote: > > > Hi, > > > > I logged a debug session (and configs) here on the 4th of September > > apparently showing the queueing > > system dropping data when logging from a syslog client to a central > server > > via either TCP or > > RELP (during disconnect/reconnect). > > I've managed to replicate this on separate systems with slightly > different > > configurations. > > > > After looking at the code, trying to address the issue is out-of-scope > for > > me on this project .. so before I > > switch to an alternative mechanism for transporting data from client to > > server, does anyone have any > > words of wisdom that might help, or alternatively can you recommend an > > alternative? > > (no data loss is a primary requirement) > > > > > Just FYI: I have verified, and it looks like a real bug. However, I need to > find time to look at it. It's on the list, but there are lots of things > going on. > > Rainer > > > > fluentd seems to be top of my list at the moment .. (???) > > > > tia > > Gareth. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
