2014-10-29 8:13 GMT+01:00 David Lang <[email protected]>: > I thought this was identified as the polling problem? > > Using polling mode is a workaround. Even then, I guess, some message can be lost when an incomplete message exists at the end of the polling interval.
I haven't had time to look at this in more depth, but I think the issue is that in non-regular read modes the partially read data does not survive to the next iteration. And iterations happen more frequently in inotify mode. But again, it's just a guess, I may be totally wrong. That said, there is no fix yet for the issue. Rainer > David Lang > > > On Wed, 29 Oct 2014, Radu Gheorghe wrote: > > Hello, >> >> Sorry for resurrecting this thread - is this issue fixed in 8.5.0 (I'm not >> sure if I should test it or not)? Or should I open a github issue about >> it? >> >> If it's fixed, is there a timeline for getting a stable version out of >> 8.5.0+? >> >> -- >> Performance Monitoring * Log Analytics * Search Analytics >> Solr & Elasticsearch Support * http://sematext.com/ >> >> On Sat, Oct 18, 2014 at 8:06 PM, Radu Gheorghe < >> [email protected]> >> wrote: >> >> Hi David, >>> >>> Thanks for commenting - I'm aware of this (normal) behavior. >>> >>> It's just a dummy file, really, I do `echo blabla >> /var/log/....` >>> repeatedly. I don't expect to see the very last message because of what >>> you >>> said, but imfile doesn't do anything in inotify mode. In polling mode it >>> works as expected. >>> >>> -- >>> Performance Monitoring * Log Analytics * Search Analytics >>> Solr & Elasticsearch Support * http://sematext.com/ >>> >>> On Sat, Oct 18, 2014 at 12:35 PM, David Lang <[email protected]> wrote: >>> >>> Remember that with readmode != 0, rsyslog needs to see the start of the >>>> next log message before it can be sure that the prior one has finished. >>>> So >>>> you have to have it reading a file with more than one message in it. >>>> >>>> What does the file you are realing look like? >>>> >>>> David Lang >>>> >>>> On Fri, 17 Oct 2014, Radu Gheorghe wrote: >>>> >>>> Date: Fri, 17 Oct 2014 14:32:10 +0300 >>>> >>>>> From: Radu Gheorghe <[email protected]> >>>>> Reply-To: rsyslog-users <[email protected]> >>>>> To: rsyslog-users <[email protected]> >>>>> Subject: [rsyslog] not forwarding when imfile has readmode=2 >>>>> >>>>> >>>>> Hello, >>>>> >>>>> I was just testing an rsyslog 8.4.2 with our Logsene and I didn't >>>>> understand why I couldn't see no logs from the tailed files. The >>>>> problem >>>>> seems to be that, with ReadMode set to 2, imfile doesn't pass events on >>>>> no >>>>> matter what you put in the file. >>>>> >>>>> I always see this in the debug log and nothing else: >>>>> >>>>> 3954.342512201:imfile.c : DDDD: imfile: wd 3 got file 1, dir -1 >>>>> 3954.342522434:imfile.c : strm 0x7fc6e00045b0: file 7 read 9 >>>>> bytes >>>>> 3954.342528353:imfile.c : strm 0x7fc6e00045b0: file 7 read 0 >>>>> bytes >>>>> 3954.342537453:imfile.c : stream checking for file change on >>>>> '/var/log/cassandra/system.log', inode 263735/263735 >>>>> 3962.854107352:imfile.c : DDDD: imfile event notification: rd >>>>> 16[0], >>>>> wd (3, mask 00000002, cookie 0000, len 0) >>>>> 3962.854124058:imfile.c : watch was MODIFID >>>>> >>>>> For reference, here's a complete configuration that normally works with >>>>> Logsene: >>>>> >>>>> module(load="imfile") >>>>> >>>>> input(type="imfile" >>>>> File="/var/log/cassandra/system.log" >>>>> Tag="cassandra-system:" >>>>> StateFile="cassandra-system" >>>>> ReadMode="2") >>>>> >>>>> *.* @@logsene-receiver-syslog.sematext.com;RSYSLOG_ >>>>> SyslogProtocol23Format >>>>> >>>>> As soon as I set ReadMode to the default 0, things work as expected >>>>> (logs >>>>> are forwarded all the way). >>>>> >>>>> Does anyone else experience this issue? It may be recently introduced, >>>>> because I tested various versions and I never had this problem. And I >>>>> was >>>>> always using ReadMode 2. >>>>> >>>>> Best regards, >>>>> Radu >>>>> -- >>>>> Performance Monitoring * Log Analytics * Search Analytics >>>>> Solr & Elasticsearch Support * http://sematext.com/ >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> >>>>> _______________________________________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> >>> >>> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
