2015-10-05 12:45 GMT+02:00 David Lang <[email protected]>: > I have seen problems with omfile and async writers (especially when > compressing), so for now I just don't do tht > > could you post your config? > > do you have a disk queue? that would do a random read. But the output from > omfile should not be doing any reading.
99% (but not 100%) sure: I think we read and write queue files sequentially. Rainer > > David Lang > > On Mon, 5 Oct 2015, Kendall Green wrote: > >> Date: Mon, 5 Oct 2015 00:32:48 -0700 >> From: Kendall Green <[email protected]> >> Reply-To: rsyslog-users <[email protected]> >> To: rsyslog-users <[email protected]> >> Subject: Re: [rsyslog] omfile fuse >> >> >> Hi David, thank you for the suggestions. >> >> We've tried setting the checkpoint, flush interval, flush tx on end, and >> looked for options to disable or turn off compression, but there might be >> some outdated or misinformation in the omfile docs. It's probably >> unimportant or unrelated but there's at least one contradiction in the >> descriptions for default parameters for if robust zip default is off or >> on. >> >> Regardless, changes to settings had no affect on messages failing to be >> delivered to the fuse mount. Further testing showed, that messages would >> eventually disappear from the queue if waiting long enough before restart >> or HUP. Waiting for messages to reach checkpoint of flush interval would >> sometimes result in only a single message, depending on how long before >> deciding to HUP/restart. >> >> As a test with 100 messages to be parsed to the dfs fuse mount, if HUP or >> restart takes place immediately after the messages get stuck in transit, >> all 100 appear on the dfs file system. Increasing this to 1000 messages >> would process the entire batch and deliver when signal HUP, but restart >> seems to write only about 350/1000 messages before shutdown completes, >> losing the remaining logs. When sending in hundred of thousands of >> messages, about 20k showed up after HUP, so the messages don't appear to >> write when buffer is full. The buffer appears to overflow instead when >> using dfs fuse driver that doesn't support random read write access. >> >> "Attempting random read write" is a consistent error message which appears >> when rsyslog fails writing to the dfs-fuse mount. >> Unfortunately there is no option for replacing the vendor supported dfs >> fuse driver, or I'd try an alternate which support random read write. >> >> Does it make sense for this error as indicator for what is preventing the >> parser from normal delivery of logs to dfs fuse mount? >> Is there a way to set rsyslog to perform other method (guessing >> sequential) >> omfile output without attempting random read write access? >> Are there any specific configuration settings I should try? >> >> Any recommendations are greatly appreciated! >> >> >> Regards, >> Kendall >> >> >> >> On Thu, Oct 1, 2015 at 7:50 PM, David Lang <[email protected]> wrote: >> >>> On Thu, 1 Oct 2015, Kendall Green wrote: >>> >>> Rsyslog 8.13 and 8.12 have issues when writing omfile to dfs fuse mount. >>>> >>>> >>>> The messages are stuck in limbo until stopping or restarting rsyslog. >>>> >>>> Changing the omfile destination from the fuse mount path to a local disk >>>> directory results in logs output immediately. >>>> >>>> There are no apparent issues with the fuse mounted filesystem (other >>>> than >>>> rsyslog omfile delivery is stalled until service shutdown/restart). >>>> >>>> Has anyone ran across this problem? >>>> >>> >>> I'll bet that if you HUP rsyslog the contents will appear as well. It >>> sounds like the data is not getting flushed out to where it's visible to >>> readers on these more complex filesystems, but on a local filesystem you >>> are seeing the data flushed much more rapidly. >>> >>> I'd also bet that if you keep writing data, eventually you will cross >>> some >>> threshold and a large chunk of data will appear. >>> >>> writing data out compressed increases the window for this because the >>> compression algorithm really wants a full buffer before it processes the >>> data and writes it out. >>> >>> if you look at the omfile documentation, you will see soem flush and >>> checkpoint paramters that you can use to tweak this behavior. It sounds >>> as >>> if there may also be some stuff you can do with fuse to adjust the amount >>> of buffering and when the data becomes visible. >>> >>> David Lang >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
