2015-10-06 8:26 GMT+02:00 David Lang <[email protected]>: > On Tue, 6 Oct 2015, Rainer Gerhards wrote: > >> 2015-10-06 7:27 GMT+02:00 David Lang <[email protected]>: >>> >>> On Tue, 6 Oct 2015, Rainer Gerhards wrote: >>> >>>> 2015-10-06 0:09 GMT+02:00 Kendall Green <[email protected]>: >>>>> >>>>> >>>>> I've set asyncWriting="off" and the only difference is, that with 1000 >>>>> message test, all were delivered on HUP and/or service restart, but >>>>> apparently not on flushInterval or buffer full. I'm currently running >>>>> another test with 100k messages iterating about 10 logs/sec, and >>>>> watching >>>>> the output for number of lines. The output file is created on dfs-fuse >>>>> mount upon receipt of the first log, but 0 lines written. >>>>> >>>>> This current test will hopefully show how many logs get processed >>>>> without >>>>> intervention over the next ~3 hours, and if still no logs appear will >>>>> see >>>>> how many are written upon HUP signal. >>>>> >>>>> I'm unsure how the asyncWriting off impacts the rest of the config, and >>>>> if >>>>> the current config has disabled compression... or not with no zip. >>>>> >>>>> This note is from omfile asyncWriting doc: >>>>> *Note that in order to enable FlushInterval, AsyncWriting must be set >>>>> to >>>>> “on”. Otherwise, the flush interval will be ignored. Also note that >>>>> when >>>>> FlushOnTXEnd is “on” but AsyncWriting is off, output will only be >>>>> written >>>>> when the buffer is full. This may take several hours, or even require a >>>>> rsyslog shutdown. However, a buffer flush can be forced in that case by >>>>> sending rsyslogd a HUP signal.* >>>>> >>>>> Please help clarify how these parameters correspond and what the >>>>> settings >>>>> can achieve expedited log transport, avoiding long delays waiting for >>>>> buffering or flush intervals. This goal is why have chosen low settings >>>>> for >>>>> IOBufferSize and flushInterval. >>>> >>>> >>>> >>>> Without async writing, everything is written sync on when it arrives >>>> (more precisely at commit of the current batch) aka "as quickly as >>>> possible". Thus, these params do not exist because they have no >>>> meaning. Note that with a very large batch and a small buffer, partial >>>> records may be written. >>> >>> >>> >>> although without the checkpoint parameter the writes are to the OS and >>> they >>> get flushed to disk at a later point. >> >> >> you mean "sync", right? > > > I may be misunderstanding, but i thought the checkpoint (or checkpoint > interval) parameter controleld how frequently you did the sync (so turning > sync on didn't make it per every message)
That checkopoint interval is for queues (which share the same util object), not for omfile. There, what you say is right. For omfile (if not set to async writing), the sync happens at the end of each batch, which usually means after each write because usually a full batch goes into the write buffer. If the batch has more data than the write buffer fits, all buffer writes are done and then the flush. I've also updated the doc to be more precise: https://github.com/rsyslog/rsyslog-doc/commit/b2bb5d93787f9662eb41499c86f881d2bd649dce Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
