2016-02-23 17:38 GMT+01:00 Kane Kim <[email protected]>: > Hello Rainer, thanks for the prompt reply! To give you some context: I want > to write module that both using batching and also can't loose messages in > any circumstances. Are you saying it is by design that rsyslog can't do > that together? According to documentation rsyslog will retry if module > returns any error. Do you plan to fix this in rsyslog or update > documentation to say batching and retries don't work?
It depends on many things. In almost all cases, the retry should work well (and does so in practice). Unfortunately, I am pretty swamped. I need to go to a conference tomorrow and have had quite some unexpected work today. It would probably be good if you could ping me next week to see if we can look into more details what is causing you pain. But I can't guarantee that I will be available early next week. In general, we cannot handle a fatal error here from an engine PoV, because everything is already processed and we do no longer have the original messages. This is simply needed if you want to process messages one after another through the full config (a goal for v8 that was muuuuch requested). As I said, the solution is to use batches of one, because otherwise we would really need to turn back time and undo everything that was already done on the messages in question by other modules (including state advances). Rainer > > What would be your recommendation for my case? Is there any workaround to > have batches and retries? > > Thanks! > On Feb 22, 2016 11:03 PM, "Rainer Gerhards" <[email protected]> > wrote: > >> 2016-02-23 0:08 GMT+01:00 Kane Kim <[email protected]>: >> >> > I return RS_RET_SUSPENDED from EndTransaction and expect rsyslog to retry >> > failed messages. My configuration looks like following: >> > >> > queue.spoolDirectory="/var/spool/rsyslog" queue.filename="some_filename" >> > queue.type="Disk" queue.maxdiskspace="1G" queue.saveonshutdown="on" >> > action.resumeRetryCount="-1" action.reportSuspensionContinuation="on" >> > action.resumeInterval="1" >> > >> > After logging a line: >> > >> > 6130.843876518:action 0 queue:Reg/w0: Action 0 transitioned to state: rdy >> > 6130.844112940:action 0 queue:Reg/w0: Action 0 transitioned to state: itx >> > 6130.844131796:action 0 queue:Reg/w0: Action 0 transitioned to state: >> rtry >> > 6130.844137106:action 0 queue:Reg/w0: actionCommit, in retry loop, iRet >> > -2007 >> > 6130.844142731:action 0 queue:Reg/w0: regular consumer finished, >> > iret=-2007, szlog 0 sz phys 1 >> > 6130.844149164:action 0 queue:Reg/w0: DeleteProcessedBatch: we deleted >> > 1 objects and enqueued 0 objects >> > 6130.844154698:action 0 queue:Reg/w0: doDeleteBatch: delete batch from >> > store, new sizes: log 0, phys 0 >> > 6130.844160252:action 0 queue:Reg/w0: regular consumer finished, >> > iret=4, szlog 0 sz phys 0 >> > 6130.844165414:action 0 queue:Reg/w0: action 0 queue:Reg/w0: worker >> > IDLE, waiting for work. >> > >> > That message is not resent by rsyslog after failure. Is there special >> > configuration? >> > >> > I've found TODO here: >> > https://github.com/rsyslog/rsyslog/blob/master/action.c#L1236 >> > That says "For the time being, I do NOT implement all of this (not even >> > retry!)". I Wonder if retry functionality is not finished in rsyslog? >> > >> > >> That comment is no longer really true, as many things have been changed on >> the upper level. However, what is true is that we should write an error >> file. This actually slipped through. It is right that we cannot do a retry >> if we used the batch commit feature, because we do no longer have all the >> data in place. This was a tradeoff we had to make when moving to the v8 >> engine. You can configure a batch size of 1, though, if you need to have >> retry under all cases. >> >> Rainer >> >> > Thanks! >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > DON'T LIKE THAT. >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
