Thanks for your help Rainer, I'll try to debug what's going on, so far it seems rsyslog doesn't retry even with batch size 1. It doesn't retry if I return RS_RET_SUSPENDED from DoAction as well.
On Tue, Feb 23, 2016 at 9:05 AM, Rainer Gerhards <[email protected]> wrote: > 2016-02-23 18:03 GMT+01:00 David Lang <[email protected]>: > > On Tue, 23 Feb 2016, Rainer Gerhards wrote: > > > >> 2016-02-23 17:38 GMT+01:00 Kane Kim <[email protected]>: > >>> > >>> Hello Rainer, thanks for the prompt reply! To give you some context: I > >>> want > >>> to write module that both using batching and also can't loose messages > in > >>> any circumstances. Are you saying it is by design that rsyslog can't do > >>> that together? According to documentation rsyslog will retry if module > >>> returns any error. Do you plan to fix this in rsyslog or update > >>> documentation to say batching and retries don't work? > >> > >> > >> It depends on many things. In almost all cases, the retry should work > >> well (and does so in practice). Unfortunately, I am pretty swamped. I > >> need to go to a conference tomorrow and have had quite some unexpected > >> work today. It would probably be good if you could ping me next week > >> to see if we can look into more details what is causing you pain. But > >> I can't guarantee that I will be available early next week. > >> > >> In general, we cannot handle a fatal error here from an engine PoV, > >> because everything is already processed and we do no longer have the > >> original messages. This is simply needed if you want to process > >> messages one after another through the full config (a goal for v8 that > >> was muuuuch requested). As I said, the solution is to use batches of > >> one, because otherwise we would really need to turn back time and undo > >> everything that was already done on the messages in question by other > >> modules (including state advances). > > > > > > I thought that if a batch failed, it pushed all the messages back on the > > queue and retried with a half size batch until it got to the one message > > that could not be processed and only did a fatal fail on that message. > > > > Now, there is a big difference between a module giving a hard error "this > > message is never going to be able to be processed no matter how many > times > > it's retried" vs a soft error "there is a problem delivering things to > this > > destination right now, retry later". I thought the batch processing > handled > > these differently. > > That's no longer possible with v8, at least generically. As I said > above, we would need to turn back time. > > But I really run out of time now... > > Rainer > > > > David Lang > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
