Still swamped from my days out at consulting, so I just want to give a quick heads-up on the original question: the doc says this with a reason, and I actually got a request not so long ago from someone who actually wants to have this. The use case is even simpler, and so it makes some sense (but I can't disclose more on the use case as this may show off more than I am permitted to). So it *might* happen that this actually gets implemented some day. If so, it'll be optional and not stay in the way of what the rest of the world needs ;)
Rainer 2015-10-23 17:20 GMT+02:00 David Lang <[email protected]>: > On Fri, 23 Oct 2015, Tomas Heinrich wrote: > >> On 10/23/15 15:06, David Lang wrote: >>> >>> On Thu, 22 Oct 2015, Tomas Heinrich wrote: >>> >>>> Hello, >>>> >>>> I've been reading some RELP documentation[0] and it mentions a planned >>>> capability to only acknowledge reception after the _output_ module of >>>> the recipient acknowledges delivery. >>>> I don't think this has been implemented yet but I was wondering >>>> whether this is still on the roadmap and whether it is close to or far >>>> from being completed. >>> >>> >>> I don't believe that this can be done reasonably with the current >>> architecture. >> >> >> Right, that's what I think too. But the docs mention it and they are >> probably pretty old, so I was wondering whether there was some provision >> conceived for this use case - or whether this idea was abandoned (for the >> time being). >> >>> The problem is that the queues get in the way and the >>> input module can't easily know when the message has been fully processed >>> when it could end up going through multiple queues and get forked to >>> multiple copies in the process. >> >> >> Yeah, the async processing is the problem in this case. And then there are >> the partial successes and message duplication... >> >>> What you can do is to use a disk-only queue with syncing turned on and >>> checkpintinterval set to 1 (and IIRC another parameter or two) and this >>> will guarantee that the message won't be lost. >> >> >> Well, this would ack the enqueuing and not the actual delivery. I think >> anything else than a direct queue would be problematic from a reliability >> POV. >> >>> Or you could set the main queue type to direct (and not use any action >>> queues), to have it processed immediately (I'm not sure if this would >>> quite do what you want, but it would be very close) >> >> >> There could be even a stronger requirement - not only guaranteeing >> delivery in an infinite time (over a chain of relays) but also signaling a >> failure and aborting the sending attempts after a timeout so that the >> original sender could perhaps fail-over to a different destination. >> >>> Unfortunantly, both of these solutions will absolutly kill performance >>> (by a factor of ~1000x or so) >> >> >> Yes, with the current architecture this mode of operation seems >> unfeasible. >> >> Thanks for you thoughts David. > > > absolute guaranteed delivery through multiple relays is especially > problematic. What do you do if the log message goes to multiple destinations > and some work and others don't? > > Do you really want to freeze the sending box if there is a problem further > down the line that delays the delivery? > > If you really want absolute guaranteed delivery to a single destination, you > are looking for a database that provides ACID guarantees, not a general > purpose logging system. There are good reasons why people don't put logs in > databases, with performance being one of the biggest ones. > > There are some 'log' solutions that claim to give you end-to-end > acknowlegement, but if you look into them, you find that performance suffers > and your app needs to keep track of a LOT of outstanding state because it > can take long enough for the ack to come back that you can end up with a LOT > of data in flight. > > David Lang > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
