On Tue, Oct 22, 2013 at 11:37 AM, David Lang <[email protected]> wrote: > On Tue, 22 Oct 2013, Rainer Gerhards wrote: > > On Tue, Oct 22, 2013 at 11:09 AM, David Lang <[email protected]> wrote: >> >> On Tue, 22 Oct 2013, Rainer Gerhards wrote: >>> >>> On Tue, Oct 22, 2013 at 10:23 AM, David Lang <[email protected]> wrote: >>> >>>> >>>> >>>> Most folks would consider this to be an error ;) I agree to Pavel that >>>> this is so far from being what one expects, that even documenting how >>>> things work will not really help getting away from that error-like feeling. >>>> Add the fact that the result is non-deterministic as it depends on the >>>> current batch size, I would also conclude that this is errornous *design*. >>>> >>>> I probably was not clear enoug in my initial question: as I said >>>> yesterday evening, I have now removed global variable support. It won't be >>>> available in 7.5.6. My question thus was on the use cases that could guide >>>> us if working toward a correct implementation makes any sense. This would >>>> require a big design change. >>>> >>>> >>> the use case wasn't trying to be something as precise as you are talking >>> about, but rather a much corser ability that after you see message X show >>> up, start doing something different. This could be to write files to a >>> different location because a failover has happened and you are now the >>> active box, it could be that if a particular type of log message shows up >>> you want to log debug messages, or something else. >>> >>> given that logs can be re-ordered by the network, by relays in the >>> middle, >>> or by the results of batch processing, it was never going to be an exact >>> thing, so if a change only takes effect for the next batch, that's not >>> unreasonable. >>> >>> yes, this is edging towards log correlation territory, but it's an >>> extremely handy way to be able to turn things on and off, or to cause >>> logs >>> to be handled differently without requiring on-the-fly editing of config >>> files and a complete restart. >>> >>> the log losses from a complete restart are massive compared to the global >>> variable impresision of only setting the variable on the next batch. >>> >>> >>> >>> global variables are not that useful as something that you would set for >>> each message that you process, but if they are only occasionally changed, >>> they can be very useful and not a significant performance issue. >>> >>> >>> It's not the performance issue -- it's the "strong smell of >> incorrectness". >> Remember that I must support that whole thing in the future. People will >> continously ask why it "does not work", and some of them will >> (even/hopefully/probably) have support contracts, so I am actually in >> charge to deliver a solution. If I then just describe "it is as it is" >> that >> will create very unsatisfied customers. So better do it right or do it not >> at all (which is what I currently opt for, and my peers will kill me if I >> knowingly introduce a feature that potentially creates great >> user-desatisfaction). >> > > > > by default the batch size is still 1 correct? it's only people who are > scaleing up who will be setting it higher. > > no, for a very long time not. I think when we introduced batching, the default was 8, and today it is either 256 or 512 (all for main q). A lot of optimizations (think transaction support) depend on relatively large batches.
> I think that if you just document that setting global variables is racy, > and as such it's not suitable for accruate counting, only for changing > rsyslog behavior without a restart you should be good on the expectations > front. the current documentation leans heavily on the counting aspect of > things, but that can be changed. remove any suggestion of future atomic > opertations and emphisise that atomic operations are not possible. > OK, so let's go back to use cases: you mean "change behaviour after an approximate number of messages has been processed" right? > being able to enable or disable e-mail messages, change what the > destination address is, change the filename or patch when the box becomes > active are all very useful items that I would hate to loose. > all done based on counts? In any case, let's see what happens when 7.5.6 does no longer support global vars. Maybe some folks show up and tell their actual pain, and then we can see if there is a better cure... Rainer > even the load balancing hack works 'well enough' once you accept that you > are balancing per batch rather than per message (even if you did balance > per message, you really have no idea how expensive a particulare message is > going to be, so you are not really balancing the work precisely, you are > only doing so statistically, and balancing per batch rather than per > message is just as valid statistically) > > > David Lang > ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> > http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
