On Mon, Oct 21, 2013 at 12:20 PM, Rainer Gerhards <[email protected]>wrote:
> On Mon, Oct 21, 2013 at 12:16 PM, David Lang <[email protected]> wrote: > >> On Mon, 21 Oct 2013, Rainer Gerhards wrote: >> >> On Mon, Oct 21, 2013 at 10:42 AM, Rainer Gerhards >>> <[email protected]>**wrote: >>> >>> On Mon, Oct 21, 2013 at 8:52 AM, Rainer Gerhards < >>>> [email protected] >>>> >>>>> wrote: >>>>> >>>> >>>> On Sun, Oct 20, 2013 at 6:03 PM, Pavel Levshin <[email protected] >>>>> >wrote: >>>>> >>>>> >>>>>> I am unable to reproduce this behaviour with global variables. This is >>>>>> what I've tried among others: >>>>>> >>>>>> if $/zz % 3 == 0 or $/zz % 3 == 1 then { >>>>>> set $/zz = $/zz + 1; >>>>>> action(...) >>>>>> } else { >>>>>> set $/zz = 0; >>>>>> action(...) >>>>>> } >>>>>> >>>>>> Could you please explain how is it supposed to work? >>>>>> >>>>>> >>>>> It's supposed to work just as you describe it. But indeed, it doesn't >>>>> do >>>>> so, I can reproduce the problem. Looks like a regression. Thanks for >>>>> reporting, will now look into it. >>>>> >>>>> >>>> OK, looks like I stumbled into my own trap. In script, you access >>>> properties via $<propname>. Global variables have the name $/zz (with zz >>>> being the real name). So to access them, you need to access $$/zz. >>>> >>>> I think I got confused about this some time ago, and the doc is also not >>>> correct or at least inconsistent. I now need to work my way through that >>>> mess. Just thought I give you some explanation. >>>> >>>> >>> ... and it gets even worse: when I quickly implemented global vars before >>> my vacation, I did not fully think out the implications :-( >>> >>> The use case then was to have some incrementing counters. That works. >>> However, due to the way the rule engine works, this use case here does >>> not >>> work as expected. The reason is that the rule engine is a SIMD machine: >>> it >>> applies all operations to all message inside the batch. As such, it does >>> the if-evaluation on the whole batch, then the variable increment, than >>> the >>> action and so on. It works this way because this is a very big >>> performance >>> gain. However, this means when zz is 0 at start of a batch of 10 >>> messages, >>> the if will work with 0 for all 10 of them. Then, zz will be ten times >>> incremented, leding to value 10. Then, the action is executed (10 times). >>> >>> I am not sure what the proper solution is. The most clean solution would >>> be >>> to remove support for global variables, the best but very costly would be >>> to enhance the engine to have a special execution mode (much slower) for >>> statements that involve global variables. The quick and dirty solution is >>> to leave everything as is- but document this behaviour -- as there are >>> valid use cases for the current state. >>> >>> I think I will take the "just document" path for now, and think about >>> removing global variables (after all, so far they are only in the >>> devel/experimental tree). Comments are appreciated. >>> >>> @Paval: did you check that mmsequence actually works as you expect? IMHO >>> it should expose the same behaviour as global variables. >>> >> >> I think it's fine to just document for now. >> >> I think the 'right' answer is to create a magic variable that is the >> number of messages in the batch. This will allow you to have a counter of >> the number of messages. >> >> now, the counter will not give every message a unique value if you have >> batch size > 1, but this is actually pretty much the case with other things >> related to the batch (like timestamps on received messages), so it's not >> unusual. >> >> As far as load balancing goes, balancing a batch of messages at a time is >> actually 'good enough', and since it saves you manipulating individual >> messages, it's a performance win as well. >> > > I think this variable shuffling is the right approach > OK, I *really* need a break "is ***NOT*** the right approach" it should read... Rianer > to load balancing. As I wrote yesterday, I think it is fairly easy to do > it inside the engine. If I had not been lost in the vars, I would have > started with this today and were positive I could have finished it quickly. > If the remaining time permits, I'll probably still try to squeeze it in. > > > >> >> Since this can be dealt with by just setting the batch size = 1, I think >> that it's acceptable. >> > > yup, except for performance (the ES http request with batch size 1 ;)). > But your are basically right for most things. > > Rainer > >> >> David Lang >> >> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
