On Tue, Oct 22, 2013 at 3:40 PM, Rainer Gerhards <[email protected]>wrote:
> just an important one: > > > On Tue, Oct 22, 2013 at 3:31 PM, David Lang <[email protected]> wrote: > >> NO! Because you once again use global space between statements. Problem >> >>> reapears. The operation MUST happen and result returned immediately. This >>> would work: >>> >>> set $.myhostcounts = globalinc(var='$/hostcounts' subvar='$hostname') >>> >>> because it is in a single statement. Note the difference (two vs. one >>> stmts). >>> >> >> you aren't always going to be able to combine things into a single >> statement, sometimes you will be dealing with multiple counters you want to >> manage (say hostname based and application based). >> > > than it does NOT work! Keep on your mind that each statement is executed > for *all* messages, and *then*, the rule engine advances to the next > statement. That's the SIMD mode -- even though it's of course not true > SIMD, the analogy is very strong (doing real SIMD would be much too > fine-grained and result in giantic overhead). > > >> or what if you want to do an if condition based on a global variable, and >> then modify it based on the result? (think reset a counter for example) >> >> my idea is that any function that modifies a global variable must do so >> instantly and atomically, and any read of the global variable must also be >> instant. >> > > But that DOES NOT help you with the real problem. Maybe it helps a bit if > you play with the current implementation in various cases. I am trying to > get this message over, but I fail, we are not on the same line. The > "problem" is that the SIMD engine and global state do not play well > together as soon as multiple statements are involved. > > I just got another approach trying to explain this. Again, take a conf with two statements (doesn't matter what they are): stmt1 stmt 2 Then we have 4 messages a,b,c,d. I think you expect this execution order: stmt1 a stmt2 a stmt1 b stmt2 b stmt3 a stmt3 b stmt4 a stmt4 b but in reality, it is this one: stmt1 a stmt1 b stmt1 c stmt1 d stmt2 a stmt2 b stmt3 c stmt3 d And now think about the implications if stmt1 modifies some global state and stmt2 uses it. Then do the same for some property that is specific to the message (aka "contained inside" a,b,c,d). It may help to get a piece of paper and a pencil ;) You'll see a big difference... Rainer So the global() function must support to do all that is usually required in > a single shot. Thus I would tend to have a single one with different modes. > At least for the prototype, there is another reason, I have to admit: it's > much quicker to write. > > >> If you have multiple statements that use a global variable, it may be >> changed between those statements by other threads, which is why you should >> make a local copy and refer to that. >> > > and it will be changed by *all other message* in the same *batch* within > the *same thread*! Nothing you can prevent with traditional locking. > > Rainer > >> >> and then use them in the message >>>> >>>> >>>> Then you would need something like >>>> >>>> globalReportReset(var='$/****varname') >>>> >>>> >>>> that you would use instead of $/hostcounts in the set above that would >>>> atomically set $.myhostcounts and then destroy all the variables (the >>>> reset >>>> portion) >>>> >>>> The one other thing that I think would end up being needed, is some way >>>> to >>>> do something to access a variable based on a property, something like: >>>> >>>> $.myhostcounts!{$hostname} >>>> >>>> returning the value for the hostnmae of the current message >>>> >>>> >>>> I think I'll make a test implementation of the global() function as I >>> described it. HOWEVER, I am still in the middle of that refactoring for >>> variable names, and for some reason I have not made much progress with >>> that >>> ;) So before coming to the interesting thing (global()). there is a lot >>> more of important boring work in front of me. >>> >> >> this was intended to get ideas out there and have people (including me) >> thinking about options for a bit before we get to the implementation stage. >> >> I think that we have very similar functions in mind, we differ on some >> syntactic things, and I think I've identified a couple corner cases that I >> don't see in your version, but I'll wait until you have time to look at >> things more (and get the current cleanup done) >> >> >> David Lang >> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
