Hi all, I just wanted to share my latest state of thinking.
First of all, I got a new idea of how I could keep the old syntax that was used for global variables. As David said, that would definitely be a plus from the doc/howto side of things. But it's to early to talk about specifics (and I'd like to get forward with the refactoring first). However, as there is a chance that the syntax can survive, I will keep the current global var system in 7.5.6, but as an *undocumented* feature (acutally, it will be documented as removed, but old configs will continue to work). The plan is to have a 7.5.7 release as soon as possible which will have the new systems (whatever it is). Note that 7.5.7 will be considerably different and have great regression potential in the variable handling area. Secondly, we should switch from the name "global variables". This name implies very much to everyone who has done at least a little bit of programming. Most of these implicatons are not true in our context. So I suggest we call it "state variables" -- or any better name. For now, I will use "state variables". Then, in order to find a solution, we need to have some correctness criterion. As such, I need to write up the restrictions that apply to state variables. This needs to be thought out, and I can't do it immediately. One thing I already know, and it is the most important restriction: State variables cannot be modified more than once when processing a specific *message*. This means multiple statements with modifications are permitted, but only one of them can be executed during ruleset evaluation. An example (pseudo-code): if $msg contains "tag1" then set $/glob = "1"; if $msg contains "tag2" then set $/glob = "2"; This code will execute correctly if msg contains either "tag1", "trag2" or none of them, but will trigger an runtime-error if msg contains both "tag1" and "tag2" (as it violates the constraint). Please note that this constraint is close to what Pavel said: "a global variable cannot be used more than once", but is a relaxed condition, as it permits multiple use, and only restricts multiple updates. Comments on this restriction are very welcome. Please focus the discussion on the restriction and NOT on the fact if or how it could be implemented. I don't want to spend time on talking implementation as long as we do not know if the restriction itself is suitable (but obviously I have a weak idea how it could be implemented). Once we got the restrictions documented, we can actually think about the state variable system. If all turns out well (I am NOT yet saying it does!), we could end up with something that looks much like the 7.5.4 global variable system, but with known restrictions attached to it that make it actually work ;) Rainer On Tue, Oct 22, 2013 at 6:12 PM, Pavel Levshin <[email protected]> wrote: > > 22.10.2013 17:54, David Lang: > > > Then, there is need for each global variable to be lockable. But then, it >>> can lead to a deadlocks... >>> >> >> Each global variable needs to eithe be lockable or manipulated with >> atomic statements. >> > > And if we talk about global containers which can contain properties, it > becomes even harder. > > > Rainer is saying that he thinks they may be able to be used in if >> statements, you already can't use functions in templates, so I don't see >> that as a problem. >> >> Let's see, I would say. > > I think that, given current SIMD engine, the problem arises every time > when a global variable is used more than once for one message. But how to > prevent this? Having global(key="$/var") as a function leads to natural > expressions like this: > > if global(key="$/var") % 3 == 0 then > Action(1) > else if global(key="$/var") % 3 == 1 then > Action(2) > else > Action(3) > > But, obviously, it is not better than having $/ directly. > > So, what to do? > > We can force user to export global value to userspace, having a way to > access globals, but not using them in expressions. By an action, for > example: > > action(type="mmglobal" mode="get" global="$/var" var=".counter") > > It is still not prevents a user to access globals twice, but makes it > unnatural. (And it can even be useful to count messages in a batch.) > > > And one more idea. What if engine caches global value to message space on > first access, then all subsequent reads of the same global are served > locally? It seemes facilitating main issue with batching. Locking and > atomicity are still problematic and need to be resolved, but for counters > it can be easily done with atomic_increment(). But... If batch size is 10: > > set $/var = $/var + 1; > set $!var = $/var; > > It is OK so far: each message will get it's number, from 1 to 10, and > global is +10. > > But what if we change order: > > set $!var = $/var; > set $/var = $/var + 1; > > Ooops, it will be strange. Every message will get the same number, and > global will increment by 1. No, this idea is not as good. Just for the > record. > > > -- > Pavel Levshin > > > ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> > http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
