Well yes, its a bug for sure. But it doesn't affect single thread usage (considering even with multi-threaded execution, its incorrect only once in a while (typical of a data-race)).
And it causes crashes with a very low probability (like once in tens of hours), in multi-threaded setup. My question is, - Is it designed to be single-threaded (and Im using it incorrectly with multiple threads)? - Or is it designed to be multi-threaded in which case it was supposed to work, but it has a bug, and we should just fix it? Depending on which one of the two this case is, we have those 5 possible ways to fixing this. On Thu, May 21, 2015 at 10:58 PM, David Lang <[email protected]> wrote: > On Thu, 21 May 2015, singh.janmejay wrote: > >> Are $/ variable manipulations supposed to be atomic? >> >> Eg. $/var % 16 should never produce any number greater than 15, but it >> does. > > > atomic or not, this is an invalid calculation. I would call it a bug. > > David Lang > > >> Are they non-atomic by design? If so, is mmsequence the right thing to >> use (assuming it has atomic counters, I haven't checked if it really >> does)? >> >> There are few options on the table, I guess. >> >> 1. If $/ variables are supposed to be atomic, we just fix the impl. >> 2. We use mmsequence, and keep it alive (assuming it has atomic counters) >> 3. We fix mmsequence to implement atomic counters. >> 4. If $/ is not supposed to be atomic (by design), we implement $@ >> which is thread-local, so people can get atomic counters within thread >> (evenness in data(for load-balancing or sampling kind of usecase) is >> maintained by the fact that a single thread will pump a lot of >> messages, so across all threads it'll approximately be as even as it >> is in a single-thread. >> 5. We implement a random-number construct (which i believe either >> mmsequence or mmcount had) which allows acceptable load-balancing or >> sampling etc. >> >> Thoughts? >> >> On Wed, Dec 3, 2014 at 11:12 PM, David Lang <[email protected]> wrote: >>> >>> On Wed, 3 Dec 2014, Brian Knox wrote: >>> >>>> So two questions then: >>>> >>>> 1. In the case of using mmjsonparse on messages where we do not know >>>> ahead >>>> of time what all the. json attributes will be, is there a var for a >>>> place >>>> in the tree where we can get all attributes that were in the cee >>>> message? >>> >>> >>> >>> There are three 'classes' of variables >>> >>> $! are 'normal' variables, the place that mmjsonparse, etc put the >>> results >>> of parsing the cee message by default. >>> >>> $. are 'local' variables, they are a place you can put stuff that you >>> don't >>> want to end up in a message when you output $! >>> >>> $\ are 'global' variables, unlike the others, they persist from message >>> to >>> message. >>> >>> All of these variables are a tree, where you can have $!foo!bar and >>> $!foo!baz. If you just reference $!foo it will output both bar and baz as >>> JSON >>> >>> so to see everything, parse the message and then output $! >>> >>>> 2. If mmsequence is being deprecated, is there a pointer to how to >>>> increment global variables to provide a sequence? >>> >>> >>> >>> set $\var = $\var + 1; >>> >>>> 3. This morning I remembered the "mmcount" module - is this also being >>>> deprecated in favor of global variables? >>> >>> >>> >>> probably. >>> >>> David Lang >>> >>> >>>> Brian >>>> >>>> On Wed, Dec 3, 2014 at 3:52 AM, Rainer Gerhards >>>> <[email protected]> >>>> wrote: >>>> >>>>> 2014-12-02 21:59 GMT+01:00 Brian Knox <[email protected]>: >>>>> >>>>>> We have a case where we are receiving messages in @cee: format, and >>>>>> would >>>>>> like to add a sequence variable generated via mmsequence to the >>>>>> message. >>>>>> >>>>>> Before I spend too much time digging into this I wanted to ask if, >>>>>> using >>>>>> some template hackery and json subtrees, this is something that sounds >>>>>> possible - and if maybe someone else was already doing it. >>>>>> >>>>>> >>>>> David has answered the real question, just let me add that mmsequence >>>>> was >>>>> a >>>>> work-around for v7 where global variables were not available. It is now >>>>> deprecated and will probably removed in a couple of month. >>>>> >>>>> I'll check the doc if it tells this clearly enough. >>>>> >>>>> Rainer >>>>> >>>>>> Thanks! >>>>>> Brian >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com/professional-services/ >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>> myriad >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>> DON'T LIKE THAT. >>>>>> >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T >>>> LIKE THAT. >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>> LIKE THAT. >> >> >> >> >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. -- Regards, Janmejay http://codehunk.wordpress.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
