I'll create 2 implementations for this. Will first build it with hash-table (which will use exclusive lock only when adding new metric to the table, increments will happen with shared-lock).
In the second cut, I'll change it to trie (which is what I discussed in the proposal). After benchmarking both we'll be in a better position to decide which one should be kept. On Fri, Oct 9, 2015 at 3:20 AM, singh.janmejay <[email protected]> wrote: > On Thu, Oct 8, 2015 at 11:07 PM, David Lang <[email protected]> wrote: >> Atomic ops are actually rather expensive (almost as expsnsive as full >> locks). If you want a lockless metrics capability, you should do a separate >> set of variables per thread, gathering them a reporting time. And document >> that there is going to be inconsistancy between the different metrics > > Yes, the write thing to do is to maintain thread-local counters. It'll > require slightly wider set of changes for that, but the approach > allows us to make those improvements later without changing the > config-interface. > > Cost of atomic-ops is close to uncontended lock, but much lower than > contended lock. > >> >> unless you lock everything, you are going to have inconsistancies across the >> different metrics. > > Yes, but isn't that acceptable in most cases? > >> >> Also, not all platforms have the easy atomic ops you are thinking of, and >> atomic ops can't operate on all sizes of data (for example, 32 bit systems >> probably can't update a 64 bit value atomically) > > We should eventually move to thread-level accumulators for all metrics > (static or dynamic). > >> >> David Lang >> >> >> On Thu, 8 Oct 2015, singh.janmejay wrote: >> >>> Did you mean it's not atomic across different metrics? That I think should >>> be acceptable. A single metric however should get swapped losslessly with >>> atomic-swap. Either an increment of m should be applied before swap >>> (making >>> the reading n + m, or after it leaving the accumulator at m and reading at >>> n). But it should not be lost. >>> >>> Am I misunderstanding something? >>> >>> -- >>> Regards, >>> Janmejay >>> >>> PS: Please blame the typos in this mail on my phone's uncivilized soft >>> keyboard sporting it's not-so-smart-assist technology. >>> >>> On Oct 8, 2015 12:33 PM, "Rainer Gerhards" <[email protected]> >>> wrote: >>> >>>> 2015-10-08 8:30 GMT+02:00 singh.janmejay <[email protected]>: >>>>>> >>>>>> Similarly, when one thread goes to output the stats, you need to lock >>>>> >>>>> them so that there isn't a lost increment between the time that you read >>>>> the stat and the time you zero it. >>>>> >>>>> No, this involves the same shared (uncontended) lock, except >>>>> atomic-increment is replaced by atomic-swap with 0. >>>>> >>>> >>>> Just FYI: this is what the current stats system also does. It is also >>>> where some inaccuracy stems from. Reporting stats is not atomic >>>> without looks, so a stats counter may be read with value n, then m >>>> atomic increments happen to it on another thread, then value n is >>>> being reported (but we are really at n+m) and then the stats counter >>>> is reset to 0 via an atomic swap. So m updates are lost. IMHO this is >>>> perfectly acceptable, because otherwise we would lose almost all >>>> concurrency. >>>> >>>> Rainer >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>> LIKE THAT. >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. > > > > -- > Regards, > Janmejay > http://codehunk.wordpress.com -- Regards, Janmejay http://codehunk.wordpress.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
