Consider a queue-size, its not really a counter, its a guage. It will provide point-in-time value of size of the queue. Resetting such a value will make no sense. Some static metrics we report are guages (so subsystem supporting the metric must identify it to be resettable or not).
Yes, restart is a problem. But restarts are really in-frequent enough to not matter. When they do happen, dx/dt chart shows a big dip, but people still do track perpetual counters. Actually its only dyn_inc (we don't have a dyn_dec). But as we go forward, we may expose capability to support guages and may manipulate it with dyn_inc and dyn_dec. Resettable flag in dyn-stats bucket does the same thing as resettable-flag in statsobj static-counter. On Tue, Mar 29, 2016 at 3:13 AM, David Lang <[email protected]> wrote: > On Tue, 29 Mar 2016, singh.janmejay wrote: > >> Well, from counter pov, some counters by nature need to be resettable, >> some are actually guages (so resetting makes little sense) and some >> are perpetual (so again resetting is not the right thing to do). > > > can you explain the different categories better? I don't see how any stats > can be 'by nature' one form or another, since it's possible to convert the > data from one form to another. > > The idea of perpetual counters doesn't work if the contents of the counters > are lost at restart. > > It could be that I didn't read the documentation well enough and missed > references to ways to manipulate them besides dyn_inc and dyn_dec > > David Lang > > >> A global control needs to be at impstats level because some users may >> want to access data in rate-of-change (dx/dt) form rather than >> absolute value form (it can be derived by adding up the numbers and >> studying the slope, but its prone to error when some messages are >> dropped etc). So user needs to have a say in how counters should be >> reported, should they be reset or not. >> >> Same goes for dyn-stats. User may want a perpetual counter for one >> dyn-stats bucket (regardless of other buckets being configured to >> reset), so its important to allow a particular bucket to be configured >> as non-resetting. >> >> However, if user chooses to view all the counters in cumulative form, >> the same config that works for static-counters should work for >> dyn-stats counters too (so impstats flag is relevant as well). >> >> And'ing is necessary because it doesn't make sense to reset a >> by-nature non-resettable counter. >> >> >> On Tue, Mar 29, 2016 at 12:25 AM, David Lang <[email protected]> wrote: >>> >>> What is the thinking in having two switches that get ANDed together for >>> resetting things? I could see two independent controls, or I could see >>> the >>> pstats setting controlling everything, but I don't understand two getting >>> ANDed together. >>> >>> David Lang >>> >>> >>> On Tue, 29 Mar 2016, singh.janmejay wrote: >>> >>>> Will add this to the documentation when I touch it for >>>> output-restructuring. >>>> >>>> On Tue, Mar 29, 2016 at 12:11 AM, David Lang <[email protected]> wrote: >>>>> >>>>> >>>>> ahh, that's not at all clear from the documentation. >>>>> >>>>> I turned off resetting the counters at the pstats level because I was >>>>> getting soem errors/coredumps from them (quite a few months ago, so it >>>>> may >>>>> be solved now) I'm currently getting >100 lines of pstats output now, >>>>> before >>>>> I add dyn_stats to things. >>>>> >>>>> I'll wait for the restructuring of the dyn_stats output and the new >>>>> foreach >>>>> and then try again. >>>>> >>>>> David Lang >>>>> >>>>> On Mon, 28 Mar 2016, singh.janmejay wrote: >>>>> >>>>>> Date: Mon, 28 Mar 2016 23:59:46 +0530 >>>>>> >>>>>> From: singh.janmejay <[email protected]> >>>>>> Reply-To: rsyslog-users <[email protected]> >>>>>> To: rsyslog-users <[email protected]> >>>>>> Subject: Re: [rsyslog] another dyn_stats question >>>>>> >>>>>> Sorry, I meant and'ed not or'ed. >>>>>> >>>>>> On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay >>>>>> <[email protected]> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> No, this reset mechanism is no different than the mechanism that >>>>>>> static-counters use. If you turn it off at impstats level, it won't >>>>>>> reset. >>>>>>> >>>>>>> This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h). >>>>>>> >>>>>>> Its or'ed with stats-reporter's choice of resetting or not in this >>>>>>> way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where >>>>>>> bResetCtrs comes from that boolean param in impstats. >>>>>>> >>>>>>> On Mon, Mar 28, 2016 at 9:58 PM, David Lang <[email protected]> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> the first one should not be resetting, but the rest should be, >>>>>>>> correct? >>>>>>>> >>>>>>>> David Lang >>>>>>>> >>>>>>>> >>>>>>>> dyn_stats(name="message_framing" resettable="off" >>>>>>>> maxCardinality="3000" >>>>>>>> unusedMetricLife="1200") >>>>>>>> dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000" >>>>>>>> unusedMetricLife="1200") >>>>>>>> dyn_stats(name="msgs_per_edge_relay" resettable="on" >>>>>>>> maxCardinality="3000" >>>>>>>> unusedMetricLife="1200") >>>>>>>> dyn_stats(name="msgs_per_core_relay" resettable="on" >>>>>>>> maxCardinality="3000" >>>>>>>> unusedMetricLife="1200") >>>>>>>> dyn_stats(name="msgs_per_program" resettable="on" >>>>>>>> maxCardinality="3000" >>>>>>>> unusedMetricLife="1200") >>>>>>>> dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000" >>>>>>>> unusedMetricLife="1200") >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, 28 Mar 2016, singh.janmejay wrote: >>>>>>>> >>>>>>>>> Date: Mon, 28 Mar 2016 14:37:12 +0530 >>>>>>>>> From: singh.janmejay <[email protected]> >>>>>>>>> Reply-To: rsyslog-users <[email protected]> >>>>>>>>> To: rsyslog-users <[email protected]> >>>>>>>>> Subject: Re: [rsyslog] another dyn_stats question >>>>>>>>> >>>>>>>>> >>>>>>>>> David, >>>>>>>>> >>>>>>>>> You have resetting turned-off in the config you shared with me. >>>>>>>>> That >>>>>>>>> would prevent resetting of counters. >>>>>>>>> >>>>>>>>> On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay >>>>>>>>> <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> That is not supposed to happen. Although very unlikely, it may be >>>>>>>>>> an >>>>>>>>>> environment specific bug. Do all dyn-stats tests pass in your >>>>>>>>>> local >>>>>>>>>> env? >>>>>>>>>> >>>>>>>>>> Also, are you using something other than impstats to report it? >>>>>>>>>> >>>>>>>>>> On Mar 25, 2016 7:24 PM, "David Lang" <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> resettable counters don't seem to be resetting for me. >>>>>>>>>>> >>>>>>>>>>> David Lang >>>>>>>>>>> >>>>>>>>>>> On Fri, 25 Mar 2016, singh.janmejay wrote: >>>>>>>>>>> >>>>>>>>>>>> Well, the bug is metric-ttl ends up killing all metric instead >>>>>>>>>>>> of >>>>>>>>>>>> unused-metrics. This works for resettable counters, but for >>>>>>>>>>>> non-resettable >>>>>>>>>>>> counters it kills the accumulation prematurely. >>>>>>>>>>>> >>>>>>>>>>>> The fix will identify which counters are being used and will not >>>>>>>>>>>> kill >>>>>>>>>>>> them. >>>>>>>>>>>> So it won't discard accumulated value as long as it is being >>>>>>>>>>>> incremented >>>>>>>>>>>> at >>>>>>>>>>>> least once within the TTL time. >>>>>>>>>>>> On Mar 25, 2016 10:53 AM, "David Lang" <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> What I'm seeing is that unless the reset time matches the >>>>>>>>>>>>> pstats >>>>>>>>>>>>> interval, >>>>>>>>>>>>> the data output doesn't reset each pstats report. >>>>>>>>>>>>> >>>>>>>>>>>>> David Lang >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, 25 Mar 2016, singh.janmejay wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> This is a bug. I'll provide the fix on Monday(or in the worst >>>>>>>>>>>>> case >>>>>>>>>>>>> in >>>>>>>>>>>>> a >>>>>>>>>>>>> few >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> days, in case I don't get a chance to work on this on Monday). >>>>>>>>>>>>>> >>>>>>>>>>>>>> Basically, I was supposed to have a shadow table to mark used >>>>>>>>>>>>>> counters, >>>>>>>>>>>>>> and >>>>>>>>>>>>>> kill unused counters in next run of metric-ttl-cycle which >>>>>>>>>>>>>> wasn't >>>>>>>>>>>>>> present >>>>>>>>>>>>>> in first cut because I started with resetting counter test >>>>>>>>>>>>>> cases >>>>>>>>>>>>>> first, >>>>>>>>>>>>>> then somehow this slipped my mind. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Mar 25, 2016 4:12 AM, "David Lang" <[email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> the unusedMetricLife parameter when setting up a stat says >>>>>>>>>>>>>> that >>>>>>>>>>>>>> it's >>>>>>>>>>>>>> to >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> purge an unused stat. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> But I've got an example where I create about 5 stats in the >>>>>>>>>>>>>>> bucket >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>> what I think I'm seeing is that the stats are cumulative, not >>>>>>>>>>>>>>> reset >>>>>>>>>>>>>>> each >>>>>>>>>>>>>>> time they are dumped out, but they get reset every >>>>>>>>>>>>>>> unusedMetricLife >>>>>>>>>>>>>>> seconds. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is this my imagination? or is the behavior not matching the >>>>>>>>>>>>>>> documentation? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> David Lang >>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>> rsyslog mailing list >>>>>>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >>>>>>>>>>>>>>> by >>>>>>>>>>>>>>> a >>>>>>>>>>>>>>> myriad >>>>>>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>>>>>>>> POST >>>>>>>>>>>>>>> if >>>>>>>>>>>>>>> you >>>>>>>>>>>>>>> DON'T LIKE THAT. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> rsyslog mailing list >>>>>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >>>>>>>>>>>>>> by >>>>>>>>>>>>>> a >>>>>>>>>>>>>> myriad >>>>>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>>>>>>> POST >>>>>>>>>>>>>> if >>>>>>>>>>>>>> you >>>>>>>>>>>>>> DON'T LIKE THAT. >>>>>>>>>>>>>> >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> rsyslog mailing list >>>>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >>>>>>>>>>>>> a >>>>>>>>>>>>> myriad >>>>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>>>>>>>> if >>>>>>>>>>>>> you >>>>>>>>>>>>> DON'T LIKE THAT. >>>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> rsyslog mailing list >>>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >>>>>>>>>>>> a >>>>>>>>>>>> myriad >>>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>>>>>>> if >>>>>>>>>>>> you >>>>>>>>>>>> DON'T >>>>>>>>>>>> LIKE THAT. >>>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> rsyslog mailing list >>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>>>> myriad >>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>>>>>> if >>>>>>>>>>> you >>>>>>>>>>> DON'T >>>>>>>>>>> LIKE THAT. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> rsyslog mailing list >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>> myriad >>>>>>>> of >>>>>>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>>>> DON'T >>>>>>>> LIKE THAT. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Janmejay >>>>>>> http://codehunk.wordpress.com >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of >>>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T >>>>> LIKE THAT. >>>> >>>> >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>> LIKE THAT. >> >> >> >> >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. -- Regards, Janmejay http://codehunk.wordpress.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
