On Wed, 30 Mar 2016, singh.janmejay wrote:
Consider a queue-size, its not really a counter, its a guage. It will
provide point-in-time value of size of the queue. Resetting such a
value will make no sense. Some static metrics we report are guages (so
subsystem supporting the metric must identify it to be resettable or
not).
but that won't work well with the current policy.
you can't have anything as a gauge unless you eliminate the possiblity for any
of the pstats data to be gauges.
If pstats doesn't reset values, then dyn_stats can't implement gauges. If pststs
does reset values, you have turned a lot of things that probably shouldn't reset
into gauges.
it would make far more sense to define this purely on a per-stats basis.
pstats has some items that are always gauges (size/maxsize/etc) and other items
that are allowed to be treated as gauges or as counters.
In my case, I don't want to reset those things (submitted/enqueued) that can be
counters, but that prevents me from having dyn_stats items that are gauges.
David Lang
Yes, restart is a problem. But restarts are really in-frequent enough
to not matter. When they do happen, dx/dt chart shows a big dip, but
people still do track perpetual counters.
Actually its only dyn_inc (we don't have a dyn_dec). But as we go
forward, we may expose capability to support guages and may manipulate
it with dyn_inc and dyn_dec.
Resettable flag in dyn-stats bucket does the same thing as
resettable-flag in statsobj static-counter.
On Tue, Mar 29, 2016 at 3:13 AM, David Lang <[email protected]> wrote:
On Tue, 29 Mar 2016, singh.janmejay wrote:
Well, from counter pov, some counters by nature need to be resettable,
some are actually guages (so resetting makes little sense) and some
are perpetual (so again resetting is not the right thing to do).
can you explain the different categories better? I don't see how any stats
can be 'by nature' one form or another, since it's possible to convert the
data from one form to another.
The idea of perpetual counters doesn't work if the contents of the counters
are lost at restart.
It could be that I didn't read the documentation well enough and missed
references to ways to manipulate them besides dyn_inc and dyn_dec
David Lang
A global control needs to be at impstats level because some users may
want to access data in rate-of-change (dx/dt) form rather than
absolute value form (it can be derived by adding up the numbers and
studying the slope, but its prone to error when some messages are
dropped etc). So user needs to have a say in how counters should be
reported, should they be reset or not.
Same goes for dyn-stats. User may want a perpetual counter for one
dyn-stats bucket (regardless of other buckets being configured to
reset), so its important to allow a particular bucket to be configured
as non-resetting.
However, if user chooses to view all the counters in cumulative form,
the same config that works for static-counters should work for
dyn-stats counters too (so impstats flag is relevant as well).
And'ing is necessary because it doesn't make sense to reset a
by-nature non-resettable counter.
On Tue, Mar 29, 2016 at 12:25 AM, David Lang <[email protected]> wrote:
What is the thinking in having two switches that get ANDed together for
resetting things? I could see two independent controls, or I could see
the
pstats setting controlling everything, but I don't understand two getting
ANDed together.
David Lang
On Tue, 29 Mar 2016, singh.janmejay wrote:
Will add this to the documentation when I touch it for
output-restructuring.
On Tue, Mar 29, 2016 at 12:11 AM, David Lang <[email protected]> wrote:
ahh, that's not at all clear from the documentation.
I turned off resetting the counters at the pstats level because I was
getting soem errors/coredumps from them (quite a few months ago, so it
may
be solved now) I'm currently getting >100 lines of pstats output now,
before
I add dyn_stats to things.
I'll wait for the restructuring of the dyn_stats output and the new
foreach
and then try again.
David Lang
On Mon, 28 Mar 2016, singh.janmejay wrote:
Date: Mon, 28 Mar 2016 23:59:46 +0530
From: singh.janmejay <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] another dyn_stats question
Sorry, I meant and'ed not or'ed.
On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
<[email protected]> wrote:
No, this reset mechanism is no different than the mechanism that
static-counters use. If you turn it off at impstats level, it won't
reset.
This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).
Its or'ed with stats-reporter's choice of resetting or not in this
way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
bResetCtrs comes from that boolean param in impstats.
On Mon, Mar 28, 2016 at 9:58 PM, David Lang <[email protected]> wrote:
the first one should not be resetting, but the rest should be,
correct?
David Lang
dyn_stats(name="message_framing" resettable="off"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_edge_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_core_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_program" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")
On Mon, 28 Mar 2016, singh.janmejay wrote:
Date: Mon, 28 Mar 2016 14:37:12 +0530
From: singh.janmejay <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] another dyn_stats question
David,
You have resetting turned-off in the config you shared with me.
That
would prevent resetting of counters.
On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
<[email protected]> wrote:
That is not supposed to happen. Although very unlikely, it may be
an
environment specific bug. Do all dyn-stats tests pass in your
local
env?
Also, are you using something other than impstats to report it?
On Mar 25, 2016 7:24 PM, "David Lang" <[email protected]> wrote:
resettable counters don't seem to be resetting for me.
David Lang
On Fri, 25 Mar 2016, singh.janmejay wrote:
Well, the bug is metric-ttl ends up killing all metric instead
of
unused-metrics. This works for resettable counters, but for
non-resettable
counters it kills the accumulation prematurely.
The fix will identify which counters are being used and will not
kill
them.
So it won't discard accumulated value as long as it is being
incremented
at
least once within the TTL time.
On Mar 25, 2016 10:53 AM, "David Lang" <[email protected]> wrote:
What I'm seeing is that unless the reset time matches the
pstats
interval,
the data output doesn't reset each pstats report.
David Lang
On Fri, 25 Mar 2016, singh.janmejay wrote:
This is a bug. I'll provide the fix on Monday(or in the worst
case
in
a
few
days, in case I don't get a chance to work on this on Monday).
Basically, I was supposed to have a shadow table to mark used
counters,
and
kill unused counters in next run of metric-ttl-cycle which
wasn't
present
in first cut because I started with resetting counter test
cases
first,
then somehow this slipped my mind.
On Mar 25, 2016 4:12 AM, "David Lang" <[email protected]> wrote:
the unusedMetricLife parameter when setting up a stat says
that
it's
to
purge an unused stat.
But I've got an example where I create about 5 stats in the
bucket
and
what I think I'm seeing is that the stats are cumulative, not
reset
each
time they are dumped out, but they get reset every
unusedMetricLife
seconds.
Is this my imagination? or is the behavior not matching the
documentation?
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED
by
a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST
if
you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED
by
a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST
if
you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by
a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if
you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by
a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if
you
DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if
you
DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T
LIKE THAT.
--
Regards,
Janmejay
http://codehunk.wordpress.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
