Re: [rsyslog] another dyn_stats question

[singh.janmejay]

Here is the PR for unused-metric-ttl bug:
https://github.com/rsyslog/rsyslog/pull/955

On Wed, Apr 6, 2016 at 8:35 AM, David Lang <da...@lang.hm> wrote:
> looking pretty good. I've been running it for a couple days now.
>
> David Lang
>
> On Thu, 31 Mar 2016, singh.janmejay wrote:
>
>> Date: Thu, 31 Mar 2016 00:55:44 +0530
>>
>> From: singh.janmejay <singh.janme...@gmail.com>
>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] another dyn_stats question
>>
>> Have opened a PR for this. Will fold it with other changes in the same
>> area. Here is the PR: https://github.com/rsyslog/rsyslog/pull/931 (not
>> ready for merge yet)
>>
>> On Wed, Mar 30, 2016 at 8:01 AM, singh.janmejay
>> <singh.janme...@gmail.com> wrote:
>>>
>>> Cool, I'll make dyn-stats bucket resettable flag the only thing that
>>> controls resetting of its accumulators. It will or won't reset regardless
>>> of
>>> impstats reset flag value.
>>>
>>> This is again changing behavior from previous release but is a fairly
>>> minor
>>> thing, so we'll just document it and not worry about backward
>>> compatibility
>>> of its behavior.
>>>
>>> On Mar 30, 2016 2:55 AM, "David Lang" <da...@lang.hm> wrote:
>>>>
>>>>
>>>> On Wed, 30 Mar 2016, singh.janmejay wrote:
>>>>
>>>>> Metric owner is the subsystem (action is a perfect example). End-user
>>>>> is
>>>>> author of the config.
>>>>
>>>>
>>>>
>>>> Except that dyn_stats doen't exist except in the config, so it's the
>>>> "End-user" who sets everything about them.
>>>>
>>>>> In this case ideal situation would be to let action decide which of its
>>>>> metrics are resettable, and let end-user decide which of the resettable
>>>>> metric accumulators(subset of all metrics) do they actually want to
>>>>> reset.
>>>>
>>>>
>>>>
>>>> I disagree, the module author should not be deciding if something is
>>>> resettable or not.
>>>>
>>>> The module author is either reporting the current state of something
>>>> (like
>>>> queue length) where there is no resetting, or they are reporting the
>>>> state
>>>> of a counter (number of messages processed), and it should be up to the
>>>> config author if they want to reset this every reporting period or not.
>>>>
>>>>> In case of dyn-stats, end-user is metric-owner.
>>>>>
>>>>> So we should probably disassociate from impstats reset flag and let
>>>>> bucket-level resettable flag be the only way to control it.
>>>>>
>>>>> That is what you expected the behavior would be, right? So it's more
>>>>> intuitive too.
>>>>>
>>>>> Should we do this?
>>>>
>>>>
>>>>
>>>> I think so.
>>>>
>>>>> For static counters though, the problem still remains. If you want to
>>>>> reset
>>>>> omkafka resettable counters, but don't want resetting for imudp
>>>>> resettable
>>>>> counters, as of now we have no way to configure it like that.
>>>>
>>>>
>>>>
>>>> Well, I see that as a legacy interface, so I don't think we need to
>>>> worry
>>>> about changing that.
>>>>
>>>> Monitoring systems are almost always able to handle counters and convert
>>>> them to rate-of-change.
>>>>
>>>> resetting a counter is always racy with updates.
>>>>
>>>> If a monitoring system is looking at the stats, you almost always want
>>>> them to be non-reset counters. It's only if a person is looking at them
>>>> that
>>>> you want to show the delta since the last report in the output.
>>>>
>>>> David Lang
>>>>
>>>>> On Mar 30, 2016 2:06 AM, "David Lang" <da...@lang.hm> wrote:
>>>>>
>>>>>> On Wed, 30 Mar 2016, singh.janmejay wrote:
>>>>>>
>>>>>> The problem is, pstats level control is an all or nothing kind of
>>>>>>>
>>>>>>>
>>>>>>> arrangement.
>&g

Re: [rsyslog] another dyn_stats question

[David Lang]

looking pretty good. I've been running it for a couple days now.

David Lang

On Thu, 31 Mar 2016, singh.janmejay wrote:


Date: Thu, 31 Mar 2016 00:55:44 +0530
From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question

Have opened a PR for this. Will fold it with other changes in the same
area. Here is the PR: https://github.com/rsyslog/rsyslog/pull/931 (not
ready for merge yet)

On Wed, Mar 30, 2016 at 8:01 AM, singh.janmejay
<singh.janme...@gmail.com> wrote:

Cool, I'll make dyn-stats bucket resettable flag the only thing that
controls resetting of its accumulators. It will or won't reset regardless of
impstats reset flag value.

This is again changing behavior from previous release but is a fairly minor
thing, so we'll just document it and not worry about backward compatibility
of its behavior.

On Mar 30, 2016 2:55 AM, "David Lang" <da...@lang.hm> wrote:


On Wed, 30 Mar 2016, singh.janmejay wrote:


Metric owner is the subsystem (action is a perfect example). End-user is
author of the config.



Except that dyn_stats doen't exist except in the config, so it's the
"End-user" who sets everything about them.


In this case ideal situation would be to let action decide which of its
metrics are resettable, and let end-user decide which of the resettable
metric accumulators(subset of all metrics) do they actually want to
reset.



I disagree, the module author should not be deciding if something is
resettable or not.

The module author is either reporting the current state of something (like
queue length) where there is no resetting, or they are reporting the state
of a counter (number of messages processed), and it should be up to the
config author if they want to reset this every reporting period or not.


In case of dyn-stats, end-user is metric-owner.

So we should probably disassociate from impstats reset flag and let
bucket-level resettable flag be the only way to control it.

That is what you expected the behavior would be, right? So it's more
intuitive too.

Should we do this?



I think so.


For static counters though, the problem still remains. If you want to
reset
omkafka resettable counters, but don't want resetting for imudp
resettable
counters, as of now we have no way to configure it like that.



Well, I see that as a legacy interface, so I don't think we need to worry
about changing that.

Monitoring systems are almost always able to handle counters and convert
them to rate-of-change.

resetting a counter is always racy with updates.

If a monitoring system is looking at the stats, you almost always want
them to be non-reset counters. It's only if a person is looking at them that
you want to show the delta since the last report in the output.

David Lang


On Mar 30, 2016 2:06 AM, "David Lang" <da...@lang.hm> wrote:


On Wed, 30 Mar 2016, singh.janmejay wrote:

The problem is, pstats level control is an all or nothing kind of


arrangement.

Ideal setup would be to let metric-owner decide if metric should be
resettable or not(which allows choosing if its a guage or not) and
then let end-user choose if they really want to reset each resettable
metric(counter) (or do they want it as a perpetual counter).



there is only one entity in control of the rsyslog.conf file, how do you
get to 'metric-owner' vs 'end-user'?

David Lang

In our case, the end-user control is very coarse-grained, where one


can either get all counters as perpetual or all as resetting.

On Wed, Mar 30, 2016 at 1:41 AM, David Lang <da...@lang.hm> wrote:


On Wed, 30 Mar 2016, singh.janmejay wrote:

Consider a queue-size, its not really a counter, its a guage. It will


provide point-in-time value of size of the queue. Resetting such a
value will make no sense. Some static metrics we report are guages
(so
subsystem supporting the metric must identify it to be resettable or
not).




but that won't work well with the current policy.

you can't have anything as a gauge unless you eliminate the possiblity
for
any of the pstats data to be gauges.

If pstats doesn't reset values, then dyn_stats can't implement gauges.
If
pststs does reset values, you have turned a lot of things that
probably
shouldn't reset into gauges.

it would make far more sense to define this purely on a per-stats
basis.

pstats has some items that are always gauges (size/maxsize/etc) and
other
items that are allowed to be treated as gauges or as counters.

In my case, I don't want to reset those things (submitted/enqueued)
that
can
be counters, but that prevents me from having dyn_stats items that are
gauges.

David Lang


Yes, restart is a problem. But restarts are really in-frequent enough


to not matter. When they do happen, dx/dt chart shows a big dip, but
people still do track perpetual counters.

Actually its only dyn_inc (we don't ha

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

ce it's possible to
>>>>>>>>>> convert
>>>>>>>>>> the
>>>>>>>>>> data from one form to another.
>>>>>>>>>>
>>>>>>>>>> The idea of perpetual counters doesn't work if the contents of the
>>>>>>>>>> counters
>>>>>>>>>> are lost at restart.
>>>>>>>>>>
>>>>>>>>>> It could be that I didn't read the documentation well enough and
>>>>>>>>>> missed
>>>>>>>>>> references to ways to manipulate them besides dyn_inc and dyn_dec
>>>>>>>>>>
>>>>>>>>>> David Lang
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> A global control needs to be at impstats level because some users
>>>>>>>>>> may
>>>>>>>>>>>
>>>>>>>>>>> want to access data in rate-of-change (dx/dt) form rather than
>>>>>>>>>>> absolute value form (it can be derived by adding up the numbers and
>>>>>>>>>>> studying the slope, but its prone to error when some messages are
>>>>>>>>>>> dropped etc). So user needs to have a say in how counters should be
>>>>>>>>>>> reported, should they be reset or not.
>>>>>>>>>>>
>>>>>>>>>>> Same goes for dyn-stats. User may want a perpetual counter for one
>>>>>>>>>>> dyn-stats bucket (regardless of other buckets being configured to
>>>>>>>>>>> reset), so its important to allow a particular bucket to be
>>>>>>>>>>> configured
>>>>>>>>>>> as non-resetting.
>>>>>>>>>>>
>>>>>>>>>>> However, if user chooses to view all the counters in cumulative
>>>>>>>>>>> form,
>>>>>>>>>>> the same config that works for static-counters should work for
>>>>>>>>>>> dyn-stats counters too (so impstats flag is relevant as well).
>>>>>>>>>>>
>>>>>>>>>>> And'ing is necessary because it doesn't make sense to reset a
>>>>>>>>>>> by-nature non-resettable counter.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 29, 2016 at 12:25 AM, David Lang <da...@lang.hm> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> What is the thinking in having two switches that get ANDed
>>>>>>>>>>>> together
>>>>>>>>>>>> for
>>>>>>>>>>>> resetting things? I could see two independent controls, or I could
>>>>>>>>>>>> see
>>>>>>>>>>>> the
>>>>>>>>>>>> pstats setting controlling everything, but I don't understand two
>>>>>>>>>>>> getting
>>>>>>>>>>>> ANDed together.
>>>>>>>>>>>>
>>>>>>>>>>>> David Lang
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, 29 Mar 2016, singh.janmejay wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Will add this to the documentation when I touch it for
>>>>>>>>>>>>>
>>>>>>>>>>>>> output-restructuring.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ahh, that's not at all clear from the documentation.
>>>>>>>

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

>>>>>
>>>>>>> Consider a queue-size, its not really a counter, its a guage. It will
>>>>>>>>
>>>>>>>> provide point-in-time value of size of the queue. Resetting such a
>>>>>>>> value will make no sense. Some static metrics we report are guages
>>>>>>>> (so
>>>>>>>> subsystem supporting the metric must identify it to be resettable or
>>>>>>>> not).
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> but that won't work well with the current policy.
>>>>>>>
>>>>>>> you can't have anything as a gauge unless you eliminate the possiblity
>>>>>>> for
>>>>>>> any of the pstats data to be gauges.
>>>>>>>
>>>>>>> If pstats doesn't reset values, then dyn_stats can't implement gauges.
>>>>>>> If
>>>>>>> pststs does reset values, you have turned a lot of things that
>>>>>>> probably
>>>>>>> shouldn't reset into gauges.
>>>>>>>
>>>>>>> it would make far more sense to define this purely on a per-stats
>>>>>>> basis.
>>>>>>>
>>>>>>> pstats has some items that are always gauges (size/maxsize/etc) and
>>>>>>> other
>>>>>>> items that are allowed to be treated as gauges or as counters.
>>>>>>>
>>>>>>> In my case, I don't want to reset those things (submitted/enqueued)
>>>>>>> that
>>>>>>> can
>>>>>>> be counters, but that prevents me from having dyn_stats items that are
>>>>>>> gauges.
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>>
>>>>>>> Yes, restart is a problem. But restarts are really in-frequent enough
>>>>>>>>
>>>>>>>> to not matter. When they do happen, dx/dt chart shows a big dip, but
>>>>>>>> people still do track perpetual counters.
>>>>>>>>
>>>>>>>> Actually its only dyn_inc (we don't have a dyn_dec). But as we go
>>>>>>>> forward, we may expose capability to support guages and may
>>>>>>>> manipulate
>>>>>>>> it with dyn_inc and dyn_dec.
>>>>>>>>
>>>>>>>> Resettable flag in dyn-stats bucket does the same thing as
>>>>>>>> resettable-flag in statsobj static-counter.
>>>>>>>>
>>>>>>>> On Tue, Mar 29, 2016 at 3:13 AM, David Lang <da...@lang.hm> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, 29 Mar 2016, singh.janmejay wrote:
>>>>>>>>>
>>>>>>>>> Well, from counter pov, some counters by nature need to be
>>>>>>>>> resettable,
>>>>>>>>>>
>>>>>>>>>> some are actually guages (so resetting makes little sense) and some
>>>>>>>>>> are perpetual (so again resetting is not the right thing to do).
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> can you explain the different categories better? I don't see how any
>>>>>>>>> stats
>>>>>>>>> can be 'by nature' one form or another, since it's possible to
>>>>>>>>> convert
>>>>>>>>> the
>>>>>>>>> data from one form to another.
>>>>>>>>>
>>>>>>>>> The idea of perpetual counters doesn't work if the contents of the
>>>>>>>>> counters
>>>>>>>>> are lost at restart.
>>>>>>>>>
>>>>>>>>> It could be that I didn't read the documentation well enough and
>>>>>>>>> missed
>>>>>>>>> references to ways to manipulate them besides dyn_inc and dyn_dec
>>>>>>>>>
>>>>>>>>> David Lang
>>>>>>>>>
>>>>>>>>>
>>>>&

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

t;>>>>>
>>>>>>
>>>>>>
>>>>>> but that won't work well with the current policy.
>>>>>>
>>>>>> you can't have anything as a gauge unless you eliminate the possiblity
>>>>>> for
>>>>>> any of the pstats data to be gauges.
>>>>>>
>>>>>> If pstats doesn't reset values, then dyn_stats can't implement gauges.
>>>>>> If
>>>>>> pststs does reset values, you have turned a lot of things that
>>>>>> probably
>>>>>> shouldn't reset into gauges.
>>>>>>
>>>>>> it would make far more sense to define this purely on a per-stats
>>>>>> basis.
>>>>>>
>>>>>> pstats has some items that are always gauges (size/maxsize/etc) and
>>>>>> other
>>>>>> items that are allowed to be treated as gauges or as counters.
>>>>>>
>>>>>> In my case, I don't want to reset those things (submitted/enqueued)
>>>>>> that
>>>>>> can
>>>>>> be counters, but that prevents me from having dyn_stats items that are
>>>>>> gauges.
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>>
>>>>>> Yes, restart is a problem. But restarts are really in-frequent enough
>>>>>>>
>>>>>>> to not matter. When they do happen, dx/dt chart shows a big dip, but
>>>>>>> people still do track perpetual counters.
>>>>>>>
>>>>>>> Actually its only dyn_inc (we don't have a dyn_dec). But as we go
>>>>>>> forward, we may expose capability to support guages and may
>>>>>>> manipulate
>>>>>>> it with dyn_inc and dyn_dec.
>>>>>>>
>>>>>>> Resettable flag in dyn-stats bucket does the same thing as
>>>>>>> resettable-flag in statsobj static-counter.
>>>>>>>
>>>>>>> On Tue, Mar 29, 2016 at 3:13 AM, David Lang <da...@lang.hm> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, 29 Mar 2016, singh.janmejay wrote:
>>>>>>>>
>>>>>>>> Well, from counter pov, some counters by nature need to be
>>>>>>>> resettable,
>>>>>>>>>
>>>>>>>>> some are actually guages (so resetting makes little sense) and some
>>>>>>>>> are perpetual (so again resetting is not the right thing to do).
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> can you explain the different categories better? I don't see how any
>>>>>>>> stats
>>>>>>>> can be 'by nature' one form or another, since it's possible to
>>>>>>>> convert
>>>>>>>> the
>>>>>>>> data from one form to another.
>>>>>>>>
>>>>>>>> The idea of perpetual counters doesn't work if the contents of the
>>>>>>>> counters
>>>>>>>> are lost at restart.
>>>>>>>>
>>>>>>>> It could be that I didn't read the documentation well enough and
>>>>>>>> missed
>>>>>>>> references to ways to manipulate them besides dyn_inc and dyn_dec
>>>>>>>>
>>>>>>>> David Lang
>>>>>>>>
>>>>>>>>
>>>>>>>> A global control needs to be at impstats level because some users
>>>>>>>> may
>>>>>>>>>
>>>>>>>>> want to access data in rate-of-change (dx/dt) form rather than
>>>>>>>>> absolute value form (it can be derived by adding up the numbers and
>>>>>>>>> studying the slope, but its prone to error when some messages are
>>>>>>>>> dropped etc). So user needs to have a say in how counters should be
>>>>>>>>> reported, should they be reset or not.
>>>>>>>>>
>>>>>>>>> Same goes for dyn-stats. User may want a perpetual counter for one
>>>>>>>>> dyn-stats bucket (regardless of other bu

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

>>>>>
>>>>> pstats has some items that are always gauges (size/maxsize/etc) and
>>>>> other
>>>>> items that are allowed to be treated as gauges or as counters.
>>>>>
>>>>> In my case, I don't want to reset those things (submitted/enqueued)
>>>>> that
>>>>> can
>>>>> be counters, but that prevents me from having dyn_stats items that are
>>>>> gauges.
>>>>>
>>>>> David Lang
>>>>>
>>>>>
>>>>> Yes, restart is a problem. But restarts are really in-frequent enough
>>>>>
>>>>>> to not matter. When they do happen, dx/dt chart shows a big dip, but
>>>>>> people still do track perpetual counters.
>>>>>>
>>>>>> Actually its only dyn_inc (we don't have a dyn_dec). But as we go
>>>>>> forward, we may expose capability to support guages and may manipulate
>>>>>> it with dyn_inc and dyn_dec.
>>>>>>
>>>>>> Resettable flag in dyn-stats bucket does the same thing as
>>>>>> resettable-flag in statsobj static-counter.
>>>>>>
>>>>>> On Tue, Mar 29, 2016 at 3:13 AM, David Lang <da...@lang.hm> wrote:
>>>>>>
>>>>>>
>>>>>>> On Tue, 29 Mar 2016, singh.janmejay wrote:
>>>>>>>
>>>>>>> Well, from counter pov, some counters by nature need to be
>>>>>>> resettable,
>>>>>>>
>>>>>>>> some are actually guages (so resetting makes little sense) and some
>>>>>>>> are perpetual (so again resetting is not the right thing to do).
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> can you explain the different categories better? I don't see how any
>>>>>>> stats
>>>>>>> can be 'by nature' one form or another, since it's possible to
>>>>>>> convert
>>>>>>> the
>>>>>>> data from one form to another.
>>>>>>>
>>>>>>> The idea of perpetual counters doesn't work if the contents of the
>>>>>>> counters
>>>>>>> are lost at restart.
>>>>>>>
>>>>>>> It could be that I didn't read the documentation well enough and
>>>>>>> missed
>>>>>>> references to ways to manipulate them besides dyn_inc and dyn_dec
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>>
>>>>>>> A global control needs to be at impstats level because some users may
>>>>>>>
>>>>>>>> want to access data in rate-of-change (dx/dt) form rather than
>>>>>>>> absolute value form (it can be derived by adding up the numbers and
>>>>>>>> studying the slope, but its prone to error when some messages are
>>>>>>>> dropped etc). So user needs to have a say in how counters should be
>>>>>>>> reported, should they be reset or not.
>>>>>>>>
>>>>>>>> Same goes for dyn-stats. User may want a perpetual counter for one
>>>>>>>> dyn-stats bucket (regardless of other buckets being configured to
>>>>>>>> reset), so its important to allow a particular bucket to be
>>>>>>>> configured
>>>>>>>> as non-resetting.
>>>>>>>>
>>>>>>>> However, if user chooses to view all the counters in cumulative
>>>>>>>> form,
>>>>>>>> the same config that works for static-counters should work for
>>>>>>>> dyn-stats counters too (so impstats flag is relevant as well).
>>>>>>>>
>>>>>>>> And'ing is necessary because it doesn't make sense to reset a
>>>>>>>> by-nature non-resettable counter.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Mar 29, 2016 at 12:25 AM, David Lang <da...@lang.hm> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> What is the thinking in having two switches that get ANDed together
>>>>&g

Re: [rsyslog] another dyn_stats question

[David Lang]

se some users may

want to access data in rate-of-change (dx/dt) form rather than
absolute value form (it can be derived by adding up the numbers and
studying the slope, but its prone to error when some messages are
dropped etc). So user needs to have a say in how counters should be
reported, should they be reset or not.

Same goes for dyn-stats. User may want a perpetual counter for one
dyn-stats bucket (regardless of other buckets being configured to
reset), so its important to allow a particular bucket to be configured
as non-resetting.

However, if user chooses to view all the counters in cumulative form,
the same config that works for static-counters should work for
dyn-stats counters too (so impstats flag is relevant as well).

And'ing is necessary because it doesn't make sense to reset a
by-nature non-resettable counter.


On Tue, Mar 29, 2016 at 12:25 AM, David Lang <da...@lang.hm> wrote:




What is the thinking in having two switches that get ANDed together
for
resetting things? I could see two independent controls, or I could
see
the
pstats setting controlling everything, but I don't understand two
getting
ANDed together.

David Lang


On Tue, 29 Mar 2016, singh.janmejay wrote:

Will add this to the documentation when I touch it for

output-restructuring.

On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm> wrote:





ahh, that's not at all clear from the documentation.

I turned off resetting the counters at the pstats level because I
was
getting soem errors/coredumps from them (quite a few months ago, so
it
may
be solved now) I'm currently getting >100 lines of pstats output
now,
before
I add dyn_stats to things.

I'll wait for the restructuring of the dyn_stats output and the new
foreach
and then try again.

David Lang

On Mon, 28 Mar 2016, singh.janmejay wrote:

Date: Mon, 28 Mar 2016 23:59:46 +0530


From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question

Sorry, I meant and'ed not or'ed.

On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:






No, this reset mechanism is no different than the mechanism that
static-counters use. If you turn it off at impstats level, it
won't
reset.

This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).

Its or'ed with stats-reporter's choice of resetting or not in
this
way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
bResetCtrs comes from that boolean param in impstats.

On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm>
wrote:






the first one should not be resetting, but the rest should be,
correct?

David Lang


dyn_stats(name="message_framing" resettable="off"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_host" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_edge_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_core_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_program" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_tag" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")




On Mon, 28 Mar 2016, singh.janmejay wrote:

Date: Mon, 28 Mar 2016 14:37:12 +0530

From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question


David,

You have resetting turned-off in the config you shared with me.
That
would prevent resetting of counters.

On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:







That is not supposed to happen. Although very unlikely, it may
be
an
environment specific bug. Do all dyn-stats tests pass in your
local
env?

Also, are you using something other than impstats to report
it?

On Mar 25, 2016 7:24 PM, "David Lang" <da...@lang.hm> wrote:








resettable counters don't seem to be resetting for me.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:

Well, the bug is metric-ttl ends up killing all metric

instead
of
unused-metrics. This works for resettable counters, but for
non-resettable
counters it kills the accumulation prematurely.

The fix will identify which counters are being used and will
not
kill
them.
So it won't discard accumulated value as long as it is being
incremented
at
least once within the TTL time.
On Mar 25, 2016 10:53 AM, "David Lang" <da...@lang.hm>
wrote:

What I'm seeing is that 

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

cumentation well enough and missed
>>>>> references to ways to manipulate them besides dyn_inc and dyn_dec
>>>>>
>>>>> David Lang
>>>>>
>>>>>
>>>>> A global control needs to be at impstats level because some users may
>>>>>> want to access data in rate-of-change (dx/dt) form rather than
>>>>>> absolute value form (it can be derived by adding up the numbers and
>>>>>> studying the slope, but its prone to error when some messages are
>>>>>> dropped etc). So user needs to have a say in how counters should be
>>>>>> reported, should they be reset or not.
>>>>>>
>>>>>> Same goes for dyn-stats. User may want a perpetual counter for one
>>>>>> dyn-stats bucket (regardless of other buckets being configured to
>>>>>> reset), so its important to allow a particular bucket to be configured
>>>>>> as non-resetting.
>>>>>>
>>>>>> However, if user chooses to view all the counters in cumulative form,
>>>>>> the same config that works for static-counters should work for
>>>>>> dyn-stats counters too (so impstats flag is relevant as well).
>>>>>>
>>>>>> And'ing is necessary because it doesn't make sense to reset a
>>>>>> by-nature non-resettable counter.
>>>>>>
>>>>>>
>>>>>> On Tue, Mar 29, 2016 at 12:25 AM, David Lang <da...@lang.hm> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> What is the thinking in having two switches that get ANDed together
>>>>>>> for
>>>>>>> resetting things? I could see two independent controls, or I could
>>>>>>> see
>>>>>>> the
>>>>>>> pstats setting controlling everything, but I don't understand two
>>>>>>> getting
>>>>>>> ANDed together.
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>>
>>>>>>> On Tue, 29 Mar 2016, singh.janmejay wrote:
>>>>>>>
>>>>>>> Will add this to the documentation when I touch it for
>>>>>>>> output-restructuring.
>>>>>>>>
>>>>>>>> On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ahh, that's not at all clear from the documentation.
>>>>>>>>>
>>>>>>>>> I turned off resetting the counters at the pstats level because I
>>>>>>>>> was
>>>>>>>>> getting soem errors/coredumps from them (quite a few months ago, so
>>>>>>>>> it
>>>>>>>>> may
>>>>>>>>> be solved now) I'm currently getting >100 lines of pstats output
>>>>>>>>> now,
>>>>>>>>> before
>>>>>>>>> I add dyn_stats to things.
>>>>>>>>>
>>>>>>>>> I'll wait for the restructuring of the dyn_stats output and the new
>>>>>>>>> foreach
>>>>>>>>> and then try again.
>>>>>>>>>
>>>>>>>>> David Lang
>>>>>>>>>
>>>>>>>>> On Mon, 28 Mar 2016, singh.janmejay wrote:
>>>>>>>>>
>>>>>>>>> Date: Mon, 28 Mar 2016 23:59:46 +0530
>>>>>>>>>>
>>>>>>>>>> From: singh.janmejay <singh.janme...@gmail.com>
>>>>>>>>>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>>>>>> Subject: Re: [rsyslog] another dyn_stats question
>>>>>>>>>>
>>>>>>>>>> Sorry, I meant and'ed not or'ed.
>>>>>>>>>>
>>>>>>>>>> On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
>>>>>>>>>> <singh.janme...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>&

Re: [rsyslog] another dyn_stats question

[David Lang]

-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question

Sorry, I meant and'ed not or'ed.

On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:





No, this reset mechanism is no different than the mechanism that
static-counters use. If you turn it off at impstats level, it won't
reset.

This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).

Its or'ed with stats-reporter's choice of resetting or not in this
way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
bResetCtrs comes from that boolean param in impstats.

On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm> wrote:





the first one should not be resetting, but the rest should be,
correct?

David Lang


dyn_stats(name="message_framing" resettable="off"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_host" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_edge_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_core_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_program" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_tag" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")




On Mon, 28 Mar 2016, singh.janmejay wrote:


Date: Mon, 28 Mar 2016 14:37:12 +0530
From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question


David,

You have resetting turned-off in the config you shared with me.
That
would prevent resetting of counters.

On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:






That is not supposed to happen. Although very unlikely, it may
be
an
environment specific bug. Do all dyn-stats tests pass in your
local
env?

Also, are you using something other than impstats to report it?

On Mar 25, 2016 7:24 PM, "David Lang" <da...@lang.hm> wrote:







resettable counters don't seem to be resetting for me.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:


Well, the bug is metric-ttl ends up killing all metric instead
of
unused-metrics. This works for resettable counters, but for
non-resettable
counters it kills the accumulation prematurely.

The fix will identify which counters are being used and will
not
kill
them.
So it won't discard accumulated value as long as it is being
incremented
at
least once within the TTL time.
On Mar 25, 2016 10:53 AM, "David Lang" <da...@lang.hm> wrote:


What I'm seeing is that unless the reset time matches the
pstats
interval,
the data output doesn't reset each pstats report.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:

This is a bug. I'll provide the fix on Monday(or in the worst
case
in
a
few







days, in case I don't get a chance to work on this on
Monday).

Basically, I was supposed to have a shadow table to mark
used
counters,
and
kill unused counters in next run of metric-ttl-cycle which
wasn't
present
in first cut because I started with resetting counter test
cases
first,
then somehow this slipped my mind.



On Mar 25, 2016 4:12 AM, "David Lang" <da...@lang.hm> wrote:








the unusedMetricLife parameter when setting up a stat says
that
it's
to







purge an unused stat.

But I've got an example where I create about 5 stats in the
bucket
and
what I think I'm seeing is that the stats are cumulative,
not
reset
each
time they are dumped out, but they get reset every
unusedMetricLife
seconds.

Is this my imagination? or is the behavior not matching the
documentation?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow
https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are
ARCHIVED
by
a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST
if
you
DON'T LIKE THAT.

___







rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED
by
a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST
if
you
DON'T LIKE THAT.

___







rsyslog mailing list
http://lists.adiscon.net/mailman/list

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

;>> pstats setting controlling everything, but I don't understand two
>>>>> getting
>>>>> ANDed together.
>>>>>
>>>>> David Lang
>>>>>
>>>>>
>>>>> On Tue, 29 Mar 2016, singh.janmejay wrote:
>>>>>
>>>>>> Will add this to the documentation when I touch it for
>>>>>> output-restructuring.
>>>>>>
>>>>>> On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ahh, that's not at all clear from the documentation.
>>>>>>>
>>>>>>> I turned off resetting the counters at the pstats level because I was
>>>>>>> getting soem errors/coredumps from them (quite a few months ago, so
>>>>>>> it
>>>>>>> may
>>>>>>> be solved now) I'm currently getting >100 lines of pstats output now,
>>>>>>> before
>>>>>>> I add dyn_stats to things.
>>>>>>>
>>>>>>> I'll wait for the restructuring of the dyn_stats output and the new
>>>>>>> foreach
>>>>>>> and then try again.
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>> On Mon, 28 Mar 2016, singh.janmejay wrote:
>>>>>>>
>>>>>>>> Date: Mon, 28 Mar 2016 23:59:46 +0530
>>>>>>>>
>>>>>>>> From: singh.janmejay <singh.janme...@gmail.com>
>>>>>>>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>>>> Subject: Re: [rsyslog] another dyn_stats question
>>>>>>>>
>>>>>>>> Sorry, I meant and'ed not or'ed.
>>>>>>>>
>>>>>>>> On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
>>>>>>>> <singh.janme...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> No, this reset mechanism is no different than the mechanism that
>>>>>>>>> static-counters use. If you turn it off at impstats level, it won't
>>>>>>>>> reset.
>>>>>>>>>
>>>>>>>>> This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).
>>>>>>>>>
>>>>>>>>> Its or'ed with stats-reporter's choice of resetting or not in this
>>>>>>>>> way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
>>>>>>>>> bResetCtrs comes from that boolean param in impstats.
>>>>>>>>>
>>>>>>>>> On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> the first one should not be resetting, but the rest should be,
>>>>>>>>>> correct?
>>>>>>>>>>
>>>>>>>>>> David Lang
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> dyn_stats(name="message_framing" resettable="off"
>>>>>>>>>> maxCardinality="3000"
>>>>>>>>>> unusedMetricLife="1200")
>>>>>>>>>> dyn_stats(name="msgs_per_host" resettable="on"
>>>>>>>>>> maxCardinality="3000"
>>>>>>>>>> unusedMetricLife="1200")
>>>>>>>>>> dyn_stats(name="msgs_per_edge_relay" resettable="on"
>>>>>>>>>> maxCardinality="3000"
>>>>>>>>>> unusedMetricLife="1200")
>>>>>>>>>> dyn_stats(name="msgs_per_core_relay" resettable="on"
>>>>>>>>>> maxCardinality="3000"
>>>>>>>>>> unusedMetricLife="1200")
>>>>>>>>>> dyn_stats(name="msgs_per_pr

Re: [rsyslog] another dyn_stats question

[David Lang]

On Wed, 30 Mar 2016, singh.janmejay wrote:


Consider a queue-size, its not really a counter, its a guage. It will
provide point-in-time value of size of the queue. Resetting such a
value will make no sense. Some static metrics we report are guages (so
subsystem supporting the metric must identify it to be resettable or
not).


but that won't work well with the current policy.

you can't have anything as a gauge unless you eliminate the possiblity for any 
of the pstats data to be gauges.


If pstats doesn't reset values, then dyn_stats can't implement gauges. If pststs 
does reset values, you have turned a lot of things that probably shouldn't reset 
into gauges.


it would make far more sense to define this purely on a per-stats basis.

pstats has some items that are always gauges (size/maxsize/etc) and other items 
that are allowed to be treated as gauges or as counters.


In my case, I don't want to reset those things (submitted/enqueued) that can be 
counters, but that prevents me from having dyn_stats items that are gauges.


David Lang


Yes, restart is a problem. But restarts are really in-frequent enough
to not matter. When they do happen, dx/dt chart shows a big dip, but
people still do track perpetual counters.

Actually its only dyn_inc (we don't have a dyn_dec). But as we go
forward, we may expose capability to support guages and may manipulate
it with dyn_inc and dyn_dec.

Resettable flag in dyn-stats bucket does the same thing as
resettable-flag in statsobj static-counter.

On Tue, Mar 29, 2016 at 3:13 AM, David Lang <da...@lang.hm> wrote:

On Tue, 29 Mar 2016, singh.janmejay wrote:


Well, from counter pov, some counters by nature need to be resettable,
some are actually guages (so resetting makes little sense) and some
are perpetual (so again resetting is not the right thing to do).



can you explain the different categories better? I don't see how any stats
can be 'by nature' one form or another, since it's possible to convert the
data from one form to another.

The idea of perpetual counters doesn't work if the contents of the counters
are lost at restart.

It could be that I didn't read the documentation well enough and missed
references to ways to manipulate them besides dyn_inc and dyn_dec

David Lang



A global control needs to be at impstats level because some users may
want to access data in rate-of-change (dx/dt) form rather than
absolute value form (it can be derived by adding up the numbers and
studying the slope, but its prone to error when some messages are
dropped etc). So user needs to have a say in how counters should be
reported, should they be reset or not.

Same goes for dyn-stats. User may want a perpetual counter for one
dyn-stats bucket (regardless of other buckets being configured to
reset), so its important to allow a particular bucket to be configured
as non-resetting.

However, if user chooses to view all the counters in cumulative form,
the same config that works for static-counters should work for
dyn-stats counters too (so impstats flag is relevant as well).

And'ing is necessary because it doesn't make sense to reset a
by-nature non-resettable counter.


On Tue, Mar 29, 2016 at 12:25 AM, David Lang <da...@lang.hm> wrote:


What is the thinking in having two switches that get ANDed together for
resetting things? I could see two independent controls, or I could see
the
pstats setting controlling everything, but I don't understand two getting
ANDed together.

David Lang


On Tue, 29 Mar 2016, singh.janmejay wrote:


Will add this to the documentation when I touch it for
output-restructuring.

On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm> wrote:



ahh, that's not at all clear from the documentation.

I turned off resetting the counters at the pstats level because I was
getting soem errors/coredumps from them (quite a few months ago, so it
may
be solved now) I'm currently getting >100 lines of pstats output now,
before
I add dyn_stats to things.

I'll wait for the restructuring of the dyn_stats output and the new
foreach
and then try again.

David Lang

On Mon, 28 Mar 2016, singh.janmejay wrote:


Date: Mon, 28 Mar 2016 23:59:46 +0530

From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question

Sorry, I meant and'ed not or'ed.

On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:




No, this reset mechanism is no different than the mechanism that
static-counters use. If you turn it off at impstats level, it won't
reset.

This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).

Its or'ed with stats-reporter's choice of resetting or not in this
way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
bResetCtrs comes from that boolean param in impstats.

On Mon, Mar 28, 2016 at 9:58 PM, David Lang <

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

Consider a queue-size, its not really a counter, its a guage. It will
provide point-in-time value of size of the queue. Resetting such a
value will make no sense. Some static metrics we report are guages (so
subsystem supporting the metric must identify it to be resettable or
not).

Yes, restart is a problem. But restarts are really in-frequent enough
to not matter. When they do happen, dx/dt chart shows a big dip, but
people still do track perpetual counters.

Actually its only dyn_inc (we don't have a dyn_dec). But as we go
forward, we may expose capability to support guages and may manipulate
it with dyn_inc and dyn_dec.

Resettable flag in dyn-stats bucket does the same thing as
resettable-flag in statsobj static-counter.

On Tue, Mar 29, 2016 at 3:13 AM, David Lang <da...@lang.hm> wrote:
> On Tue, 29 Mar 2016, singh.janmejay wrote:
>
>> Well, from counter pov, some counters by nature need to be resettable,
>> some are actually guages (so resetting makes little sense) and some
>> are perpetual (so again resetting is not the right thing to do).
>
>
> can you explain the different categories better? I don't see how any stats
> can be 'by nature' one form or another, since it's possible to convert the
> data from one form to another.
>
> The idea of perpetual counters doesn't work if the contents of the counters
> are lost at restart.
>
> It could be that I didn't read the documentation well enough and missed
> references to ways to manipulate them besides dyn_inc and dyn_dec
>
> David Lang
>
>
>> A global control needs to be at impstats level because some users may
>> want to access data in rate-of-change (dx/dt) form rather than
>> absolute value form (it can be derived by adding up the numbers and
>> studying the slope, but its prone to error when some messages are
>> dropped etc). So user needs to have a say in how counters should be
>> reported, should they be reset or not.
>>
>> Same goes for dyn-stats. User may want a perpetual counter for one
>> dyn-stats bucket (regardless of other buckets being configured to
>> reset), so its important to allow a particular bucket to be configured
>> as non-resetting.
>>
>> However, if user chooses to view all the counters in cumulative form,
>> the same config that works for static-counters should work for
>> dyn-stats counters too (so impstats flag is relevant as well).
>>
>> And'ing is necessary because it doesn't make sense to reset a
>> by-nature non-resettable counter.
>>
>>
>> On Tue, Mar 29, 2016 at 12:25 AM, David Lang <da...@lang.hm> wrote:
>>>
>>> What is the thinking in having two switches that get ANDed together for
>>> resetting things? I could see two independent controls, or I could see
>>> the
>>> pstats setting controlling everything, but I don't understand two getting
>>> ANDed together.
>>>
>>> David Lang
>>>
>>>
>>> On Tue, 29 Mar 2016, singh.janmejay wrote:
>>>
>>>> Will add this to the documentation when I touch it for
>>>> output-restructuring.
>>>>
>>>> On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm> wrote:
>>>>>
>>>>>
>>>>> ahh, that's not at all clear from the documentation.
>>>>>
>>>>> I turned off resetting the counters at the pstats level because I was
>>>>> getting soem errors/coredumps from them (quite a few months ago, so it
>>>>> may
>>>>> be solved now) I'm currently getting >100 lines of pstats output now,
>>>>> before
>>>>> I add dyn_stats to things.
>>>>>
>>>>> I'll wait for the restructuring of the dyn_stats output and the new
>>>>> foreach
>>>>> and then try again.
>>>>>
>>>>> David Lang
>>>>>
>>>>> On Mon, 28 Mar 2016, singh.janmejay wrote:
>>>>>
>>>>>> Date: Mon, 28 Mar 2016 23:59:46 +0530
>>>>>>
>>>>>> From: singh.janmejay <singh.janme...@gmail.com>
>>>>>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>> Subject: Re: [rsyslog] another dyn_stats question
>>>>>>
>>>>>> Sorry, I meant and'ed not or'ed.
>>>>>>
>>>>>> On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
>>>>>> <singh.janme...@gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>

Re: [rsyslog] another dyn_stats question

[David Lang]

On Tue, 29 Mar 2016, singh.janmejay wrote:


Well, from counter pov, some counters by nature need to be resettable,
some are actually guages (so resetting makes little sense) and some
are perpetual (so again resetting is not the right thing to do).


can you explain the different categories better? I don't see how any stats can 
be 'by nature' one form or another, since it's possible to convert the data from 
one form to another.


The idea of perpetual counters doesn't work if the contents of the counters are 
lost at restart.


It could be that I didn't read the documentation well enough and missed 
references to ways to manipulate them besides dyn_inc and dyn_dec


David Lang


A global control needs to be at impstats level because some users may
want to access data in rate-of-change (dx/dt) form rather than
absolute value form (it can be derived by adding up the numbers and
studying the slope, but its prone to error when some messages are
dropped etc). So user needs to have a say in how counters should be
reported, should they be reset or not.

Same goes for dyn-stats. User may want a perpetual counter for one
dyn-stats bucket (regardless of other buckets being configured to
reset), so its important to allow a particular bucket to be configured
as non-resetting.

However, if user chooses to view all the counters in cumulative form,
the same config that works for static-counters should work for
dyn-stats counters too (so impstats flag is relevant as well).

And'ing is necessary because it doesn't make sense to reset a
by-nature non-resettable counter.


On Tue, Mar 29, 2016 at 12:25 AM, David Lang <da...@lang.hm> wrote:

What is the thinking in having two switches that get ANDed together for
resetting things? I could see two independent controls, or I could see the
pstats setting controlling everything, but I don't understand two getting
ANDed together.

David Lang


On Tue, 29 Mar 2016, singh.janmejay wrote:


Will add this to the documentation when I touch it for
output-restructuring.

On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm> wrote:


ahh, that's not at all clear from the documentation.

I turned off resetting the counters at the pstats level because I was
getting soem errors/coredumps from them (quite a few months ago, so it
may
be solved now) I'm currently getting >100 lines of pstats output now,
before
I add dyn_stats to things.

I'll wait for the restructuring of the dyn_stats output and the new
foreach
and then try again.

David Lang

On Mon, 28 Mar 2016, singh.janmejay wrote:


Date: Mon, 28 Mar 2016 23:59:46 +0530

From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question

Sorry, I meant and'ed not or'ed.

On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:



No, this reset mechanism is no different than the mechanism that
static-counters use. If you turn it off at impstats level, it won't
reset.

This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).

Its or'ed with stats-reporter's choice of resetting or not in this
way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
bResetCtrs comes from that boolean param in impstats.

On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm> wrote:



the first one should not be resetting, but the rest should be,
correct?

David Lang


dyn_stats(name="message_framing" resettable="off"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_edge_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_core_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_program" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")




On Mon, 28 Mar 2016, singh.janmejay wrote:


Date: Mon, 28 Mar 2016 14:37:12 +0530
From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question


David,

You have resetting turned-off in the config you shared with me. That
would prevent resetting of counters.

On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:




That is not supposed to happen. Although very unlikely, it may be an
environment specific bug. Do all dyn-stats tests pass in your local
env?

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

Well, from counter pov, some counters by nature need to be resettable,
some are actually guages (so resetting makes little sense) and some
are perpetual (so again resetting is not the right thing to do).

A global control needs to be at impstats level because some users may
want to access data in rate-of-change (dx/dt) form rather than
absolute value form (it can be derived by adding up the numbers and
studying the slope, but its prone to error when some messages are
dropped etc). So user needs to have a say in how counters should be
reported, should they be reset or not.

Same goes for dyn-stats. User may want a perpetual counter for one
dyn-stats bucket (regardless of other buckets being configured to
reset), so its important to allow a particular bucket to be configured
as non-resetting.

However, if user chooses to view all the counters in cumulative form,
the same config that works for static-counters should work for
dyn-stats counters too (so impstats flag is relevant as well).

And'ing is necessary because it doesn't make sense to reset a
by-nature non-resettable counter.


On Tue, Mar 29, 2016 at 12:25 AM, David Lang <da...@lang.hm> wrote:
> What is the thinking in having two switches that get ANDed together for
> resetting things? I could see two independent controls, or I could see the
> pstats setting controlling everything, but I don't understand two getting
> ANDed together.
>
> David Lang
>
>
> On Tue, 29 Mar 2016, singh.janmejay wrote:
>
>> Will add this to the documentation when I touch it for
>> output-restructuring.
>>
>> On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm> wrote:
>>>
>>> ahh, that's not at all clear from the documentation.
>>>
>>> I turned off resetting the counters at the pstats level because I was
>>> getting soem errors/coredumps from them (quite a few months ago, so it
>>> may
>>> be solved now) I'm currently getting >100 lines of pstats output now,
>>> before
>>> I add dyn_stats to things.
>>>
>>> I'll wait for the restructuring of the dyn_stats output and the new
>>> foreach
>>> and then try again.
>>>
>>> David Lang
>>>
>>> On Mon, 28 Mar 2016, singh.janmejay wrote:
>>>
>>>> Date: Mon, 28 Mar 2016 23:59:46 +0530
>>>>
>>>> From: singh.janmejay <singh.janme...@gmail.com>
>>>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>> Subject: Re: [rsyslog] another dyn_stats question
>>>>
>>>> Sorry, I meant and'ed not or'ed.
>>>>
>>>> On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
>>>> <singh.janme...@gmail.com> wrote:
>>>>>
>>>>>
>>>>> No, this reset mechanism is no different than the mechanism that
>>>>> static-counters use. If you turn it off at impstats level, it won't
>>>>> reset.
>>>>>
>>>>> This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).
>>>>>
>>>>> Its or'ed with stats-reporter's choice of resetting or not in this
>>>>> way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
>>>>> bResetCtrs comes from that boolean param in impstats.
>>>>>
>>>>> On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm> wrote:
>>>>>>
>>>>>>
>>>>>> the first one should not be resetting, but the rest should be,
>>>>>> correct?
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>>
>>>>>> dyn_stats(name="message_framing" resettable="off"
>>>>>> maxCardinality="3000"
>>>>>> unusedMetricLife="1200")
>>>>>> dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000"
>>>>>> unusedMetricLife="1200")
>>>>>> dyn_stats(name="msgs_per_edge_relay" resettable="on"
>>>>>> maxCardinality="3000"
>>>>>> unusedMetricLife="1200")
>>>>>> dyn_stats(name="msgs_per_core_relay" resettable="on"
>>>>>> maxCardinality="3000"
>>>>>> unusedMetricLife="1200")
>>>>>> dyn_stats(name="msgs_per_program" resettable="on"
>>>>>> maxCardinality="3000"
>>>>>> unused

Re: [rsyslog] another dyn_stats question

[David Lang]

What is the thinking in having two switches that get ANDed together for 
resetting things? I could see two independent controls, or I could see the 
pstats setting controlling everything, but I don't understand two getting ANDed 
together.


David Lang

On Tue, 29 Mar 2016, singh.janmejay wrote:


Will add this to the documentation when I touch it for output-restructuring.

On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm> wrote:

ahh, that's not at all clear from the documentation.

I turned off resetting the counters at the pstats level because I was
getting soem errors/coredumps from them (quite a few months ago, so it may
be solved now) I'm currently getting >100 lines of pstats output now, before
I add dyn_stats to things.

I'll wait for the restructuring of the dyn_stats output and the new foreach
and then try again.

David Lang

On Mon, 28 Mar 2016, singh.janmejay wrote:


Date: Mon, 28 Mar 2016 23:59:46 +0530

From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question

Sorry, I meant and'ed not or'ed.

On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:


No, this reset mechanism is no different than the mechanism that
static-counters use. If you turn it off at impstats level, it won't
reset.

This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).

Its or'ed with stats-reporter's choice of resetting or not in this
way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
bResetCtrs comes from that boolean param in impstats.

On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm> wrote:


the first one should not be resetting, but the rest should be, correct?

David Lang


dyn_stats(name="message_framing" resettable="off" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_edge_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_core_relay" resettable="on"
maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_program" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")




On Mon, 28 Mar 2016, singh.janmejay wrote:


Date: Mon, 28 Mar 2016 14:37:12 +0530
From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question


David,

You have resetting turned-off in the config you shared with me. That
would prevent resetting of counters.

On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:



That is not supposed to happen. Although very unlikely, it may be an
environment specific bug. Do all dyn-stats tests pass in your local
env?

Also, are you using something other than impstats to report it?

On Mar 25, 2016 7:24 PM, "David Lang" <da...@lang.hm> wrote:




resettable counters don't seem to be resetting for me.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:


Well, the bug is metric-ttl ends up killing all metric instead of
unused-metrics. This works for resettable counters, but for
non-resettable
counters it kills the accumulation prematurely.

The fix will identify which counters are being used and will not
kill
them.
So it won't discard accumulated value as long as it is being
incremented
at
least once within the TTL time.
On Mar 25, 2016 10:53 AM, "David Lang" <da...@lang.hm> wrote:


What I'm seeing is that unless the reset time matches the pstats
interval,
the data output doesn't reset each pstats report.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:

This is a bug. I'll provide the fix on Monday(or in the worst case
in
a
few




days, in case I don't get a chance to work on this on Monday).

Basically, I was supposed to have a shadow table to mark used
counters,
and
kill unused counters in next run of metric-ttl-cycle which wasn't
present
in first cut because I started with resetting counter test cases
first,
then somehow this slipped my mind.



On Mar 25, 2016 4:12 AM, "David Lang" <da...@lang.hm> wrote:





the unusedMetricLife parameter when setting up a stat says that
it's
to




purge an unused stat.

But I've got an example where I create about 5 stats in the
bucket
and
what I think I'm seeing is that the stats are cumulative, not
reset
each
time they are dumped out, but they

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

Will add this to the documentation when I touch it for output-restructuring.

On Tue, Mar 29, 2016 at 12:11 AM, David Lang <da...@lang.hm> wrote:
> ahh, that's not at all clear from the documentation.
>
> I turned off resetting the counters at the pstats level because I was
> getting soem errors/coredumps from them (quite a few months ago, so it may
> be solved now) I'm currently getting >100 lines of pstats output now, before
> I add dyn_stats to things.
>
> I'll wait for the restructuring of the dyn_stats output and the new foreach
> and then try again.
>
> David Lang
>
> On Mon, 28 Mar 2016, singh.janmejay wrote:
>
>> Date: Mon, 28 Mar 2016 23:59:46 +0530
>>
>> From: singh.janmejay <singh.janme...@gmail.com>
>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] another dyn_stats question
>>
>> Sorry, I meant and'ed not or'ed.
>>
>> On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
>> <singh.janme...@gmail.com> wrote:
>>>
>>> No, this reset mechanism is no different than the mechanism that
>>> static-counters use. If you turn it off at impstats level, it won't
>>> reset.
>>>
>>> This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).
>>>
>>> Its or'ed with stats-reporter's choice of resetting or not in this
>>> way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
>>> bResetCtrs comes from that boolean param in impstats.
>>>
>>> On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm> wrote:
>>>>
>>>> the first one should not be resetting, but the rest should be, correct?
>>>>
>>>> David Lang
>>>>
>>>>
>>>> dyn_stats(name="message_framing" resettable="off" maxCardinality="3000"
>>>> unusedMetricLife="1200")
>>>> dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000"
>>>> unusedMetricLife="1200")
>>>> dyn_stats(name="msgs_per_edge_relay" resettable="on"
>>>> maxCardinality="3000"
>>>> unusedMetricLife="1200")
>>>> dyn_stats(name="msgs_per_core_relay" resettable="on"
>>>> maxCardinality="3000"
>>>> unusedMetricLife="1200")
>>>> dyn_stats(name="msgs_per_program" resettable="on" maxCardinality="3000"
>>>> unusedMetricLife="1200")
>>>> dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000"
>>>> unusedMetricLife="1200")
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 28 Mar 2016, singh.janmejay wrote:
>>>>
>>>>> Date: Mon, 28 Mar 2016 14:37:12 +0530
>>>>> From: singh.janmejay <singh.janme...@gmail.com>
>>>>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>> Subject: Re: [rsyslog] another dyn_stats question
>>>>>
>>>>>
>>>>> David,
>>>>>
>>>>> You have resetting turned-off in the config you shared with me. That
>>>>> would prevent resetting of counters.
>>>>>
>>>>> On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
>>>>> <singh.janme...@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>> That is not supposed to happen. Although very unlikely, it may be an
>>>>>> environment specific bug. Do all dyn-stats tests pass in your local
>>>>>> env?
>>>>>>
>>>>>> Also, are you using something other than impstats to report it?
>>>>>>
>>>>>> On Mar 25, 2016 7:24 PM, "David Lang" <da...@lang.hm> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> resettable counters don't seem to be resetting for me.
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>> On Fri, 25 Mar 2016, singh.janmejay wrote:
>>>>>>>
>>>>>>>> Well, the bug is metric-ttl ends up killing all metric instead of
>>>>>>>> unused-metrics. This works for resettable counters, but for
>>&g

Re: [rsyslog] another dyn_stats question

[David Lang]

ahh, that's not at all clear from the documentation.

I turned off resetting the counters at the pstats level because I was getting 
soem errors/coredumps from them (quite a few months ago, so it may be solved 
now) I'm currently getting >100 lines of pstats output now, before I add 
dyn_stats to things.


I'll wait for the restructuring of the dyn_stats output and the new foreach and 
then try again.


David Lang

On Mon, 28 Mar 2016, singh.janmejay wrote:


Date: Mon, 28 Mar 2016 23:59:46 +0530
From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question

Sorry, I meant and'ed not or'ed.

On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:

No, this reset mechanism is no different than the mechanism that
static-counters use. If you turn it off at impstats level, it won't
reset.

This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).

Its or'ed with stats-reporter's choice of resetting or not in this
way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
bResetCtrs comes from that boolean param in impstats.

On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm> wrote:

the first one should not be resetting, but the rest should be, correct?

David Lang


dyn_stats(name="message_framing" resettable="off" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_edge_relay" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_core_relay" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_program" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")
dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000"
unusedMetricLife="1200")




On Mon, 28 Mar 2016, singh.janmejay wrote:


Date: Mon, 28 Mar 2016 14:37:12 +0530
From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question


David,

You have resetting turned-off in the config you shared with me. That
would prevent resetting of counters.

On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:


That is not supposed to happen. Although very unlikely, it may be an
environment specific bug. Do all dyn-stats tests pass in your local env?

Also, are you using something other than impstats to report it?

On Mar 25, 2016 7:24 PM, "David Lang" <da...@lang.hm> wrote:



resettable counters don't seem to be resetting for me.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:


Well, the bug is metric-ttl ends up killing all metric instead of
unused-metrics. This works for resettable counters, but for
non-resettable
counters it kills the accumulation prematurely.

The fix will identify which counters are being used and will not kill
them.
So it won't discard accumulated value as long as it is being
incremented
at
least once within the TTL time.
On Mar 25, 2016 10:53 AM, "David Lang" <da...@lang.hm> wrote:


What I'm seeing is that unless the reset time matches the pstats
interval,
the data output doesn't reset each pstats report.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:

This is a bug. I'll provide the fix on Monday(or in the worst case in
a
few



days, in case I don't get a chance to work on this on Monday).

Basically, I was supposed to have a shadow table to mark used
counters,
and
kill unused counters in next run of metric-ttl-cycle which wasn't
present
in first cut because I started with resetting counter test cases
first,
then somehow this slipped my mind.



On Mar 25, 2016 4:12 AM, "David Lang" <da...@lang.hm> wrote:




the unusedMetricLife parameter when setting up a stat says that it's
to



purge an unused stat.

But I've got an example where I create about 5 stats in the bucket
and
what I think I'm seeing is that the stats are cumulative, not reset
each
time they are dumped out, but they get reset every unusedMetricLife
seconds.

Is this my imagination? or is the behavior not matching the
documentation?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. P

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

Sorry, I meant and'ed not or'ed.

On Mon, Mar 28, 2016 at 11:02 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:
> No, this reset mechanism is no different than the mechanism that
> static-counters use. If you turn it off at impstats level, it won't
> reset.
>
> This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).
>
> Its or'ed with stats-reporter's choice of resetting or not in this
> way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
> bResetCtrs comes from that boolean param in impstats.
>
> On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm> wrote:
>> the first one should not be resetting, but the rest should be, correct?
>>
>> David Lang
>>
>>
>> dyn_stats(name="message_framing" resettable="off" maxCardinality="3000"
>> unusedMetricLife="1200")
>> dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000"
>> unusedMetricLife="1200")
>> dyn_stats(name="msgs_per_edge_relay" resettable="on" maxCardinality="3000"
>> unusedMetricLife="1200")
>> dyn_stats(name="msgs_per_core_relay" resettable="on" maxCardinality="3000"
>> unusedMetricLife="1200")
>> dyn_stats(name="msgs_per_program" resettable="on" maxCardinality="3000"
>> unusedMetricLife="1200")
>> dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000"
>> unusedMetricLife="1200")
>>
>>
>>
>>
>> On Mon, 28 Mar 2016, singh.janmejay wrote:
>>
>>> Date: Mon, 28 Mar 2016 14:37:12 +0530
>>> From: singh.janmejay <singh.janme...@gmail.com>
>>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>> Subject: Re: [rsyslog] another dyn_stats question
>>>
>>>
>>> David,
>>>
>>> You have resetting turned-off in the config you shared with me. That
>>> would prevent resetting of counters.
>>>
>>> On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
>>> <singh.janme...@gmail.com> wrote:
>>>>
>>>> That is not supposed to happen. Although very unlikely, it may be an
>>>> environment specific bug. Do all dyn-stats tests pass in your local env?
>>>>
>>>> Also, are you using something other than impstats to report it?
>>>>
>>>> On Mar 25, 2016 7:24 PM, "David Lang" <da...@lang.hm> wrote:
>>>>>
>>>>>
>>>>> resettable counters don't seem to be resetting for me.
>>>>>
>>>>> David Lang
>>>>>
>>>>> On Fri, 25 Mar 2016, singh.janmejay wrote:
>>>>>
>>>>>> Well, the bug is metric-ttl ends up killing all metric instead of
>>>>>> unused-metrics. This works for resettable counters, but for
>>>>>> non-resettable
>>>>>> counters it kills the accumulation prematurely.
>>>>>>
>>>>>> The fix will identify which counters are being used and will not kill
>>>>>> them.
>>>>>> So it won't discard accumulated value as long as it is being
>>>>>> incremented
>>>>>> at
>>>>>> least once within the TTL time.
>>>>>> On Mar 25, 2016 10:53 AM, "David Lang" <da...@lang.hm> wrote:
>>>>>>
>>>>>>> What I'm seeing is that unless the reset time matches the pstats
>>>>>>> interval,
>>>>>>> the data output doesn't reset each pstats report.
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>> On Fri, 25 Mar 2016, singh.janmejay wrote:
>>>>>>>
>>>>>>> This is a bug. I'll provide the fix on Monday(or in the worst case in
>>>>>>> a
>>>>>>> few
>>>>>>>>
>>>>>>>>
>>>>>>>> days, in case I don't get a chance to work on this on Monday).
>>>>>>>>
>>>>>>>> Basically, I was supposed to have a shadow table to mark used
>>>>>>>> counters,
>>>>>>>> and
>>>>>>>> kill unused counters in next run of metric-ttl-cycle which wasn't
>>>>>>>> present
>>>>&

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

No, this reset mechanism is no different than the mechanism that
static-counters use. If you turn it off at impstats level, it won't
reset.

This flag basically sets CTR_FLAG_RESETTABLE (statsobj.h).

Its or'ed with stats-reporter's choice of resetting or not in this
way: (bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)), where
bResetCtrs comes from that boolean param in impstats.

On Mon, Mar 28, 2016 at 9:58 PM, David Lang <da...@lang.hm> wrote:
> the first one should not be resetting, but the rest should be, correct?
>
> David Lang
>
>
> dyn_stats(name="message_framing" resettable="off" maxCardinality="3000"
> unusedMetricLife="1200")
> dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
> dyn_stats(name="msgs_per_edge_relay" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
> dyn_stats(name="msgs_per_core_relay" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
> dyn_stats(name="msgs_per_program" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
> dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
>
>
>
>
> On Mon, 28 Mar 2016, singh.janmejay wrote:
>
>> Date: Mon, 28 Mar 2016 14:37:12 +0530
>> From: singh.janmejay <singh.janme...@gmail.com>
>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] another dyn_stats question
>>
>>
>> David,
>>
>> You have resetting turned-off in the config you shared with me. That
>> would prevent resetting of counters.
>>
>> On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
>> <singh.janme...@gmail.com> wrote:
>>>
>>> That is not supposed to happen. Although very unlikely, it may be an
>>> environment specific bug. Do all dyn-stats tests pass in your local env?
>>>
>>> Also, are you using something other than impstats to report it?
>>>
>>> On Mar 25, 2016 7:24 PM, "David Lang" <da...@lang.hm> wrote:
>>>>
>>>>
>>>> resettable counters don't seem to be resetting for me.
>>>>
>>>> David Lang
>>>>
>>>> On Fri, 25 Mar 2016, singh.janmejay wrote:
>>>>
>>>>> Well, the bug is metric-ttl ends up killing all metric instead of
>>>>> unused-metrics. This works for resettable counters, but for
>>>>> non-resettable
>>>>> counters it kills the accumulation prematurely.
>>>>>
>>>>> The fix will identify which counters are being used and will not kill
>>>>> them.
>>>>> So it won't discard accumulated value as long as it is being
>>>>> incremented
>>>>> at
>>>>> least once within the TTL time.
>>>>> On Mar 25, 2016 10:53 AM, "David Lang" <da...@lang.hm> wrote:
>>>>>
>>>>>> What I'm seeing is that unless the reset time matches the pstats
>>>>>> interval,
>>>>>> the data output doesn't reset each pstats report.
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>> On Fri, 25 Mar 2016, singh.janmejay wrote:
>>>>>>
>>>>>> This is a bug. I'll provide the fix on Monday(or in the worst case in
>>>>>> a
>>>>>> few
>>>>>>>
>>>>>>>
>>>>>>> days, in case I don't get a chance to work on this on Monday).
>>>>>>>
>>>>>>> Basically, I was supposed to have a shadow table to mark used
>>>>>>> counters,
>>>>>>> and
>>>>>>> kill unused counters in next run of metric-ttl-cycle which wasn't
>>>>>>> present
>>>>>>> in first cut because I started with resetting counter test cases
>>>>>>> first,
>>>>>>> then somehow this slipped my mind.
>>>>>>>
>>>>>>
>>>>>> On Mar 25, 2016 4:12 AM, "David Lang" <da...@lang.hm> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> the unusedMetricLife parameter when setting up a stat say

Re: [rsyslog] another dyn_stats question

[David Lang]

the first one should not be resetting, but the rest should be, correct?

David Lang


dyn_stats(name="message_framing" resettable="off" maxCardinality="3000" 
unusedMetricLife="1200")
dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000" 
unusedMetricLife="1200")
dyn_stats(name="msgs_per_edge_relay" resettable="on" maxCardinality="3000" 
unusedMetricLife="1200")
dyn_stats(name="msgs_per_core_relay" resettable="on" maxCardinality="3000" 
unusedMetricLife="1200")
dyn_stats(name="msgs_per_program" resettable="on" maxCardinality="3000" 
unusedMetricLife="1200")
dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000" 
unusedMetricLife="1200")





On Mon, 28 Mar 2016, singh.janmejay wrote:


Date: Mon, 28 Mar 2016 14:37:12 +0530
From: singh.janmejay <singh.janme...@gmail.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] another dyn_stats question

David,

You have resetting turned-off in the config you shared with me. That
would prevent resetting of counters.

On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
<singh.janme...@gmail.com> wrote:

That is not supposed to happen. Although very unlikely, it may be an
environment specific bug. Do all dyn-stats tests pass in your local env?

Also, are you using something other than impstats to report it?

On Mar 25, 2016 7:24 PM, "David Lang" <da...@lang.hm> wrote:


resettable counters don't seem to be resetting for me.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:


Well, the bug is metric-ttl ends up killing all metric instead of
unused-metrics. This works for resettable counters, but for
non-resettable
counters it kills the accumulation prematurely.

The fix will identify which counters are being used and will not kill
them.
So it won't discard accumulated value as long as it is being incremented
at
least once within the TTL time.
On Mar 25, 2016 10:53 AM, "David Lang" <da...@lang.hm> wrote:


What I'm seeing is that unless the reset time matches the pstats
interval,
the data output doesn't reset each pstats report.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:

This is a bug. I'll provide the fix on Monday(or in the worst case in a
few


days, in case I don't get a chance to work on this on Monday).

Basically, I was supposed to have a shadow table to mark used counters,
and
kill unused counters in next run of metric-ttl-cycle which wasn't
present
in first cut because I started with resetting counter test cases first,
then somehow this slipped my mind.



On Mar 25, 2016 4:12 AM, "David Lang" <da...@lang.hm> wrote:



the unusedMetricLife parameter when setting up a stat says that it's to


purge an unused stat.

But I've got an example where I create about 5 stats in the bucket and
what I think I'm seeing is that the stats are cumulative, not reset
each
time they are dumped out, but they get reset every unusedMetricLife
seconds.

Is this my imagination? or is the behavior not matching the
documentation?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.

___


rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.

___


rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.


___
rsyslog mailing list

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

David,

You have resetting turned-off in the config you shared with me. That
would prevent resetting of counters.

On Fri, Mar 25, 2016 at 8:21 PM, singh.janmejay
 wrote:
> That is not supposed to happen. Although very unlikely, it may be an
> environment specific bug. Do all dyn-stats tests pass in your local env?
>
> Also, are you using something other than impstats to report it?
>
> On Mar 25, 2016 7:24 PM, "David Lang"  wrote:
>>
>> resettable counters don't seem to be resetting for me.
>>
>> David Lang
>>
>> On Fri, 25 Mar 2016, singh.janmejay wrote:
>>
>>> Well, the bug is metric-ttl ends up killing all metric instead of
>>> unused-metrics. This works for resettable counters, but for
>>> non-resettable
>>> counters it kills the accumulation prematurely.
>>>
>>> The fix will identify which counters are being used and will not kill
>>> them.
>>> So it won't discard accumulated value as long as it is being incremented
>>> at
>>> least once within the TTL time.
>>> On Mar 25, 2016 10:53 AM, "David Lang"  wrote:
>>>
 What I'm seeing is that unless the reset time matches the pstats
 interval,
 the data output doesn't reset each pstats report.

 David Lang

 On Fri, 25 Mar 2016, singh.janmejay wrote:

 This is a bug. I'll provide the fix on Monday(or in the worst case in a
 few
>
> days, in case I don't get a chance to work on this on Monday).
>
> Basically, I was supposed to have a shadow table to mark used counters,
> and
> kill unused counters in next run of metric-ttl-cycle which wasn't
> present
> in first cut because I started with resetting counter test cases first,
> then somehow this slipped my mind.
>

 On Mar 25, 2016 4:12 AM, "David Lang"  wrote:
>
>
> the unusedMetricLife parameter when setting up a stat says that it's to
>>
>> purge an unused stat.
>>
>> But I've got an example where I create about 5 stats in the bucket and
>> what I think I'm seeing is that the stats are cumulative, not reset
>> each
>> time they are dumped out, but they get reset every unusedMetricLife
>> seconds.
>>
>> Is this my imagination? or is the behavior not matching the
>> documentation?
>>
>> David Lang
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>> ___
>
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
> ___

 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.

>>> ___
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>>> LIKE THAT.
>>>
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.



-- 
Regards,
Janmejay
http://codehunk.wordpress.com
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

That is not supposed to happen. Although very unlikely, it may be an
environment specific bug. Do all dyn-stats tests pass in your local env?

Also, are you using something other than impstats to report it?
On Mar 25, 2016 7:24 PM, "David Lang"  wrote:

> resettable counters don't seem to be resetting for me.
>
> David Lang
>
> On Fri, 25 Mar 2016, singh.janmejay wrote:
>
> Well, the bug is metric-ttl ends up killing all metric instead of
>> unused-metrics. This works for resettable counters, but for non-resettable
>> counters it kills the accumulation prematurely.
>>
>> The fix will identify which counters are being used and will not kill
>> them.
>> So it won't discard accumulated value as long as it is being incremented
>> at
>> least once within the TTL time.
>> On Mar 25, 2016 10:53 AM, "David Lang"  wrote:
>>
>> What I'm seeing is that unless the reset time matches the pstats interval,
>>> the data output doesn't reset each pstats report.
>>>
>>> David Lang
>>>
>>> On Fri, 25 Mar 2016, singh.janmejay wrote:
>>>
>>> This is a bug. I'll provide the fix on Monday(or in the worst case in a
>>> few
>>>
 days, in case I don't get a chance to work on this on Monday).

 Basically, I was supposed to have a shadow table to mark used counters,
 and
 kill unused counters in next run of metric-ttl-cycle which wasn't
 present
 in first cut because I started with resetting counter test cases first,
 then somehow this slipped my mind.


>>> On Mar 25, 2016 4:12 AM, "David Lang"  wrote:
>>>

 the unusedMetricLife parameter when setting up a stat says that it's to

> purge an unused stat.
>
> But I've got an example where I create about 5 stats in the bucket and
> what I think I'm seeing is that the stats are cumulative, not reset
> each
> time they are dumped out, but they get reset every unusedMetricLife
> seconds.
>
> Is this my imagination? or is the behavior not matching the
> documentation?
>
> David Lang
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
> ___
>
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.

 ___

>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>> DON'T LIKE THAT.
>>>
>>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] another dyn_stats question

[David Lang]

resettable counters don't seem to be resetting for me.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:


Well, the bug is metric-ttl ends up killing all metric instead of
unused-metrics. This works for resettable counters, but for non-resettable
counters it kills the accumulation prematurely.

The fix will identify which counters are being used and will not kill them.
So it won't discard accumulated value as long as it is being incremented at
least once within the TTL time.
On Mar 25, 2016 10:53 AM, "David Lang"  wrote:


What I'm seeing is that unless the reset time matches the pstats interval,
the data output doesn't reset each pstats report.

David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:

This is a bug. I'll provide the fix on Monday(or in the worst case in a few

days, in case I don't get a chance to work on this on Monday).

Basically, I was supposed to have a shadow table to mark used counters,
and
kill unused counters in next run of metric-ttl-cycle which wasn't present
in first cut because I started with resetting counter test cases first,
then somehow this slipped my mind.



On Mar 25, 2016 4:12 AM, "David Lang"  wrote:


the unusedMetricLife parameter when setting up a stat says that it's to

purge an unused stat.

But I've got an example where I create about 5 stats in the bucket and
what I think I'm seeing is that the stats are cumulative, not reset each
time they are dumped out, but they get reset every unusedMetricLife
seconds.

Is this my imagination? or is the behavior not matching the
documentation?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.

___

rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.

___

rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

Well, the bug is metric-ttl ends up killing all metric instead of
unused-metrics. This works for resettable counters, but for non-resettable
counters it kills the accumulation prematurely.

The fix will identify which counters are being used and will not kill them.
So it won't discard accumulated value as long as it is being incremented at
least once within the TTL time.
On Mar 25, 2016 10:53 AM, "David Lang"  wrote:

> What I'm seeing is that unless the reset time matches the pstats interval,
> the data output doesn't reset each pstats report.
>
> David Lang
>
> On Fri, 25 Mar 2016, singh.janmejay wrote:
>
> This is a bug. I'll provide the fix on Monday(or in the worst case in a few
>> days, in case I don't get a chance to work on this on Monday).
>>
>> Basically, I was supposed to have a shadow table to mark used counters,
>> and
>> kill unused counters in next run of metric-ttl-cycle which wasn't present
>> in first cut because I started with resetting counter test cases first,
>> then somehow this slipped my mind.
>>
>
> On Mar 25, 2016 4:12 AM, "David Lang"  wrote:
>>
>> the unusedMetricLife parameter when setting up a stat says that it's to
>>> purge an unused stat.
>>>
>>> But I've got an example where I create about 5 stats in the bucket and
>>> what I think I'm seeing is that the stats are cumulative, not reset each
>>> time they are dumped out, but they get reset every unusedMetricLife
>>> seconds.
>>>
>>> Is this my imagination? or is the behavior not matching the
>>> documentation?
>>>
>>> David Lang
>>> ___
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>> DON'T LIKE THAT.
>>>
>>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] another dyn_stats question

[David Lang]

What I'm seeing is that unless the reset time matches the pstats interval, the 
data output doesn't reset each pstats report.


David Lang

On Fri, 25 Mar 2016, singh.janmejay wrote:


This is a bug. I'll provide the fix on Monday(or in the worst case in a few
days, in case I don't get a chance to work on this on Monday).

Basically, I was supposed to have a shadow table to mark used counters, and
kill unused counters in next run of metric-ttl-cycle which wasn't present
in first cut because I started with resetting counter test cases first,
then somehow this slipped my mind.



On Mar 25, 2016 4:12 AM, "David Lang"  wrote:


the unusedMetricLife parameter when setting up a stat says that it's to
purge an unused stat.

But I've got an example where I create about 5 stats in the bucket and
what I think I'm seeing is that the stats are cumulative, not reset each
time they are dumped out, but they get reset every unusedMetricLife seconds.

Is this my imagination? or is the behavior not matching the documentation?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] another dyn_stats question

[singh.janmejay]

This is a bug. I'll provide the fix on Monday(or in the worst case in a few
days, in case I don't get a chance to work on this on Monday).

Basically, I was supposed to have a shadow table to mark used counters, and
kill unused counters in next run of metric-ttl-cycle which wasn't present
in first cut because I started with resetting counter test cases first,
then somehow this slipped my mind.
On Mar 25, 2016 4:12 AM, "David Lang"  wrote:

> the unusedMetricLife parameter when setting up a stat says that it's to
> purge an unused stat.
>
> But I've got an example where I create about 5 stats in the bucket and
> what I think I'm seeing is that the stats are cumulative, not reset each
> time they are dumped out, but they get reset every unusedMetricLife seconds.
>
> Is this my imagination? or is the behavior not matching the documentation?
>
> David Lang
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] another dyn_stats question

[David Lang]

the unusedMetricLife parameter when setting up a stat says that it's to purge an 
unused stat.


But I've got an example where I create about 5 stats in the bucket and what I 
think I'm seeing is that the stats are cumulative, not reset each time they are 
dumped out, but they get reset every unusedMetricLife seconds.


Is this my imagination? or is the behavior not matching the documentation?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.