On Tue, 6 Oct 2015, singh.janmejay wrote:
Hi,
I am working on support for stats with dynamic-name. This comes handy
in situations where metric-name is dependent upon value of a certain
attribute of the message.
Say, for a central log-aggregation service, its valuable to know what
is inbound message-count distribution across application-clusters that
send logs to it, or for a shared-server, its valuable to know what is
the log-volume generation across users etc.
Im thinking of using functions-like interface to support this. It may
look similar to this:
====================
dyn_stats("user_msg_count")
...
ruleset(...) {
...
dyn_inc("user_msg_count", $.user)
...
}
====================
dyn_stats signature looks like:
dyn_stats(<name_space>, <resettable: default=true>, <max_cardinality:
default=10k>, <recycle_metric_names_after: default=1hr>)
dyn_inc signature looks like:
dyn_inc(<name_space>, <metric_name>)
Reporting would work similar to static-metric via impstats. Mapping:
statsobj_s.name = name_space
statsobj_s.origin = "dyn"
ctr_s.name = "foo" (say $.user had value foo)
Thoughts / suggestions?
how is this different/better than global variables? (although we may need to
implement soem functions, atomic inc/dec copy+clear) If you have pstats output
in json format, you can even piggyback on it's schedule to output the data.
things like stats can very easily end up being expensive in terms of locking
(something global variables already have figured out), and it sounds like you
are proposing adding a scheduler of some sort to output the data.
variables should not need to be 'recycled', either they contain data or they
don't. If they contain data, you need to keep the data until you do something
with it, if they don't, you don't have to track them.
I am actually doing this sort of thing external to rsyslog in SEC
I have a template in rsyslog that contains hostname, fromhost-ip, programname
and I output it via improg to SEC. SEC accumulates counters and has scheduled
outputs to files.
before I started using SEC for this, I used the same template to output to a
file and then for reports, used cut + sort + uniq -c to extract the data I need.
When the files only contain the significant data, this is actually not bad to
do, even at higher volumes.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
