Inline.
On Tue, Mar 22, 2016 at 5:01 AM, David Lang <da...@lang.hm> wrote:
> I added this to my configs
>
> # grep -B 1 -A 1 dyn_ /etc/rsyslog.conf
> # custom stats to track in rsyslog
> dyn_stats(name="msgs_per_host" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
> dyn_stats(name="msgs_per_edge_relay" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
> dyn_stats(name="msgs_per_core_relay" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
> dyn_stats(name="msgs_per_program" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
> dyn_stats(name="msgs_per_tag" resettable="on" maxCardinality="3000"
> unusedMetricLife="1200")
>
> --
> /var/log/sources-messages;sources
> set $.inc = dyn_inc("msgs_per_host", $hostname);
> set $.inc = dyn_inc("msgs_per_program", $programname);
> # if $!trusted!edge!relay != "" then {
> # set $.inc = dyn_inc("msgs_per_edge_relay", $!trusted!edge!relay);
> # }
> # if $!trusted!core!relay != "" then {
> # set $.inc = dyn_inc("msgs_per_edge_relay", $!trusted!core!relay);
> # }
> --
> }
> #set $.inc = dyn_inc("msgs_per_tag", $.custommessage);
> /var/log/messages-tags;manual
>
> if I uncomment any of the lines that refer to $! or $. variables, rsyslog
> basically stops processing messages (a few messages seem to be processed in
> spurts, bit a lot of processing never happens)
>
> any chance that this has problems with locking around user defined
> variables?
Not sure, I run it in production (66 6-cpu(HT) haswell VMs handling an
aggregate of ~2.5 M-msg / sec (~1.5kB each)) and haven't seen such
jitters, but then I don't use 3 level deep keys either and in my case,
the feature is backported to 8.12 branch), so still uses json-c (not
libfastjson). I do use it with user-defined variables too, just not
nested as deep.
We don't treat variables differently after dereferencing them in the
top-most layer handling rainerscript fn-calls, so $hostname and
$!x!y!z, as far as any rainerscript fn invocation is considered, are
the same (performance-wise) except for the top-level dereference
(outside of fn-impl-body).
Some more info may come handy in reproducing it:
- What is the message throughput (in msg/s) and what is the avg msg size?
- Do you see messages being written to sources-messages file, while
they don't get written to messages-tags file?
- Do you see a significant jump in voluntary context switches(say per
10 seconds or even per second) when you uncomment those lines? How big
is the jump?
- How does the observation change if you replace it with this kind of
variable access pattern:
set $.x = $!trusted!edge!relay;
if $.x != "" then {
set $.inc = dyn_inc("msgs_per_edge_relay", $.x);
}
- Is this stock 8.17 build?
- Is this x86-64 Linux? What distro (and version)?
>
> Also, the documentation doesn't say why we are setting $.inc to the result
> of the command, what does $.inc contain after the command is run? from the
> documentation, it looks like it contains an error code (at least the way
> it's used is similar to error code returns in other languages), but I don't
> see what the errors are in the doc page.
Yes, it is the error-code. It has value 0 when successful, an non-zero
when it fails to increment (it may fail due to maxCardinality being
hit or due to a contended for lock etc (it doesn't wait for the lock,
it uses try-lock and doesn't increment the counter if it can't get a
shared-lock over the bucket).
These are rsyslog error-codes. It does an error pass-thru.
I'll add this to documentation (didn't know this was missed).
>
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
--
Regards,
Janmejay
http://codehunk.wordpress.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
