By default you will only use one core for mmnormalize, but you can configure it
to use more (although doing so when you don't need to ends up slowing you down
due to contention on the queue)
I don't have info on how fast to expect it to be, it would vary by quite a bit
depending on the speed of your cpu/ram/cache.
the number of rules does not significantly affect the speed of mmnormalize, the
way that it compiles the rules when loading the ruleset means that the length of
the log message (and how complex the rule is that match it) matter more than how
many rules there are.
If you are running into bottlenecks at 10s of logs/sec then something is very
wrong. Normally rsyslog can do hundreds of thousands of logs/sec, so I would
still expect thousands of logs/sec with mmnormalize (as long as you don't use
regex matches)
If you think that it's being limited by only using one core, use the 'H' option
in top to show the individual threads, see if any of the rsyslog threads show
that they are using 100% cpu. If not then you have some other problem.
David Lang
On Wed, 28 Jan 2015, Micah Yoder wrote:
Hi,
Couple questions...
Does mmnormalize use multiple cores? I see in the liblognorm
documentation that it does not internally, but it can be called from
multiple threads.
Any ideas as to how many messages it should be able to process per
second on a beefy server with, say, 12 cores? I'm sure it depends on the
number of rules. With, say, 50 rules, think it could do 50 messages a
second?
We had a logging bottleneck with I think fewer messages than this with
fewer rules, but CPU load was low - hence it didn't look like it was
utilizing all the cores. Then we disabled normalization for now. That
was with rsyslog 8.2.2. Has performance improved in 8.7?
Thanks!
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
