Thank you, good to know! On 1/28/15 5:30 PM, David Lang wrote: > By default you will only use one core for mmnormalize, but you can > configure it to use more (although doing so when you don't need to ends > up slowing you down due to contention on the queue) > > I don't have info on how fast to expect it to be, it would vary by quite > a bit depending on the speed of your cpu/ram/cache. > > the number of rules does not significantly affect the speed of > mmnormalize, the way that it compiles the rules when loading the ruleset > means that the length of the log message (and how complex the rule is > that match it) matter more than how many rules there are. > > If you are running into bottlenecks at 10s of logs/sec then something is > very wrong. Normally rsyslog can do hundreds of thousands of logs/sec, > so I would still expect thousands of logs/sec with mmnormalize (as long > as you don't use regex matches) > > If you think that it's being limited by only using one core, use the 'H' > option in top to show the individual threads, see if any of the rsyslog > threads show that they are using 100% cpu. If not then you have some > other problem. > > David Lang > > On Wed, 28 Jan 2015, Micah Yoder wrote: > >> Hi, >> >> Couple questions... >> >> Does mmnormalize use multiple cores? I see in the liblognorm >> documentation that it does not internally, but it can be called from >> multiple threads. >> >> Any ideas as to how many messages it should be able to process per >> second on a beefy server with, say, 12 cores? I'm sure it depends on the >> number of rules. With, say, 50 rules, think it could do 50 messages a >> second? >> >> We had a logging bottleneck with I think fewer messages than this with >> fewer rules, but CPU load was low - hence it didn't look like it was >> utilizing all the cores. Then we disabled normalization for now. That >> was with rsyslog 8.2.2. Has performance improved in 8.7? >> >> Thanks! >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if you DON'T LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT.
_______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
