Re: [rsyslog] performance tweaking

[David Lang]

On Tue, 24 Sep 2013, David Lang wrote:

On Tue, 24 Sep 2013, Robert wrote:

**************************************

#$ActionQueueWorkerThreads 8
#$ActionQueueSize 10000000
#$ActionQueueType FixedArray #LinkedList - use asynchronous processing
$OMFileAsyncWriting on
$OMFileFlushOnTXEnd on
$OMFileIOBufferSize 1000k
:hostname, contains, "fdfw"/logs/f_fw

**************************************

#:hostname, contains, "fdfw" action(type="omfile"
DirCreateMode="0700"
FileCreateMode="0644"
File="/logs/f_fw"
AsyncWriting="on"
FlushOnTXEnd="on"
IOBufferSize="1000k")

I have attached my mpstats, my overall cpu stats, and the rsyslog-stats from the impstats module. Another question I had was: when I write these Action and OMfile settings in the legacy syntax I think this is right, but what is the right way of doing so with the new syntax?...not sure how to add the queue sizes?

let's try a more modest per-action queue setup

:hostname, contains, "fdfw" action(type="omfile"
DirCreateMode="0700"
FileCreateMode="0644"
File="/logs/f_fw"
AsyncWriting="on"
FlushOnTXEnd="on"
IOBufferSize="8k"
queue.type="FixedArray" queue.dequeuebatchsize="100" queue.size="10000
)

and change the main queue threads from 8 to 2

ruleset(name="set1" ){
  some rules
}
ruleset(name="set2") {
  more rules
}



then you would have







David Lang

the next step is going to be to split the rules into a couple of rulesets, and have each ruleset use a queue instead of each action where 'some rules' and 'more rules' are some of your existing rules (without any queue parameters) action(type="omruleset" name="set1" queue.type="FixedArray" queue.dequeuebatchsize="100" queue.size="10000") action(type="omruleset" name="set2" queue.type="FixedArray" queue.dequeuebatchsize="100" queue.size="10000") note that I'm not sure this is exactly the right syntax for the name, it may be rulesetname instead of just name

rsyslog-stats
Description: Attachment: rsyslog-stats

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.