Hello, I'm not sure which of the following will apply to your version of rsyslog, which is quite old. By the way, simply upgrading rsyslog to one of the latest releases might improve your performance quite dramatically. If that is an option for you.
I don't know how to improve that particular sample, but you might want to try to tweak your main queue (the one that receives your messages and, if I'm not missing something, does the stuff from your config snippet): - increase the number of threads for your main queue via $MainMsgQueueWorkerThreads<http://www.rsyslog.com/doc/rsyslog_conf_global.html>. The default is 1, and I guess you can go up on a multi-core machine - increase the size of the queue via $MainMsgQueueSize<http://www.rsyslog.com/doc/rsconf1_mainmsgqueuesize.html>. By default it's 10K messages, which might be a little small at the rate you're processing logs Also, unless you're having a slow output, like a database, you're probably better off having no action queues. More information on why here<http://blog.gerhards.net/2013/06/rsyslog-performance-main-and-action.html>. And more information about queues in general here<http://www.rsyslog.com/doc/queues.html> . Best regards, Radu 2013/7/30 Esmq <[email protected]> > hi,all > > > i have about 20 servers running as rsyslog client that forward logs to a > rsyslog server via TCP, > > > each client forwards at a rate of 200 messages per second, that's to say > my rsyslog server should process 5k messages per second. > > > though i really know that process 5k messages per seconds should not a > problem of rsyslog ~ > > > when i use "netstat" command, it some times show there a non-zero value of > Recv-Q on rsyslog-server, which means rsyslog unable to accept the incoming > message quickly... > > > i thought there's a problem of my rsyslog configuration, especially for i > using some complex template to generate dynamic file name, and some regex > to filter the logs~ > > > here is the config sample ( i use rsyslog 4.6.4 on debian squeeze) > > ################################################################################################# > $template relayLogFormat, "%msg:2:$%\n" > #using dynamic file name, according to some specific message field > $template > valySplitLogFile,"/home/data/logs/%msg:R,ERE,2,ZERO:.*(chl|channel)=([^&? > ]+)\.(test\.com|example\.com).*--end%-valy-%$YEAR%%$MONTH%%$DAY%.access.log" > #filtering the log and put it to appropriate file > :msg, ereregex, "http://valy\.nie\.test\.com/query\?.*(chl|channel)=([^&? > ]+)\.(test\.com|example\.com).*" -?valySplitLogFile;relayLogFormat > #following is some other config... > > > i guess the above config is an extreme performance hit, but i don't konw > how to make a improvement~ > > > any suggestion for improvement will be appreciate, > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
