thanks Radu,
the explanation is cool~ and i'm considering upgrading to latest release, i think it may take some time to testing and adjust the config ^_^ At 2013-07-30 19:01:37,"Radu Gheorghe" <[email protected]> wrote: >Hello, > >I'm not sure which of the following will apply to your version of rsyslog, >which is quite old. By the way, simply upgrading rsyslog to one of the >latest releases might improve your performance quite dramatically. If that >is an option for you. > >I don't know how to improve that particular sample, but you might want to >try to tweak your main queue (the one that receives your messages and, if >I'm not missing something, does the stuff from your config snippet): >- increase the number of threads for your main queue via >$MainMsgQueueWorkerThreads<http://www.rsyslog.com/doc/rsyslog_conf_global.html>. >The default is 1, and I guess you can go up on a multi-core machine >- increase the size of the queue via >$MainMsgQueueSize<http://www.rsyslog.com/doc/rsconf1_mainmsgqueuesize.html>. >By default it's 10K messages, which might be a little small at the rate >you're processing logs > >Also, unless you're having a slow output, like a database, you're probably >better off having no action queues. More information on why >here<http://blog.gerhards.net/2013/06/rsyslog-performance-main-and-action.html>. >And more information about queues in general >here<http://www.rsyslog.com/doc/queues.html> >. > >Best regards, >Radu > > >2013/7/30 Esmq <[email protected]> > >> hi,all >> >> >> i have about 20 servers running as rsyslog client that forward logs to a >> rsyslog server via TCP, >> >> >> each client forwards at a rate of 200 messages per second, that's to say >> my rsyslog server should process 5k messages per second. >> >> >> though i really know that process 5k messages per seconds should not a >> problem of rsyslog ~ >> >> >> when i use "netstat" command, it some times show there a non-zero value of >> Recv-Q on rsyslog-server, which means rsyslog unable to accept the incoming >> message quickly... >> >> >> i thought there's a problem of my rsyslog configuration, especially for i >> using some complex template to generate dynamic file name, and some regex >> to filter the logs~ >> >> >> here is the config sample ( i use rsyslog 4.6.4 on debian squeeze) >> >> ################################################################################################# >> $template relayLogFormat, "%msg:2:$%\n" >> #using dynamic file name, according to some specific message field >> $template >> valySplitLogFile,"/home/data/logs/%msg:R,ERE,2,ZERO:.*(chl|channel)=([^&? >> ]+)\.(test\.com|example\.com).*--end%-valy-%$YEAR%%$MONTH%%$DAY%.access.log" >> #filtering the log and put it to appropriate file >> :msg, ereregex, "http://valy\.nie\.test\.com/query\?.*(chl|channel)=([^&? >> ]+)\.(test\.com|example\.com).*" -?valySplitLogFile;relayLogFormat >> #following is some other config... >> >> >> i guess the above config is an extreme performance hit, but i don't konw >> how to make a improvement~ >> >> >> any suggestion for improvement will be appreciate, >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >_______________________________________________ >rsyslog mailing list >http://lists.adiscon.net/mailman/listinfo/rsyslog >http://www.rsyslog.com/professional-services/ >What's up with rsyslog? Follow https://twitter.com/rgerhards >NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE >THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
