What happens if you remove the action queues? Based on impstats i saw, i would expect improvement.
Sent from phone, thus brief. Am 20.09.2013 23:10 schrieb "Robert" <[email protected]>: > Hello Everyone, > > So I went ahead and made those changes > > # Provides UDP syslog reception > module(load="imudp" > TimeRequery="10" > SchedulingPolicy="fifo" > SchedulingPriority="10") # needs to be done just once > input(type="imudp" port="514") > When I added the actionqueue on each of the logs, it queued alot of them > and the cpu took a big hit: > > 09:05:03 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle > 09:05:04 AM all 31.26 0.00 13.78 0.00 0.00 9.19 0.00 0.00 45.78 > 09:05:04 AM 0 15.05 0.00 60.22 0.00 0.00 0.00 0.00 0.00 24.73 > 09:05:04 AM 1 46.97 0.00 7.58 0.00 0.00 0.00 0.00 0.00 45.45 > 09:05:04 AM 2 13.70 0.00 2.74 0.00 0.00 0.00 0.00 0.00 83.56 > 09:05:04 AM 3 5.49 0.00 1.10 0.00 0.00 0.00 0.00 0.00 93.41 > 09:05:04 AM 4 61.00 0.00 19.00 0.00 0.00 20.00 0.00 0.00 0.00 > 09:05:04 AM 5 36.84 0.00 2.63 0.00 0.00 32.89 0.00 0.00 27.63 > 09:05:04 AM 6 48.35 0.00 3.30 0.00 0.00 9.89 0.00 0.00 38.46 > 09:05:04 AM 7 20.24 0.00 7.14 0.00 0.00 9.52 0.00 0.00 63.10 > > 09:05:04 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle > 09:05:05 AM all 31.44 0.00 16.07 0.00 0.00 8.39 0.00 0.00 44.10 > 09:05:05 AM 0 7.83 0.00 71.30 0.00 0.00 0.00 0.00 0.00 20.87 > 09:05:05 AM 1 41.67 0.00 6.67 0.00 0.00 0.00 0.00 0.00 51.67 > 09:05:05 AM 2 17.11 0.00 2.63 0.00 0.00 0.00 0.00 0.00 80.26 > 09:05:05 AM 3 5.49 0.00 2.20 0.00 0.00 0.00 0.00 0.00 92.31 > 09:05:05 AM 4 59.00 0.00 14.00 0.00 0.00 27.00 0.00 0.00 0.00 > 09:05:05 AM 5 37.18 0.00 3.85 0.00 0.00 32.05 0.00 0.00 26.92 > 09:05:05 AM 6 54.35 0.00 2.17 1.09 0.00 4.35 0.00 0.00 38.04 > 09:05:05 AM 7 34.41 0.00 4.30 0.00 0.00 2.15 0.00 0.00 59.14 > > I have added the old syntax because Im not sure how to write with the new > sytanx? whenever I write it it doesnt perform as well as the old syntax > but this is my current configuration file and I am currently emulating > 250k mps and Im logging about 245k mps. Do you see any other things on the > config that I could change to stabilize the logs? After reading the > action/main queue, omiobuffers, and queue sizes, its getting a little mixed > up, Im not sure which I should change to improve my system. Thanks for all > the help everyone > > # rsyslog v7 configuration file > > # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html > # If you experience problems, see > http://www.rsyslog.com/doc/troubleshoot.html > > #### MODULES #### > > module(load="imuxsock") # needs to be done just once Roberto 8-9-13 > #SysSock.FlowControl=(:"on") # enable flow control (use if needed) Roberto > 8-9-13 > #$ModLoad imuxsock # provides support for local system logging (e.g. via > logger command) > > module(load="imklog") > #$ModLoad imklog # provides kernel logging support (previously done by > rklogd) > #$ModLoad immark # provides --MARK-- message capability > > # Provides UDP syslog reception > module(load="imudp" > TimeRequery="10" > SchedulingPolicy="fifo" > SchedulingPriority="10") # needs to be done just once Roberto 8-9-13 > input(type="imudp" port="514") # Roberto 8-9-13 > #$IMUDPSchedulingPolicy fifo > #$IMUDPSchedulingPriority 10 > > #### GLOBAL DIRECTIVES #### > > # Use default timestamp format > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > # File syncing capability is disabled by default. This feature is usually > not required, > # not useful and an extreme performance hit > #$ActionFileEnableSync on > > # Include all config files in /etc/rsyslog.d/ > $IncludeConfig /etc/rsyslog.d/*.conf > > # Set Main Message Queue Size - default is 10000 > $MainMsgQueueType FixedArray #LinkedList > $MainMsgQueueSize 20000000 > $MainMsgQueueWorkerThreads 8 > #$MainMsgQueueWorkerTimeoutThreadShutdown -1 > $MainMsgQueueDequeueBatchSize 1000 > #$MainMsgQueueSaveOnShutdown on > $InputUDPMaxSessions 40000000 > > #### RULES #### > > $ActionQueueWorkerThreads 8 > #$ActionQueueSize 10000000 > $ActionQueueType FixedArray #LinkedList - use asynchronous processing > $OMFileAsyncWriting on > #$OMFileFlushOnTXEnd on > $OMFileIOBufferSize 1000k > > :hostname, contains, "fdfw" /logs/f_fw > > #$ActionQueueWorkerThreads 8 > $ActionQueueSize 10000000 > $ActionQueueType FixedArray #LinkedList - use asynchronous processing > $OMFileAsyncWriting on > #$OMFileFlushOnTXEnd on > $OMFileIOBufferSize 1000k > > :hostname, contains, "mail" /logs/f_mail > > $ActionQueueWorkerThreads 8 > #$ActionQueueSize 10000000 > $ActionQueueType FixedArray #LinkedList - use asynchronous processing > $OMFileAsyncWriting on > #$OMFileFlushOnTXEnd on > $OMFileIOBufferSize 1000k > > :hostname, contains, "shib" /logs/f_shib > > $ActionQueueWorkerThreads 8 > #$ActionQueueSize 10000000 > $ActionQueueType FixedArray #LinkedList - use asynchronous processing > $OMFileAsyncWriting on > #$OMFileFlushOnTXEnd on > $OMFileIOBufferSize 1000k > > :hostname, contains, "pdc" /logs/f_ad > > $ActionQueueWorkerThreads 8 > #$ActionQueueSize 10000000 > $ActionQueueType FixedArray #LinkedList - use asynchronous processing > $OMFileAsyncWriting on > #$OMFileFlushOnTXEnd on > $OMFileIOBufferSize 1000k > > :hostname, contains, "networks" /logs/f_networks > > $ActionQueueWorkerThreads 8 > #$ActionQueueSize 10000000 > $ActionQueueType FixedArray #LinkedList - use asynchronous processing > $OMFileAsyncWriting on > #$OMFileFlushOnTXEnd on > $OMFileIOBufferSize 1000k > > :hostname, contains, "rout" /logs/f_router > > $ActionQueueWorkerThreads 8 > #$ActionQueueSize 10000000 > $ActionQueueType FixedArray #LinkedList - use asynchronous processing > $OMFileAsyncWriting on > #$OMFileFlushOnTXEnd on > $OMFileIOBufferSize 1000k > > :hostname, contains, "vm" /logs/f_vm > > $ActionQueueWorkerThreads 8 > #$ActionQueueSize 10000000 > > $ActionQueueWorkerThreads 8 > #$ActionQueueSize 10000000 > $ActionQueueType FixedArray #LinkedList - use asynchronous processing > $OMFileAsyncWriting on > #$OMFileFlushOnTXEnd on > $OMFileIOBufferSize 1000k > > :hostname, contains, "pix" /logs/f_pix > > > Robert. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
