I have input of about 2500 packets per second of UDP traffic. rsyslog will only process about 1500 and the queue continues to accumulate messages. My main question is how can I improve performance?
I have more detailed questions to ask below, but that is the main one. I am open to any input you believe might be helpful. I'm as interested to understand better how this works as I am to find a workable solution. This is rsyslog 5.8.10 which is standard for RHEL 6. An upgrade is not an option due to policies about straying from software RedHat can support. In one instance we have 500 rules, where each rule is a single hostname. I have prioritized top talkers to the top of the ruleset, thiinking that doing so would improve performance as it would with a firewall. Each rules has a drop as the last action. For example: :hostname, isequal, "<hostname>" @?template_to_write_files_to_a_specific_folder @xxx.xxx.xx.xxx:xxx &~ Moving busier rules to the top did not make much difference in performance with regard to pps. maybe 100pps differnce even though the top talker constituted 80% of the traffic and that rule used to be ~rule 200. I only the Main Queue (excluding all the queues needed for each and every action). I only have one ruleset, and configuration is very simple and shown below. Should I distribute my rules across rulesets somehow and have a thread assigned to each one? Does rule processing begin at the beginning and end at the end? Do worker threads automatically get assigned unique workload or are they competing to match their messages to the rules? Do the action queues matter in this case? I would think not, but FYI, all destinations are UDP. $ModLoad imudp $UDPServerAddress xxx.xxx.xxx.xxx $UDPServerRun 514 $UDPServerTimeRequery 8 $ModLoad imtcp $InputTCPServerRun xxxxx #$ActionQueueType Direct $MainMsgQueueWorkerThreadMinimumMessages 100 $MainMsgQueueTimeoutEnqueue 0 $MainMsgQueueWorkerThreads 4 $MainMsgQueueWorkerTimeoutThreadShutdown 20000 $MainMsgQueueSize 50000000 $MainMsgQueueDequeueBatchSize 1000 #$MainMsgQueueType FixedArray $MainMsgQueueType LinkedList $IncludeConfig rules1.conf $IncludeConfig rules2.conf $IncludeConfig rules3.conf _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
