I'm happy to share more details, but I thought it wise to at least ask the question first. I will turn on debugging today in dev and aim some traffic at it with tcpreplay to gather some more information. I expect much of that information may be more education to me about how rsyslog works rather than what's "wrong". I expect rsyslog is functioning as it should, and that I simply need to learn how to configure it better for our purposes.
On Tue, May 13, 2014 at 10:02 AM, Justin Haynes <[email protected]>wrote: > I have input of about 2500 packets per second of UDP traffic. rsyslog > will only process about 1500 and the queue continues to accumulate > messages. My main question is how can I improve performance? > > I have more detailed questions to ask below, but that is the main one. I > am open to any input you believe might be helpful. I'm as interested to > understand better how this works as I am to find a workable solution. > > This is rsyslog 5.8.10 which is standard for RHEL 6. An upgrade is not an > option due to policies about straying from software RedHat can support. > > In one instance we have 500 rules, where each rule is a single hostname. > I have prioritized top talkers to the top of the ruleset, thiinking that > doing so would improve performance as it would with a firewall. Each rules > has a drop as the last action. For example: > > :hostname, isequal, "<hostname>" > @?template_to_write_files_to_a_specific_folder > @xxx.xxx.xx.xxx:xxx > &~ > > Moving busier rules to the top did not make much difference in performance > with regard to pps. maybe 100pps differnce even though the top talker > constituted 80% of the traffic and that rule used to be ~rule 200. > > I only the Main Queue (excluding all the queues needed for each and every > action). I only have one ruleset, and configuration is very simple and > shown below. > > Should I distribute my rules across rulesets somehow and have a thread > assigned to each one? > Does rule processing begin at the beginning and end at the end? > Do worker threads automatically get assigned unique workload or are they > competing to match their messages to the rules? > Do the action queues matter in this case? I would think not, but FYI, all > destinations are UDP. > > > $ModLoad imudp > $UDPServerAddress xxx.xxx.xxx.xxx > $UDPServerRun 514 > > $UDPServerTimeRequery 8 > > $ModLoad imtcp > $InputTCPServerRun xxxxx > > > #$ActionQueueType Direct > $MainMsgQueueWorkerThreadMinimumMessages 100 > $MainMsgQueueTimeoutEnqueue 0 > $MainMsgQueueWorkerThreads 4 > $MainMsgQueueWorkerTimeoutThreadShutdown 20000 > $MainMsgQueueSize 50000000 > $MainMsgQueueDequeueBatchSize 1000 > #$MainMsgQueueType FixedArray > $MainMsgQueueType LinkedList > > > $IncludeConfig rules1.conf > $IncludeConfig rules2.conf > $IncludeConfig rules3.conf > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
