Thanks Radu for the input, in my sysctl there is no line for net.core.rmem_default ? Is there a particular line that it needs be created in? ----- Original Message ----- From: Radu Gheorghe Sent: 08/08/13 12:08 PM To: rsyslog-users Subject: Re: [rsyslog] performance tweaking
P.S. I see you write to files, so you might need to check the documentation of omfile for tweaks. I don't write to files to recommend something, but I see nice keywords, like "buffer" and "async" :) Without tweaking those, I've managed to write around 1M logs/s in a test on my laptop with version 7.4.3 via TCP on multiple connections. 2013/8/8 Radu Gheorghe <[email protected]> > Hi Robert, > > Here are a few quick hints to search around (and you can always come back > here if you need more info): > - try with the latest version (7.4+) because it will give you way better > performance. The Adiscon guys have an RPM repo which should work with your > RHEL just fine > - try tweaking your main message queue: total size, debatch size, adjust > the number of worker threads > - for UDP, check the imudp documentation. There are some tweaks you can do > there, and the one I felt most was with checking the system time > - for TCP, try imptcp instead of imtcp, and also check the do cumentation > for useful tweaks, like the number of threads > > I assume the logs you lose are via UDP. With sysctl, I've noticed a big > improvement when increasing the default buffer size net.core.rmem_default > (eg: 2000000 should be a good start). If you have few heavy senders, I > guess higher values are OK. If you have many senders, you have to be > careful because if the OS is allocating a lot of memory for a lot of > clients, you might run out of memory. > > Speaking of memory: it's nice to make sure the machine doesn't hit the > disk: no swap, no disk queues. > > I hope this helps, and sorry for being so short :( > > Best regards, > Radu > > > 2013/8/8 Robert Ortiz <[email protected]> > >> Hey Guys, >> >> I am new to this mailing list and I wanted to see about getting some >> pointers if possible regarding tweakin rsyslog: >> >> I am pretty new to rsyslog, and I've been given a pretty fun task... to >> test rsyslog vs syslog-ng and pick the best one, I am having a prob lem with >> rsyslog where im at 25K/mps and im dropping logs, I need to get it at 100k >> mps with and I'm not sure where the misconfiguration is if anyone could >> take a look I would really appreciatte it, >> >> my current setup: >> >> rhel 6.4 x86_64 >> rsyslog-5.8.10-2.el6.x86_64 >> Dual Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz >> 32GB RAM >> 500GB 15k raid 0 >> >> >> # rsyslog v5 configuration file >> >> # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html >> # If you experience problems, see >> http://www.rsyslog.com/doc/troubleshoot.html >> >> #### MODULES #### >> >> $ModLoad imuxsock # provides support for local system logging (e.g. via >> logger command) >> $ModLoad imklog # provides kernel logging support (previously done by >> rklogd) >> #$ModLoad immark # provides --MARK-- message capability >> >> # Provides UDP syslog reception >> $ModLoad imudp >> $UDPServerRun 514 >> # $UDPServerTimeRequery 10 >> >> # Provides TCP syslog reception >> #$ModLoad imtcp >> #$InputTCPServerRun 514 >> >> >> #### GLOBAL DIRECTIVES #### >> >> # Use default timestamp format >> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat >> >> # File syncing capability is disabled by default. This feature is usually >> not required, >> # not useful and an extreme performance hit >> #$ActionFileEnableSync on >> >> # Include all config files in /etc/rsyslog.d/ >> $IncludeConfig /etc/rsyslog.d/*.conf >> >> # Set Buffer Size - default is 4k >> # $OMFileIOBufferSize 128k >> # Set Main Message Queue Size - default is 10000 >> # $MainMsgQueueSize 50000 >> >> #### RULES #### >> >> # Log all kernel messages to the console. >> # Logging much else clutters up the screen. >> #kern.* /dev/console >> >> if $hostname contains 'pdc' then /var/log/test/f_ad >> & ~ >> if $hostname contains 'fdfw' then /var/log/test/f_fw >> & ~ >> if $hostname contains 'mail' then /var/log/test/f_mail >> & ~ >> if $hostname contains 'pix' then /var/log/test/ix >> & ~ >> if $host name contains 'rout' then /var/log/test/rout >> & ~ >> if $hostname contains 'networks' then /var/log/test/net >> & ~ >> #if $fromhost-ip == '10.0.0.10' then /var/log/test/thost >> #& ~ >> #if $hostname startswith 'virtserv' then /var/log/test/test_virtserv >> #&~ >> #if $fromhost-ip startswith '10.0.6' then /var/log/test/test_10.0.6 >> #& ~ >> >> >> # Log anything (except mail) of level info or higher. >> # Don't log private authentication messages! >> #*.info;mail.none;authpriv.none;cron.none /var/log/messages >> *.debug /var/log/messages >> >> # Log all the mail messages in one place. >> mail.* -/var/log/maillog >> >> >> # Log cron stuff >> cron.* /var/log/cron >> >> # Everybody gets emergency messages >> *.emerg * >> >> # Save news errors of level crit and higher in a special file. >> uucp,news.crit /var/log/spooler >> >> # Save boot messages also to boot.log >> local7.* /var/log/boot.log >> >> >> # ### begin forwarding rule ### >> # The statement between the begin ... e nd define a SINGLE forwarding >> # rule. They belong together, do NOT split them. If you create multiple >> # The statement between the begin ... end define a SINGLE forwarding >> # rule. They belong together, do NOT split them. If you create multiple >> # forwarding rules, duplicate the whole block! >> # Remote Logging (we use TCP for reliable delivery) >> # >> # An on-disk queue is created for this action. If the remote host is >> # down, messages are spooled to disk and sent when it is up again. >> #$WorkDirectory /var/lib/rsyslog # where to place spool files >> #$ActionQueueFileName fwdRule1 # unique name prefix for spool files >> #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) >> #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown >> #$ActionQueueType LinkedList # run asynchronously >> #$ActionResumeRetryCount -1 # infinite retries if host is down >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >> #*.* @@rem ote-host:514 >> # ### end of the forwarding rule ### >> >> >> >> Robert. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. Robert. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
