Just some quick points, but I guess useful ones: The newer version offer much more performance. For details, see here: http://www.gerhards.net/download/LinuxKongress2010rsyslog.pdf
I see you use script-based filters where you could use much simpler ones. Script based filters are pretty slow. Go for selector lines or property based filters (in that order) to gain more speed. HTH Rainer > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Todd Michael Bushnell > Sent: Wednesday, March 09, 2011 11:29 PM > To: rsyslog-users > Subject: [rsyslog] Troubleshooting Rsyslog/Apache Issues > > Posted earlier on this week, but was sick at the time and thus, didn't > post enough info to warrant informed response. Here's some more > information along with some specific questions: > > Conditions: > > - Version: rsyslog-3.22.1-3.el5_5.1 > - System: Linux ******* 2.6.18-92.1.22.el5 #1 SMP Tue Dec 16 11:57:43 > EST 2008 x86_64 x86_64 x86_64 GNU/Linux > - Rsyslog Clients w/ Syslog-NG servers > - Transport: TCP > > Problem: > > Replaced Sysklog with Rsyslog. After several hours of high traffic, > Apache processes jumped from 50 to 250 and Apache eventually stopped > working. As soon as I shutdown Rsyslog and turned up Sysklog, Apache > procs went back to normal and all was resolved. > > Questions: > > 1. Running old version of Rsyslog (3.22) because this is latest > version available in CentOS repository. Will I get > performance/stability improvements upgrading to 5x (e.g. 5.6.2) or even > 6x? If so, I'll build RPMs, but assumed latest version in CentOS 5 > repository was sufficient if I don't need latest features. Am I wrong? > Should I upgrade to latest? > > 2. Couple design deficiencies (Apache & Log4 logs are double logged), > but deployed as-is until Engineering could fix deficiencies because I > wanted to emulate existing Sysklog deployment. Before fixing, I just > need to understand if there's anything about my Rsyslog configuration > that would make duplicate logging an issue when it was not an issue > with SysKlog. > > 3. The one major difference between Sysklog and Rsyslog is the use of > TCP. I know this trades performance for efficiency, but I don't know > how to determine if this is the problem. I don't see issues on the > server side and even if there are problems, I have Rsyslog configured > to queue locally if anything happens on the server side. During the > problem, there is no local queuing going on. Are there any diagnostics > I can pull to determine if remote syslog is the issue? > > Below is my config file. Is there anything in here that is a glaring > issue? If not, My ideas are to: > > 1. Upgrade, then retry. > 2. Remove duplication, then retry. > 3. Change from TCP back to UDP, then retry. > until I determine the source of the problem. > > # Configuration File > > # Provides kernel logging support (previously done by rklogd) > $ModLoad imklog > # Provides support for local system logging (e.g. via logger command) > $ModLoad imuxsock > > # Max Message Size (default 2k) > $MaxMessageSize 8192 > > # Must listen on localhost for Log4j. Need engineering to change this > $ModLoad imudp > $UDPServerAddress 127.0.0.1 > $UDPServerRun 514 > > # Use traditional timestamp format > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > # ownership/permissions > $umask 0000 > $FileOwner root > $FileGroup wheel > $FileCreateMode 0640 > > # include directory for breaking directives into separate files > (future) > $IncludeConfig /etc/rsyslog.d/ > > # forward to remote host, queueing to local disk if host is down and > memory fills up > # work (spool) files directory > $WorkDirectory /var/log/rsyslog > > # loghost1 > # in-memory queue; set for asynchronous processing (?) > $ActionQueueType LinkedList > # failover queue filename; also enables disk mode > $ActionQueueFileName failqueue-loghost1 > # infinite retries on insert failure > $ActionResumeRetryCount -1 > # save in-memory data if rsyslog shuts down > $ActionQueueSaveOnShutdown on > # remote logging of everything > *.* @@loghost1:5140 > > # loghost2 > # in-memory queue; set for asynchronous processing (?) > $ActionQueueType LinkedList > # failover queue filename; also enables disk mode > $ActionQueueFileName failqueue-loghost2 > # infinite retries on insert failure > $ActionResumeRetryCount -1 > # save in-memory data if rsyslog shuts down > $ActionQueueSaveOnShutdown on > # remote logging of everything > *.* @@loghost2:5140 > > # Log Filtering Rules > > # Emergency Messages > if $syslogseverity <= '0' then * > if $syslogseverity <= '0' then /var/log/messages > if $syslogseverity <= '0' then ~ > > # Apache > if $programname == 'logger' and ($msg contains 'access_log' or $msg > contains 'cookie_log' or $msg contains 'r > equest_log') then /var/log/http > & ~ > if $programname == 'httpd' and ($syslogfacility-text == 'local5' or > $syslogfacility-text == 'local6') then /var/log/http_err > & ~ > > # Log4j (App Logs) > if $programname == 'com.redacted.infra.syslog.Log4jSystemLogger' then > /var/log/log4j > & ~ > > # Kernel & IPTables > if $programname == 'kernel' and ($msg contains 'LOGACCEPT' or $msg > contains 'LOGDROP') then /var/log/iptables > & ~ > > # Auth Messages > if $syslogfacility-text == 'auth' or $syslogfacility-text == 'authpriv' > then /var/log/secure > & ~ > > # Mail > if $syslogfacility-text == 'mail' then /var/log/maillog > & ~ > > # Catchall for remaining log messages > *.* /var/log/messages > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
