What's the problem with impstats? Sent from phone, thus brief. Am 18.09.2013 17:26 schrieb "Robert" <[email protected]>:
> Thanks for looking at the isue, it is a little messy , but below is my > current config file, the impstats module is not working on either legacy or > current syntax, also the rules seem to perform better with the legacy "if > then", if you see something that's out of place or something to improve it > please let me know thanks! > > > ############################################################################################################# > > ######################################################################################################### > > # Run this in debugger mode to troubleshoot make sure to place it on top > of this file > module(load="impstats" interval="600" severity="7") > # # to actually gather the data: > syslog.=debug /var/log/rsyslog-stats > # > # #Legacy style impstats module > # #$ModLoad impstats > # #$PStatInterval 600 > # #$PStatSeverity 7 > # > # #syslog.=debug /var/log/rsyslog-stats > # > > # rsyslog v7 configuration file > > # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html > # If you experience problems, see > http://www.rsyslog.com/doc/troubleshoot.html > > #### MODULES #### > > module(load="imuxsock") # needs to be done just once Robert 8-9-13 > #SysSock.FlowControl=(:"on") # enable flow control (use if needed) Robert > 8-9-13 > #$ModLoad imuxsock # provides support for local system logging (e.g. via > logger command) > > module(load="imklog") > #$ModLoad imklog # provides kernel logging support (previously done by > rklogd) > #$ModLoad immark # provides --MARK-- message capability > > # Provides UDP syslog reception > module(load="imudp" > TimeRequery="10") # needs to be done just once Robert 8-9-13 > input(type="imudp" port="514") # Robert 8-9-13 > > #$ModLoad imudp > #$UDPServerRun 514 > #$UDPServerTimeRequery 10 > > # Provides TCP syslog reception > #$ModLoad imptcp > #$InputTCPServerRun 514 > > #### GLOBAL DIRECTIVES #### > > # Use default timestamp format > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > # File syncing capability is disabled by default. This feature is usually > not required, > # not useful and an extreme performance hit > #$ActionFileEnableSync on > > # Include all config files in /etc/rsyslog.d/ > $IncludeConfig /etc/rsyslog.d/*.conf > > # Set Buffer Size - default is 4k > $OMFileAsyncWriting on > #$OMFileFlushOnTXEnd on > #$OMFileFlushInterval 1 > #$OMFileZipLevel 9 > $OMFileIOBufferSize 1000k # modified 9-18-13 > > #Turn on Main Ruleset Robert 8-20-13 > #$RulesetCreateMainQueue on > > # Set Main Message Queue Size - default is 10000 > $MainMsgQueueType FixedArray #LinkedList > $MainMsgQueueSize 200000000 > $MainMsgQueueWorkerThreads 8 > #$MainMsgQueueWorkerTimeoutThreadShutdown -1 > $MainMsgQueueDequeueBatchSize 1000 > #$MainMsgQueueSaveOnShutdown on > $InputUDPMaxSessions 40000000 > > #### RULES #### > > # Log all kernel messages to the console. > # Logging much else clutters up the screen. > #kern.* > # /dev/console > > #Specific ruleset for remote messages > #$Ruleset <name> > > #Module (load="builtin:omfile") > #*.* action(type="omfile" > # DirCreateMode="0700" > # FileCreateMode="0644" > # File="/var/log/test/alllogs") > > #switch back to default ruleset > #$Ruleset RSYSLOG DefaultRuleset > > #^Fx Begin action Robert 8-20-13 > # $ActionOmrulesetRulesetName somename > $ActionQueueWorkerThreads 8 > $ActionQueueSize 10000000 > $ActionQueueType FixedArray #LinkedList - use asynchronous processing > > #$ActionQueueFileName aaaaafwd # set file name, also enables disk mode > #$ActionQueueMaxFileSize 400m # default: 1m, should be 1% of MaxDiskSpace > #$ActionQueueMaxDiskSpace 40g # space limit (use as much as possible) > #$ActionQueueTimeoutEnqueue 0 # throtteling, 0 disables throttling and > discard immediately if queue is full > #$ActionQueueDequeueBatchSize 500000 > #$ActionResumeRetryCount -1 # infinite retries on insert failure > > #$ActionResumeInterval 1 # faster than default 30 second delay > #$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down > > #*.* /var/log/test/f_all #Robert 8-21-13 > > #:hostname, contains, "fdfw" /var/log/test/f_fw > #:hostname, contains, "mail" /var/log/test/f_mail > #:hostname, contains, "shib" /var/log/test/f_shib > #:hostname, contains, "pdc" /var/log/test/f_ad > #:hostname, contains, "networks" /var/log/test/f_networks > #:hostname, contains, "rout" /var/log/test/f_router > #:hostname, contains, "vm" /var/log/test/f_vm > #:hostname, contains, "pix" /var/log/test/f_pix > > if $hostname contains 'pdc' then /var/log/test/f_ad > & ~ > if $hostname contains 'fdfw' then /var/log/test/f_fw > & ~ > if $hostname contains 'networks' then /var/log/test/f_networks > & ~ > if $hostname contains 'shib' then /var/log/test/f_shib > & ~ > if $hostname contains 'mail' then /var/log/test/f_mail > & ~ > if $hostname contains 'vm' then /var/log/test/f_vm > & ~ > if $hostname contains 'pix' then /var/log/test/f_pix > & ~ > if $hostname contains 'rout' then /var/log/test/f_router > & ~ > > # Log anything (except mail) of level info or higher. > # Don't log private authentication messages! > #*.info;mail.none;authpriv.none;cron.none /var/log/messages > #*.debug /var/log/messages > # > # The authpriv file has restricted access. > #authpriv.* /var/log/secure > # > # Log all the mail messages in one place. > #mail.* -/var/log/maillog > # > # > # Log cron stuff > #cron.* /var/log/cron > # > # Everybody gets emergency messages > #*.emerg * > # > # Save news errors of level crit and higher in a special file. > #uucp,news.crit /var/log/spooler > # > # Save boot messages also to boot.log > #local7.* /var/log/boot.log > # > # > # ### begin forwarding rule ### > # The statement between the begin ... end define a SINGLE forwarding > # rule. They belong together, do NOT split them. If you create multiple > # forwarding rules, duplicate the whole block! > # Remote Logging (we use TCP for reliable delivery) > # > # An on-disk queue is created for this action. If the remote host is > # down, messages are spooled to disk and sent when it is up again. > #$WorkDirectory /var/lib/rsyslog # where to place spool files > #$ActionQueueFileName fwdRule1 # unique name prefix for spool files > #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) > #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown > #$ActionQueueType LinkedList # run asynchronously > #$ActionResumeRetryCount -1 # infinite retries if host is down > # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > #*.* @@remote-host:514 > # ### end of the forwarding rule ### > > > ################################################################################################ > > ######################################################################################################### > ----- Original Message ----- > From: Rainer Gerhards > Sent: 09/18/13 03:00 AM > To: rsyslog-users > Subject: Re: [rsyslog] performance tweaking > > On Wed, Sep 18, 2013 at 1:42 AM, David Lang <[email protected]> wrote: > At > this point, it's probably not the input thread then, which thread is > > running out of CPU? You mention running top with the H flag, what threads > > is using most of the CPU? > > if one of the outputs is not able to keep up, > rsyslog will end up loosing > messages > Usually an excellent way to spot > the culprit is also to have a look at rsyslog's internal stats (the > usefulness for performance tweaking depends a bit on the config, > though...). I highly recommend gathering them. It's done via the impstats > module: http://www.rsyslog.com/doc/impstats.html Also, it would be good > to see the full rsyslog configuration, not just a snippet. Sorry if you > already posted it, but I couldn't get hold of it in the large volume of > mail ;) Rainer _______________________________________________ rsyslog > mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ What's up with rsy > slog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC > mailing list, posts are ARCHIVED by a myriad of sites beyond our control. > PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > > > > > > Robert. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
