First off, I see an inconsistancy in your config
On Sat, 5 Mar 2011, Olivier Fauchon wrote:
Hi.
I use rsyslogd for centralizing my webserver's logs.
Webservers runs rsyslogd too to locally process the logs, and send them to the
central log server with OMRELP module.
Here is the configuration:
$template DailyAccess,"/data/opt/logs/apache/access.%$YEAR%-%$MONTH%-%$DAY%.log"
$template DailyError,"/data/opt/logs/apache/error.%$YEAR%-%$MONTH%-%$DAY%.log"
$template apache,"%HOSTNAME% -%msg%\n"
$template tracedigitick,"/data/opt/logs/apache/%app-name%/%$now%.log"
$WorkDirectory /data/rsyslog # where to place spool files
$MainMsgQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueFileName remotequeue # unique name prefix for spool files
$ActionQueueMaxDiskSpace 10g # 1gb space limit (use as much as possible)
you set a disk space limit
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList # run asynchronously
but then you pick a memory-only queue type that doesn't use disk
there's also no particular queue type needed to run asynchronously, so the
comment here is misleading.
$ActionResumeRetryCount -1 # infinety retries if host is do
$ActionResumeInterval 10
<SNIP>
THE PROBLEM:
This morning, my central rsyslog server was unreachable, and all the
webserver went bad (high load, not responding to requests)
I discovered that they were stuck to Log 'L' state (in apache's server-status
page)
System console 'logger' command was terribly slow too. (4-5 seconds to process)
what is happening is that you have told rsyslog to be very careful about
forwarding messages and make sure that they get to the destination before
forgetting about them.
but what happens is that when the destination stops accepting messages,
rsyslog will queue the messages until it's queue is full, and then stop
accepting new messages (which causes apache to pause)
If you want to have your sending rsyslog instances use disk to store
messages that it can't get acknowldgement of you need to use the
disk-assisted queue type
If you stick with memory based queues, you probably want to bump up the
max queue size a lot to give you more time.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
