On Mon, 21 Jul 2014, Maupertuis Philippe wrote:
Hi,
We are trying to strengthen our logging architecture (hundreds of clients, soon
thousands).
We currently have disk-assisted memory queues to deal with a temporary shortage
of the central server.
However this is good only for a "short" period of time.
should the central server goes under for a "long" time the queues would fill up
and ultimately the logs are lost and the production server are strained.
We would like to implement an automatic failover in case the central server is
unreachable.
At a glance, the $ActionExecOnlyWhenPreviousIsSuspended seemed promising.
However, after reading this post,
http://blog.gerhards.net/2011/03/using-failover-and-asynchornous-actions.html I
have got the feeling that I need direct queues for this to work.
I have done a few tests which seem to confirm that.
I can't have both disk-assisted memory queues and
$ActionExecOnlyWhenPreviousIsSuspended
Can someone confirm my statement ?
you may be able to do a disk-assisted memory queue on a ruleset, and then within
that ruleset have the conditional statement.
The problem with trying to do them both together is that the worker thread is
just delivering to the queue, there's a separate thread that then tried to pull
from the queue to deliver the message, and as such, if the worker thread can put
the message in the queue it's "successful" and not suspended.
If so, I am afraid that there is no way to implement remote logging with
failover from the client in a real world.
Disk-assisted memory queues are mandatory to avoid to slow down the client when
there is a tcp hitch (a network issue for example).
Well, one other approach is to deal with this on the server side, make the
server that's receiving the logs highly available.
how large of a buffer do you need on the client side? can you size your main
queue there to be large enough to survive your outage?
Things like this are why I recommend that you have your clients send to a local
relay server and let that relay server deal with delivering to your central
system. This lets you only deal with these sort of problems on dedicated
systems.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
