On Thu, 20 Jun 2013, Timothy Ehlers wrote:
Why cant 1 rsyslog do what you need?
This is a good question to think about.
It depends on what you are doing with the data. As we have seen here over the
last few days, we do have people who have rsyslog doing enough with the data
that more than one system is needed to handle the load.
But if you do not need more than one system to handle the load, but you want to
have two servers for redundancy, there are a number of approaches that you can
take.
Personally, I like to have things configured for simple failover instead of load
balancing. It's much easier to troubleshoot when you know that the 'active' box
is going to see every message.
If you go active/passive instead of active/active, there are a few ways to keep
the data synced.
1. have the data arrive on both boxes at the same time.
1a. have the clients send to both
1b. use CLUSTERIP with UDP transport to have both boxes receive the same data at
the same time.
2. have the active system send to the backup
2a. have one send to the other (or in your example, both send to the other)
2b. have one system write the data to disk and then rsync the result
2c. use shared storage to make the data available to the backup
2d. use something like drbd to simulate shared storage
Personally I dislike shared storage. The problem with it is that if one system
corrupts the data, the corruption gets propogated to the other.
Saving the files and using rsync to get them to the backup gives you a timeframe
where the data only exists on one system. This may be acceptable to you or it
may not.
Sending to both systems at once, or having both send to the other has the
problem that the resulting data files on the two boxes are different. When a box
goes down for a while and you get it back (or build a replacement), you don't
have a good way of filling the gap (if you rsync data over from the other box,
you may duplicate or miss some logs since they arrived in a different order)
When I can get away with it, I much prefer to have one system write the files
and then rsync them over to the backup. That way when there is an outage I can
fill the gap by doing a rsync the other way.
To minimize the impact of failures, I have my core systems both receive the
data. Every minute (cron), they both roll the logs, then check to see if they
are the active box of the pair or not. If they are, they save the data to the
archive directory and push a copy of that file to the other box. If they are not
the active box, they throw the data away (when I get a chance, I'll have them
put it in a short-term trash location so that if something happens and the data
never arrives from the other box I still have something)
When a failover happens you may end up with duplicated or missing logs (since
there is no such thing as truely simultanious actions on multiple boxes), but it
will be a very small window.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
