On Mon, 30 Jan 2012, [email protected] wrote:
Do you have any thoughts on how to solve the problem?
what I do is use iptables CLUSTERIP feature on linux to setup one IP address
that gets shared across the cluster of systems. heartbeat (with the pacemaker
cluster management layer) can keep track of the cluster and make sure that
there is always a box handling the traffic
what this does is use a multicast MAC address to send the traffic to multiple
machines. The kernel then does a hash on (one or more of) source IP, source
port, destination IP, destination port. It then divides this hash into
buckets (I am machine 1 of 10) and if it falls into the bucket for this
machine, it then sends the packet on to the application, otherwise the kernel
drops the packet.
This has the advantage of not needing any other systems, it can be done
entirely on the receiving cluster.
here is a page on how to configure pacemaker to do this.
www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Clusters_from_Scratch/ch08s06.html
I primarily use this with UDP traffic, which gives me the added advantage
that I can have multiple clusters receiving the same traffic. I did
extensive testing a couple of years ago, and going across a cisco 3500
switch I was able to handle traffic up to ~380K logs/sec (~250 byte log
messages) with no message losses with UDP over several billion log
messages sent to a dozen destination machines.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
