They are in cluster and it's definitely to do with timeout on rabbit(ampq) clients, but I was after the HAProxy solution. I.e. I wanted all sessions to a DOWN node killed immediately when HAProxy detects that a node has gone DOWN. Baptiste's reply seems to hit the spot with this new feature in 1.5
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#on-marked-down I can also see that there is http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5.2-on-marked-up That is also very useful for my scenario as I would like all connections to failover back to rabbit01 when it goes back up. Resetting the connection to BACKUP will cause my client to reconnect the rabbit01 that's just came up. On Sat, Jan 18, 2014 at 11:56 AM, Thomas Spicer <[email protected]>wrote: > This may more to due with Rabbit (or AMPQ clients) than HAProxy. Do you > have Rabbit 01 and 02 in a cluster or are they separate nodes? > > > On Jan 16, 2014, at 11:28 PM, Andrei Chevenkov <[email protected]> wrote: > > Hi, > > I am pretty new to haproxy and trying to do ha setup for rabbitmq. > > Setup is pretyt simple: > > listen rabbitmq 192.168.69.106:5672 > mode tcp > balance leastconn > option tcplog > option tcpka > server rabbit01 192.168.69.107:5672 check inter 1000 downinter 5000 > fall 1 on-error mark-down > server rabbit02 192.168.69.108:5672 check inter 1000 downinter 5000 > fall 1 on-error mark-down backup > > Everything seems to work well, but I noticed one strange behaviour. If I > gracefully shut down rabbit01, message producer (client) connected to > 192.168.69.106:5672 notices broken connection pretty much straight away, > attempts a reconnect, hits rabbit02 and all is well. > > However if I simply "turn off" rabbit01, message producer (client) notices > broken connection only in around 30 seconds or so. And while the break is > undetected it keeps pushing messages thinking there is still someone on > another side. > > Now, from the statistic report I can see that as soon as I turn off the > box, rabbit01 is marked as "DOWN", but its "Sessions Current" is still set > to 1? In the logs this comes up pretty much immediately after the hard turn > off: > > Jan 17 17:13:51 prodlb01 haproxy[38459]: Server rabbitmq/rabbit01 is DOWN, > reason: Layer4 timeout, check duration: 1008ms. 0 active and 1 backup > servers left. Running on backup. 1 sessions active, 0 requeued, 0 remaining > in queue. > > After around 30 seconds, the connection is detected as broken and failover > happens successfully. At that time "Sessions Current" is set to 0 fo > rabbit01. > > Tried option nolinger and option abortonclose, but no luck. > > Any help would be much appreciated... > > Cheers, > Andrei > > >
