On Tue, 27 Jan 2015 12:01:37 -0500
"Leclerc, Sebastien" <sebastien.lecl...@saint-georges.ca> wrote:
>Hi,
>
>I have two firewalls in a carp failover setup, but the failover does
>not work as expected... The problem happens when I reboot the backup
>firewall (while in backup state). Just after the reboot, I have these
>entries in dmesg :
>
>carp0: state transition: BACKUP -> MASTER
>carp1: state transition: BACKUP -> MASTER
>carp0: state transition: MASTER -> BACKUP
>carp1: state transition: MASTER -> BACKUP
>
>Why would there be no mention of carp2?
>And no corresponding entries on the master?
>
>States are consistent (all backup on backup, and all master on
>master), but forwarded connections hang, until I force back the master
>with this :
> sudo ifconfig -g carp carpdemote 128
> sudo ifconfig -g carp -carpdemote 128
>Between these two commands, on the backup firewall, I see traffic
>coming from WAN and DMZ, but almost nothing from LAN, so it may be
>related to the LAN switch. I cannot see what the problem is though...
>
>Here is the setup :
>
>On both firewalls :
> - em0 is connected to WAN
> - em1 is connected to LAN
> - em2 is connected to DMZ
> - em3 is interconnected with a crossover cable, used for pfsync and
> rdist
>
>WAN and DMZ connections are on the same switch, but on different
>untagged VLANs (Procurve 2524) LAN is on a separate layer 3 switch
>(Procurve 5300xl)
>
>Another strange behavior :
>With tcpdump, on the backup, I can see this traffic :
> - on em1 and em2, I see only carp advertisements to the configured
> unicast IP address and physical MAC address
> - on em3, I see only pfsync packets
> - but on em0, I see carp advertisements, but also a lot of traffic
> from the ISP router's MAC, to the virtual MAC (00:00:5e:00:01:01)
>Which situation is normal? (em0 with lots of packets, or em1/em2 with
>only carp advertisements) The only difference I see :
> - on em0, both firewalls and the ISP router are connected to the
> switch
> - on em1, both firewalls are connected to the L3 switch, which is
> also the router
> - on em2, there is no router, the firewalls communicate directly with
> hosts connected on the switch
>
>
>Common configuration (public addresses anonymized, but the network
>sizes are correct) :
>
>/etc/mygate
>192.0.2.1
>
>/etc/sysctl.conf
>net.inet.carp.preempt=1
>net.inet.ip.forwarding=1
>
>/etc/pf.conf (excerpt only)
>ext_if = "em0"
>ext_if_carp = "carp0"
>int_if = "em1"
>int_if_carp = "carp1"
>dmz_if = "em2"
>dmz_if_carp = "carp2"
>sync_if = "em3"
>set skip on lo
>set skip on $sync_if
>pass quick on { $int_if, $ext_if, $dmz_if } inet proto carp keep state
>(no-sync)
>
>
>Firewall A (expected to be always master) :
>OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar 5 09:37:46 MST 2014
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
>
>/etc/hostname.em0
>inet 192.168.3.9/30
>
>/etc/hostname.em1
>inet 192.168.3.1/29
>!route add 192.168.0.0/16 192.168.3.5
>!route add 172.16.0.0/12 192.168.3.5
>
>/etc/hostname.em2
>inet 192.168.3.13/30
>
>/etc/hostname.em3
>inet 192.168.3.17 255.255.255.252
>
>/etc/hostname.carp0
>advskew 0 carpdev em0 carppeer 192.168.3.10 pass secret1 state master
>vhid 1 inet 192.0.2.2/28
>alias 192.0.2.3/32
>alias 192.0.2.4/32
>alias 192.0.2.5/32
>
>/etc/hostname.carp1
>advskew 0 carpdev em1 carppeer 192.168.3.4 pass secret2 state master
>vhid 2 inet 192.168.3.6/32
>
>/etc/hostname.carp2
>advskew 0 carpdev em2 carppeer 192.168.3.14 pass secret3 state master
>vhid 3 inet 192.0.2.17/28
>alias 192.0.2.29/32
>
>/etc/hostname.pfsync0
>up
>syncdev em3
>syncpeer 192.168.3.18
>
>
>Firewall B (expected to be always backup) :
>OpenBSD 5.6 (GENERIC.MP) #5: Thu Dec 11 09:51:08 CET 2014
>
> r...@stable-56-amd64.mtier.org:/binpatchng/work-binpatch56-amd64/src/sys/arch/amd64/compile/GENERIC.MP
>
>/etc/hostname.em0
>inet 192.168.3.10/30
>
>/etc/hostname.em1
>inet 192.168.3.4/29
>!route add 192.168.0.0/16 192.168.3.5
>!route add 172.16.0.0/12 192.168.3.5
>
>/etc/hostname.em2
>inet 192.168.3.14/30
>
>/etc/hostname.em3
>inet 192.168.3.18/30
>
>/etc/hostname.carp0
>advskew 200 carpdev em0 carppeer 192.168.3.9 pass secret1 state backup
>vhid 1 inet 192.0.2.2/28
>alias 192.0.2.3/32
>alias 192.0.2.4/32
>alias 192.0.2.5/32
>
>/etc/hostname.carp1
>advskew 200 carpdev em1 carppeer 192.168.3.1 pass secret2 state backup
>vhid 2 inet 192.168.3.6/32
>
>/etc/hostname.carp2
>advskew 200 carpdev em2 carppeer 192.168.3.13 pass secret3 state
>backup vhid 3 inet 192.0.2.17/28
>alias 192.0.2.29/32
>
>/etc/hostname.pfsync0
>up
>syncdev em3
>syncpeer 192.168.3.17
>
>
>This message is already long, but if any other information would be
>helpful, I would be glad to provide it. Any help or suggestion is
>appreciated. Thank you!
>
>Sebastien
>
Sebastien,
Well, it's been many years since I ran carp, so I cannot actually help
with the carp config, but I can absolutely say that I have experienced a
lot of unexplainable weirdness with ProCurve switches, so I can
appreciate your suspicions there. I'll never buy another.
