When one interface fails in a carp setup, it is my understanding that if
net.inet.carp.preempt is set to '1', that both interfaces on the single
machine should fail. However I'm not seeing this happening and I'm
hoping this is why I'm dropping connections during fail over. If I fail
both interfaces at the exact same time I have no problems with dropped
connections.
My setup is as follows, I'll mention that pfsync traffic is going over
the local network. Also, I've tried with setting advskew to 100 one one
firewall, as well as not setting it at all with net.inet.carp.preempt set.
Firewall 1:
# sysctl -a | grep carp
net.inet.carp.allow=1
net.inet.carp.preempt=1
net.inet.carp.log=1
net.inet.carp.arpbalance=0
# ifconfig -a
...
gem0:
flags=8b63<UP,BROADCAST,NOTRAILERS,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1500
lladdr 00:03:ba:94:5f:06
groups: egress
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 209.82.103.244 netmask 0xfffffff8 broadcast 209.82.103.247
inet6 fe80::203:baff:fe94:5f06%gem0 prefixlen 64 scopeid 0x1
gem1:
flags=8b63<UP,BROADCAST,NOTRAILERS,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1500
lladdr 00:03:ba:94:5f:07
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 192.168.121.2 netmask 0xffffff00 broadcast 192.168.121.255
inet6 fe80::203:baff:fe94:5f07%gem1 prefixlen 64 scopeid 0x2
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33192
pfsync0: flags=41<UP,RUNNING> mtu 1348
pfsync: syncdev: gem1 maxupd: 128
enc0: flags=0<> mtu 1536
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: BACKUP carpdev gem0 vhid 1 advbase 1 advskew 100
groups: carp
inet 209.82.103.246 netmask 0xfffffff8 broadcast 209.82.103.247
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: BACKUP carpdev gem1 vhid 2 advbase 1 advskew 100
groups: carp
inet 192.168.121.1 netmask 0xffffff00 broadcast 192.168.121.255
# cat pf.conf | grep -v "#"
nat on gem0 from 192.168.121.0/24 to any -> 209.82.103.246
rdr pass on gem0 proto tcp from any to any port 25 -> 192.168.121.10
rdr pass on gem0 proto udp from any to any port 53 -> 192.168.121.10
rdr pass on gem0 proto tcp from any to any port 6881 -> 192.168.121.123
pass quick on gem1 proto pfsync
pass on { gem0 gem1 } proto carp keep state
pass out on gem0 keep state
pass in on gem0 keep state
Firewall 2:
# sysctl -a | grep carp
net.inet.carp.allow=1
net.inet.carp.preempt=1
net.inet.carp.log=1
net.inet.carp.arpbalance=0
# ifconfig -a
...
gem0:
flags=8b63<UP,BROADCAST,NOTRAILERS,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1500
lladdr 00:03:ba:94:5f:1c
groups: egress
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 209.82.103.245 netmask 0xfffffff8 broadcast 209.82.103.247
inet6 fe80::203:baff:fe94:5f1c%gem0 prefixlen 64 scopeid 0x1
gem1:
flags=8b63<UP,BROADCAST,NOTRAILERS,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1500
lladdr 00:03:ba:94:5f:1d
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 192.168.121.3 netmask 0xffffff00 broadcast 192.168.121.255
inet6 fe80::203:baff:fe94:5f1d%gem1 prefixlen 64 scopeid 0x2
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33192
pfsync0: flags=41<UP,RUNNING> mtu 1348
pfsync: syncdev: gem1 maxupd: 128
enc0: flags=0<> mtu 1536
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: MASTER carpdev gem0 vhid 1 advbase 1 advskew 0
groups: carp
inet 209.82.103.246 netmask 0xfffffff8 broadcast 209.82.103.247
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: MASTER carpdev gem1 vhid 2 advbase 1 advskew 0
groups: carp
inet 192.168.121.1 netmask 0xffffff00 broadcast 192.168.121.255
# cat pf.conf | grep -v "#"
nat on gem0 from 192.168.121.0/24 to any -> 209.82.103.246
rdr pass on gem0 proto tcp from any to any port 25 -> 192.168.121.10
rdr pass on gem0 proto udp from any to any port 53 -> 192.168.121.10
rdr pass on gem0 proto tcp from any to any port 6881 -> 192.168.121.123
pass quick on { gem1 } proto pfsync
pass on { gem0 gem1 } proto carp keep state
pass out on gem0 all keep state
pass in on gem0 all keep state
Any help on this would be appreciated.
Thanks,
Chris
