Hi
I have seen strange issues with my firewall environment. It has the following
9 interfaces:
Internet - em1 (dual intel pci-e)
lan - em0 (dual intel pci-e)
pfsync - em2 (dual intel pci-x)
dmz1 - em3 (dual intel pci-x)
dmz2 - bge0 (server build in broadcom)
dmz3-6 - sis0-4 (soekris pci quad)
em0, em1 and em2 run at gig speed. All other at 100.
I use carp on all interfaces [ except pfsync ;-) ].
I also have net.inet.carp.preempt=1
The primary fw is master for all carp interfaces and everything *mostly* works
perfect.
THE PROBLEM:
Sometimes when I reboot one of the firewalls not all carp masters goes back...
(but they often do). So right now in this minute I have a primary firewall
that is carp master for all networks except for dmz3 where it's backup. The
secondary firewall is now backup for all networks except for dmz3 where is't
carp master. This leads to randow lockups when you try to access a servers on
that dmz3. This as the traffic (lan -> dmz3) goes in through one fw and back
through the other fw.
I did some tests when this problem happened before. I pulled the RJ45 pluggs
of my em3 interfaces from the switch and put them back. Still same problem...
Then I took an old 3com switch and moved the em3 interfaces to it so the carp
could start again. And now the carp goes back OK... A fw reboot could also
help... This leads to the real question if it's a carp bug or my old cisco
2948L-G3 switch that is buggy.
(the Cisco 2948L-G3 has 4 VLAN of 12 ports each for dmz:s)
I attached some output below when this problem is live.
Ohhh, by the way...
* I use 3.8 stable (cvs checkout jan 28).
* I have dmz5 and dmz6 with no connections (no cables connected) right now, So
the carp is in init state on these interfaces.
#master fw#
carp2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: MASTER carpdev em3 vhid 3 advbase 1 advskew 0
groups: carp
inet 212.247.187.129 netmask 0xfffffff0 broadcast 212.247.187.143
carp3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: MASTER carpdev bge0 vhid 4 advbase 1 advskew 0
groups: carp
inet 212.247.187.145 netmask 0xfffffff8 broadcast 212.247.187.151
carp4: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: BACKUP carpdev sis0 vhid 5 advbase 1 advskew 0
groups: carp
inet 192.168.12.1 netmask 0xffffff00 broadcast 192.168.12.255
#dump from master#
[EMAIL PROTECTED]:/etc#tcpdump -i sis0 proto carp
tcpdump: listening on sis0, link-type EN10MB
17:43:41.944998 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
17:43:43.895011 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
17:43:45.845014 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
17:43:47.795019 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
17:43:49.745030 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
17:43:51.695034 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
#backup fw#
carp2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: BACKUP carpdev em3 vhid 3 advbase 1 advskew 100
groups: carp
inet 212.247.187.129 netmask 0xfffffff0 broadcast 212.247.187.143
carp3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: BACKUP carpdev bge0 vhid 4 advbase 1 advskew 100
groups: carp
inet 212.247.187.145 netmask 0xfffffff8 broadcast 212.247.187.151
carp4: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
carp: MASTER carpdev sis0 vhid 5 advbase 1 advskew 100
groups: carp
inet 192.168.12.1 netmask 0xffffff00 broadcast 192.168.12.255
#dump from backup fw#
[EMAIL PROTECTED]:~#tcpdump -i sis0 proto carp
tcpdump: listening on sis0, link-type EN10MB
17:43:43.923264 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
17:43:45.873261 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
17:43:47.823259 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
17:43:49.773260 CARPv2-advertise 36: vhid=5 advbase=1 advskew=240 (DF) [tos
0x10]
Suggestions *very* much appreciated
Thanks in advance
/Per-Olov Sjöholm
