I have an OpenBSD 5.8 stable carp setup where one of my upstream links
is serviced by a cable provider, a static IP is assigned, and I would
normally have no IP assigned to the carpdev:
# cat hostname.vr2
up
# cat hostname.carp2
inet aa.bb.cc.dd 255.255.255.248 NONE vhid 3 pass somepass
!/sbin/route -qn add -host -mpath default aa.bb.cc.ee
This has worked for a few years. This past week, there was an equipment
change for the upstream router of that link, judging by the change in the
MAC of the gateway IP (aa.bb.cc.ee).
Since that change, that network segment hasn't been able to take any
carp-based traffic. If I shut down one of the firewalls and revert that
link
on the remaining firewall to a non-carp link, then the traffic is
handled normally.
After some splunking (by using a hub on the upstream segment and sniffing
with tcpdump from another host with a static IP on the same segment) what
I can see is that when carp is NOT in use, the upstream router will do
the usual arp request and the firewall will do an arp reply with the MAC
of vr2, and everything is fine.
However, if carp IS in use, I can see the upstream router do the arp
request, followed by the firewall arp reply (with the carp MAC), however
the upstream router seems to ignore the answer and does continuous arp
requests.
I suspect that the fix for this is out of my control (since it seems
to be the upstream that is having problems), but I'm trying to understand
what would cause it. Any thoughts?
I figured that the upstream might be blacklisting VRRP MACs, so I tried
assigning a non-VRRP MAC via lladdr in hostname.carp2, but that had no
effect on the outcome. I also tried using a high-numbered vhid in case
vhid 3 was causing a conflict.
The cable company's troubleshooting procedure includes rebooting the
cable modem on MAC changes (presumably to clear the ARP cache), so that
was done on each configuration change. A clear cache is confirmed in
that the original arp request from the upstream is sent to the ethernet
broadcast address.
Clues welcome.
Devin
