I am trying to configure ifstated on an i386 4.2 Stable pair of openbsd
firewalls but having some issues on how to determine connectivity of a
backup/secondary wan interface.
The carp states seem solid and preempt seems to work great. The only
thing I'm really worried about is an upstream link dying, carp staying
master and traffic getting blackholed. I want ifstated to simply change
the default route to the backup wan interface should connectivity out
the primary get interrupted and then switch back when primary
connectivity comes back. I'm just trying to get it figured out on one
machine first before I move to the second. I'm having trouble figuring
out if there is connectivity on the backup wan interface. I read some
posts that suggested using ping -I so that the pings go out the
appropriate interface, but this seems to not work, if i try to ping
anything other than the backup wan's gateway, it still goes out the
default route.. It is only able to ping the gateway address and with
the (-r) option the pinged host has to be on the directly connected network.
Pinging out the backup wan (fxp1) connection to some random address
(www.hotbot.com) routes packets through the default route and the ping
never succeeds (as it shouldn't going out this route)
# ping -I 192.168.2.162 www.hotbot.com
PING www.hotbot.com (209.202.229.100): 56 data bytes
--- www.hotbot.com ping statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss
tcpdump shows no icmp traffic on fxp1 and the following for fxp0:
# tcpdump -nvtttei fxp0 icmp
tcpdump: listening on fxp0, link-type EN10MB
Feb 16 09:04:01.082329 00:0e:0c:74:6d:61 00:40:f4:76:3d:d3 0800 98:
192.168.2.162 > 209.202.229.100: icmp: echo request (id:f24f seq:0) (ttl
255, id 13485, len 84)
Feb 16 09:04:02.083434 00:0e:0c:74:6d:61 00:40:f4:76:3d:d3 0800 98:
192.168.2.162 > 209.202.229.100: icmp: echo request (id:f24f seq:1) (ttl
255, id 33646, len 84)
.I found this post
http://marc.info/?l=openbsd-misc&m=112672120932213&w=2 where they have
the same situation, but that was from quite a while back and I wasn't
sure if things have changed. If this is in fact the proper behavior
for ping, even with the -I switch, how do people test connectivity of
the backup connections for ifstated? I can understand how the initial
switch could be made just by looking for loss of connectivity on the
primary, but how would you determine when the primary comes back if you
can't direct the pings out that gateway after the default route has
changed? Seems like the backup would have to lose connectivity and force
a switch over. Or, if for instance, the backup didn't have connectivity
there would be no reason for ifstated to change a state subsequently
changing the default route. Routing daemons i.e. BGP are _not_ an
option in this situation.
I have included the routing and interface info below:
Thanks in advance,
Aaron Martinez
# netstat -rnf inet
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu
Interface
default 192.168.3.94 UGS 3 477294 - fxp0
10.0.69/24 link#3 UC 0 0 - fxp2
10.23.183.0/30 link#5 UC 0 0 - rl0
127/8 127.0.0.1 UGRS 0 0 33208 lo0
127.0.0.1 127.0.0.1 UH 2 26119 33208 lo0
192.168.2.160/27 link#2 UC 0 0 - fxp1
192.168.2.161 192.168.2.161 UH 0 0 - carp1
192.168.3.64/27 link#1 UC 1 0 - fxp0
192.168.3.65 192.168.3.65 UH 0 0 - carp0
192.168.3.68 192.168.3.68 UH 0 0 - carp0
192.168.3.68/32 192.168.3.68 U 0 0 - carp0
192.168.3.69 192.168.3.69 UH 0 0 - carp0
192.168.3.69/32 192.168.3.69 U 0 0 - carp0
192.168.3.94 00:40:f4:76:3d:d3 UHLc 1 82 - fxp0
204.181.247/24 link#4 UC 1 0 - fxp3
204.181.247.26 00:08:02:0b:63:59 UHLc 1 321141 - fxp3
204.181.247.136 204.181.247.136 UH 0 0 - carp3
224/4 127.0.0.1 URS 0 0 33208 lo0
# ifconfig
-aA
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33208
groups: lo
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
fxp0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:0e:0c:74:6d:61
groups: egress
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 192.168.3.66 netmask 0xffffffe0 broadcast 192.168.3.95
inet6 fe80::20e:cff:fe74:6d61%fxp0 prefixlen 64 scopeid 0x1
fxp1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:0e:0c:3b:3f:2e
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 192.168.2.162 netmask 0xffffffe0 broadcast 192.168.2.191
inet6 fe80::20e:cff:fe3b:3f2e%fxp1 prefixlen 64 scopeid 0x2
fxp2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:0e:0c:74:6d:a2
media: Ethernet autoselect (none)
status: no carrier
inet 10.0.69.1 netmask 0xffffff00 broadcast 10.0.69.255
inet6 fe80::20e:cff:fe74:6da2%fxp2 prefixlen 64 scopeid 0x3
fxp3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:03:47:b1:2c:c4
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 204.181.247.1 netmask 0xffffff00 broadcast 204.181.247.255
inet6 fe80::203:47ff:feb1:2cc4%fxp3 prefixlen 64 scopeid 0x4
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:50:bf:72:51:c9
media: Ethernet autoselect (none)
status: no carrier
inet 10.23.183.1 netmask 0xfffffffc broadcast 10.23.183.3
inet6 fe80::250:bfff:fe72:51c9%rl0 prefixlen 64 scopeid 0x5
enc0: flags=0<> mtu 1536
pfsync0: flags=41<UP,RUNNING> mtu 1460
pfsync: syncdev: rl0 syncpeer: 224.0.0.240 maxupd: 128
groups: carp pfsync
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33208
groups: pflog
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:5e:00:01:01
carp: MASTER carpdev fxp0 vhid 1 advbase 1 advskew 0
groups: carp
inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0x9
inet 192.168.3.65 netmask 0xffffffe0 broadcast 192.168.3.95
inet 192.168.3.68 netmask 0xffffffff broadcast 192.168.3.68
inet 192.168.3.69 netmask 0xffffffff broadcast 192.168.3.69
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:5e:00:01:02
carp: MASTER carpdev fxp1 vhid 2 advbase 1 advskew 0
groups: carp
inet6 fe80::200:5eff:fe00:102%carp1 prefixlen 64 scopeid 0xa
inet 192.168.2.161 netmask 0xffffffe0 broadcast 192.168.2.191
carp2: flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:5e:00:01:03
carp: INIT carpdev fxp2 vhid 3 advbase 1 advskew 0
groups: carp
inet6 fe80::200:5eff:fe00:103%carp2 prefixlen 64 scopeid 0xb
inet 10.0.69.254 netmask 0xffffff00 broadcast 10.0.69.255
carp3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:5e:00:01:04
carp: MASTER carpdev fxp3 vhid 4 advbase 1 advskew 0
groups: carp
inet6 fe80::200:5eff:fe00:104%carp3 prefixlen 64 scopeid 0xc
inet 204.181.247.136 netmask 0xffffff00 broadcast 204.181.247.255
# cat /var/run/dmesg.boot
OpenBSD 4.2-stable (GENERIC) #0: Fri Dec 28 19:29:04 CST 2007
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Celeron(R) CPU 2.00GHz ("GenuineIntel" 686-class) 2 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID
real mem = 268005376 (255MB)
avail mem = 251502592 (239MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 07/22/03, BIOS32 rev. 0 @ 0xfb160,
SMBIOS rev. 2.3 @ 0xf0800 (38 entries)
bios0: vendor Award Software International, Inc. version "6.00 PG" date
07/22/2003
bios0: Supermicro P4SDA
apm0 at bios0: Power Management spec V1.2
apm0: AC on, battery charge unknown
apm0: flags 70102 dobusy 1 doidle 1
pcibios0 at bios0: rev 2.1 @ 0xf0000/0xdf84
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdec0/192 (10 entries)
pcibios0: PCI Exclusive IRQs: 3 5 7 9 10 11
pcibios0: PCI Interrupt Router at 000:31:0 ("Intel 82371SB ISA" rev 0x00)
pcibios0: PCI bus #2 is the last bus
bios0: ROM list: 0xc0000/0x10000 0xd0000/0x1800 0xd2000/0x1800
0xd4000/0x1800
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel 82845 Host" rev 0x04
ppb0 at pci0 dev 1 function 0 "Intel 82845 AGP" rev 0x04
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "SiS 300/305/630 VGA" rev 0x90: aperture
at 0xe0000000, size 0x400000
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb1 at pci0 dev 30 function 0 "Intel 82801BA AGP" rev 0x05
pci2 at ppb1 bus 2
fxp0 at pci2 dev 1 function 0 "Intel 8255x" rev 0x10, i82551: irq 3,
address 00:0e:0c:74:6d:61
inphy0 at fxp0 phy 1: i82555 10/100 PHY, rev. 4
fxp1 at pci2 dev 2 function 0 "Intel 8255x" rev 0x10, i82551: irq 7,
address 00:0e:0c:3b:3f:2e
inphy1 at fxp1 phy 1: i82555 10/100 PHY, rev. 4
fxp2 at pci2 dev 3 function 0 "Intel 8255x" rev 0x10, i82551: irq 5,
address 00:0e:0c:74:6d:a2
inphy2 at fxp2 phy 1: i82555 10/100 PHY, rev. 4
fxp3 at pci2 dev 4 function 0 "Intel 8255x" rev 0x08, i82559: irq 10,
address 00:03:47:b1:2c:c4
inphy3 at fxp3 phy 1: i82555 10/100 PHY, rev. 4
rl0 at pci2 dev 5 function 0 "Realtek 8139" rev 0x10: irq 11, address
00:50:bf:72:51:c9
rlphy0 at rl0 phy 0: RTL internal PHY
ichpcib0 at pci0 dev 31 function 0 "Intel 82801BA LPC" rev 0x05: 24-bit
timer at 3579545Hz
pciide0 at pci0 dev 31 function 1 "Intel 82801BA IDE" rev 0x05: DMA,
channel 0 wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: <ACCUSYS ACS75130 1.4>
wd0: 16-sector PIO, LBA48, 38166MB, 78165360 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
atapiscsi0 at pciide0 channel 1 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <SAMSUNG, CD-ROM SH-152A, C504> SCSI0
5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2
uhci0 at pci0 dev 31 function 2 "Intel 82801BA USB" rev 0x05: irq 7
ichiic0 at pci0 dev 31 function 3 "Intel 82801BA SMBus" rev 0x05: irq 11
iic0 at ichiic0
uhci1 at pci0 dev 31 function 4 "Intel 82801BA USB" rev 0x05: irq 11
isa0 at ichpcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pmsi0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pmsi0 mux 0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
lm0 at isa0 port 0x290/8: W83627HF
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
usb0 at uhci0: USB revision 1.0
uhub0 at usb0: Intel UHCI root hub, rev 1.00/1.00, addr 1
usb1 at uhci1: USB revision 1.0
uhub1 at usb1: Intel UHCI root hub, rev 1.00/1.00, addr 1
biomask ebc5 netmask efed ttymask ffef
pctr: user-level cycle counter enabled
mtrr: Pentium Pro MTRR support
dkcsum: wd0 matches BIOS drive 0x80
root on wd0a swap on wd0b dump on wd0b
carp: pfsync0 demoted group carp to 129
carp: pfsync0 demoted group pfsync to 1
carp: carp2 demoted group carp to 130
