Probably your PF rules. put in ‘pass quick proto icmp’.
> On 28 mar 2015, at 00:59, David Newman <[email protected]> wrote: > > Greetings. In preparation for upgrading two CARP+pfsync boxes to > 5.6/i386, I put together a lab network to test new firewall rules. > > Topology is pretty simple: > > outside box (vic0) <-> (vic1) two carp boxes (vic0) <-> inside box > > with a third interface on each firewall for pfsync traffic. I'm focused > here on the outside box pinging the carp box's outside CARP interface. > > In the lab network everyone can ping everyone else, except for the CARP > interfaces -- these are not pingable. Hosts on either side of the > firewall can ping the underlying interfaces that the CARP interfaces are > bound to. > > Also, 'netstat -f inet -nr' shows that CARP interfaces are bound to lo0. > On the production boxes these systems model, carp interfaces are bound > to the underlying physical interfaces. > > tcpdump on the physical interface of the master firewall says the > outside box ARPs for the CARP interface, and the firewall sends an ARP > response with the CARP interface's IP and MAC addresses. > > Thanks in advance for troubleshooting clues -- this is almost certainly > a misconfiguration but I'm not sure where. > > dn > > Outside box's hostname.vic0: > inet 12.220.174.101 255.255.255.224 12.220.174.127 > > FW1 hostname.vic1: > inet 12.220.174.99 255.255.255.224 12.220.174.127 > > FW1 hostname.carp221: > inet 12.220.174.98 255.255.255.224 12.220.174.127 vhid 221 advskew 1 > pass ***** carpdev vic1 carppeer 12.220.174.100 > > FW1 ifconfig vic1: > vic1: > flags=28b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,NOINET6> > mtu 1500 > lladdr 00:50:56:b2:33:0e > priority: 0 > groups: egress > media: Ethernet autoselect > status: active > inet 12.220.174.99 netmask 0xffffffe0 broadcast 12.220.174.127 > > FW1 ifconfig carp221: > net 12.220.174.98 255.255.255.224 12.220.174.127 vhid 221 advskew 1 pass > w00h00 carpdev vic1 carppeer 12.220.174.100 > # ifconfig carp221 > carp221: flags=28843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,NOINET6> mtu > 1500 > lladdr 00:00:5e:00:01:dd > priority: 0 > carp: MASTER carpdev vic1 vhid 221 advbase 1 advskew 1 carppeer > 12.220.174.100 > groups: carp > status: master > inet 12.220.174.98 netmask 0xffffffe0 broadcast 12.220.174.127 > > FW1 netstat -f inet -nr: > # netstat -f inet -nr > Routing tables > > Internet: > Destination Gateway Flags Refs Use Mtu Prio > Iface > default 12.220.174.97 UGS 0 38 - 8 vic1 > 12.220.174.96/27 link#2 UC 2 0 - 4 vic1 > 12.220.174.98 00:00:5e:00:01:dd HLl 0 0 - 1 > lo0 # <-- NOTE lo0 BINDING > 12.220.174.99 00:50:56:b2:33:0e UHLl 0 0 - 1 lo0 > 12.220.174.100 00:50:56:b2:32:94 UHLc 0 274 - 4 vic1 > 12.220.174.101 00:50:56:b2:5e:b5 UHLc 0 5 - 4 vic1 > 127/8 127.0.0.1 UGRS 0 0 32768 8 lo0 > 127.0.0.1 127.0.0.1 UH 1 4 32768 4 lo0 > > > FW2 hostname.vic1: > inet 12.220.174.100 255.255.255.224 12.220.174.127 > > FW2 hostname.carp221: > inet 12.220.174.98 255.255.255.224 12.220.174.127 vhid 221 advskew 128 > pass ***** carpdev vic1 carppeer 12.220.174.99 > > FW2 ifconfig carp221: > carp221: flags=28843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,NOINET6> mtu > 1500 > lladdr 00:00:5e:00:01:dd > priority: 0 > carp: BACKUP carpdev vic1 vhid 221 advbase 1 advskew 128 > carppeer 12.220.174.99 > groups: carp > status: backup > inet 12.220.174.98 netmask 0xffffffe0 broadcast 12.220.174.127 > > pf.conf on both boxes: > > # interfaces > pfsync0_if = "vic2" > carp_dev = "{ vic0, vic1 }" > > set skip on lo > > ################## > # Packet filtering > ################## > > block return # block stateless traffic > #pass # establish keep-state > > # By default, do not permit remote connections to X11 > block return in on ! lo0 proto tcp to port 6000:6010 > > # icmp handling -- FIX THIS to specify ICMP types > pass log inet proto icmp all > > # carp and pfsync > pass on { $pfsync0_if } proto pfsync > pass on $carp_dev proto carp > > FW1 dmesg: > > OpenBSD 5.6 (GENERIC.MP) #299: Fri Aug 8 00:10:33 MDT 2014 > [email protected]:/usr/src/sys/arch/i386/compile/GENERIC.MP > cpu0: Intel(R) Xeon(R) CPU E5649 @ 2.53GHz ("GenuineIntel" 686-class) > 2.54 GHz > cpu0: > FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,MMX,FXSR,SSE,SSE2,SS,NXE,LONG,SSE3,PCLMUL,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AES,LAHF,PERF,ITSC > real mem = 536309760 (511MB) > avail mem = 515063808 (491MB) > mpath0 at root > scsibus0 at mpath0: 256 targets > mainbus0 at root > bios0 at mainbus0: AT/286+ BIOS, date 04/14/14, BIOS32 rev. 0 @ 0xfd780, > SMBIOS rev. 2.4 @ 0xe0010 (364 entries) > bios0: vendor Phoenix Technologies LTD version "6.00" date 04/14/2014 > bios0: VMware, Inc. VMware Virtual Platform > acpi0 at bios0: rev 2 > acpi0: sleep states S0 S1 S4 S5 > acpi0: tables DSDT FACP BOOT APIC MCFG SRAT HPET WAET > acpi0: wakeup devices PCI0(S3) USB_(S1) P2P0(S3) S1F0(S3) S2F0(S3) > S3F0(S3) S4F0(S3) S5F0(S3) S6F0(S3) S7F0(S3) S8F0(S3) S9F0(S3) S10F(S3) > S11F(S3) S12F(S3) S13F(S3) [...] > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > cpu0 at mainbus0: apid 0 (boot processor) > mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges > cpu0: apic clock running at 65MHz > cpu1 at mainbus0: apid 1 (application processor) > cpu1: Intel(R) Xeon(R) CPU E5649 @ 2.53GHz ("GenuineIntel" 686-class) > 2.54 GHz > cpu1: > FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,MMX,FXSR,SSE,SSE2,SS,NXE,LONG,SSE3,PCLMUL,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AES,LAHF,PERF,ITSC > ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 24 pins > acpimcfg0 at acpi0 addr 0xf0000000, bus 0-127 > acpihpet0 at acpi0: 14318179 Hz > acpiprt0 at acpi0: bus 0 (PCI0) > acpicpu0 at acpi0 > acpicpu1 at acpi0 > acpibat0 at acpi0: BAT1 not present > acpibat1 at acpi0: BAT2 not present > acpiac0 at acpi0: AC unit online > acpibtn0 at acpi0: SLPB > acpibtn1 at acpi0: LID_ > bios0: ROM list: 0xc0000/0x8000 0xc8000/0x1000 0xc9000/0x1000 > 0xca000/0x1000 0xcb000/0x1000 0xcc000/0x1e00! 0xdc000/0x4000! > 0xe0000/0x8000! > vmt0 at mainbus0 > pci0 at mainbus0 bus 0: configuration mode 1 (bios) > pchb0 at pci0 dev 0 function 0 "Intel 82443BX AGP" rev 0x01 > ppb0 at pci0 dev 1 function 0 "Intel 82443BX AGP" rev 0x01 > pci1 at ppb0 bus 1 > piixpcib0 at pci0 dev 7 function 0 "Intel 82371AB PIIX4 ISA" rev 0x08 > pciide0 at pci0 dev 7 function 1 "Intel 82371AB IDE" rev 0x01: DMA, > channel 0 configured to compatibility, channel 1 configured to compatibility > pciide0: channel 0 disabled (no drives) > atapiscsi0 at pciide0 channel 1 drive 0 > scsibus1 at atapiscsi0: 2 targets > cd0 at scsibus1 targ 0 lun 0: <NECVMWar, VMware IDE CDR10, 1.00> ATAPI > 5/cdrom removable > cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2 > piixpm0 at pci0 dev 7 function 3 "Intel 82371AB Power" rev 0x08: SMBus > disabled > "VMware VMCI" rev 0x10 at pci0 dev 7 function 7 not configured > vga1 at pci0 dev 15 function 0 "VMware SVGA II" rev 0x00 > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) > ppb1 at pci0 dev 17 function 0 "VMware PCI" rev 0x02 > pci2 at ppb1 bus 2 > vic0 at pci2 dev 0 function 0 "AMD 79c970 PCnet-PCI" rev 0x10: apic 2 > int 18, address 00:50:56:b2:06:d6 > vic1 at pci2 dev 2 function 0 "AMD 79c970 PCnet-PCI" rev 0x10: apic 2 > int 16, address 00:50:56:b2:33:0e > vic2 at pci2 dev 3 function 0 "AMD 79c970 PCnet-PCI" rev 0x10: apic 2 > int 17, address 00:50:56:b2:4e:98 > vic3 at pci2 dev 4 function 0 "AMD 79c970 PCnet-PCI" rev 0x10: apic 2 > int 18, address 00:50:56:b2:40:1a > ppb2 at pci0 dev 21 function 0 "VMware PCIE" rev 0x01 > pci3 at ppb2 bus 3 > mpi0 at pci3 dev 0 function 0 "Symbios Logic SAS1068" rev 0x01: apic 2 > int 18 > mpi0: SAS3444, firmware 1.3.41.32 > scsibus2 at mpi0: 256 targets, initiator 16 > sd0 at scsibus2 targ 0 lun 0: <VMware, Virtual disk, 1.0> SCSI2 0/direct > fixed > sd0: 8192MB, 512 bytes/sector, 16777216 sectors > ppb3 at pci0 dev 21 function 1 "VMware PCIE" rev 0x01 > pci4 at ppb3 bus 4 > ppb4 at pci0 dev 21 function 2 "VMware PCIE" rev 0x01 > pci5 at ppb4 bus 5 > ppb5 at pci0 dev 21 function 3 "VMware PCIE" rev 0x01 > pci6 at ppb5 bus 6 > ppb6 at pci0 dev 21 function 4 "VMware PCIE" rev 0x01 > pci7 at ppb6 bus 7 > ppb7 at pci0 dev 21 function 5 "VMware PCIE" rev 0x01 > pci8 at ppb7 bus 8 > ppb8 at pci0 dev 21 function 6 "VMware PCIE" rev 0x01 > pci9 at ppb8 bus 9 > ppb9 at pci0 dev 21 function 7 "VMware PCIE" rev 0x01 > pci10 at ppb9 bus 10 > ppb10 at pci0 dev 22 function 0 "VMware PCIE" rev 0x01 > pci11 at ppb10 bus 11 > ppb11 at pci0 dev 22 function 1 "VMware PCIE" rev 0x01 > pci12 at ppb11 bus 12 > ppb12 at pci0 dev 22 function 2 "VMware PCIE" rev 0x01 > pci13 at ppb12 bus 13 > ppb13 at pci0 dev 22 function 3 "VMware PCIE" rev 0x01 > pci14 at ppb13 bus 14 > ppb14 at pci0 dev 22 function 4 "VMware PCIE" rev 0x01 > pci15 at ppb14 bus 15 > ppb15 at pci0 dev 22 function 5 "VMware PCIE" rev 0x01 > pci16 at ppb15 bus 16 > ppb16 at pci0 dev 22 function 6 "VMware PCIE" rev 0x01 > pci17 at ppb16 bus 17 > ppb17 at pci0 dev 22 function 7 "VMware PCIE" rev 0x01 > pci18 at ppb17 bus 18 > ppb18 at pci0 dev 23 function 0 "VMware PCIE" rev 0x01 > pci19 at ppb18 bus 19 > ppb19 at pci0 dev 23 function 1 "VMware PCIE" rev 0x01 > pci20 at ppb19 bus 20 > ppb20 at pci0 dev 23 function 2 "VMware PCIE" rev 0x01 > pci21 at ppb20 bus 21 > ppb21 at pci0 dev 23 function 3 "VMware PCIE" rev 0x01 > pci22 at ppb21 bus 22 > ppb22 at pci0 dev 23 function 4 "VMware PCIE" rev 0x01 > pci23 at ppb22 bus 23 > ppb23 at pci0 dev 23 function 5 "VMware PCIE" rev 0x01 > pci24 at ppb23 bus 24 > ppb24 at pci0 dev 23 function 6 "VMware PCIE" rev 0x01 > pci25 at ppb24 bus 25 > ppb25 at pci0 dev 23 function 7 "VMware PCIE" rev 0x01 > pci26 at ppb25 bus 26 > ppb26 at pci0 dev 24 function 0 "VMware PCIE" rev 0x01 > pci27 at ppb26 bus 27 > ppb27 at pci0 dev 24 function 1 "VMware PCIE" rev 0x01 > pci28 at ppb27 bus 28 > ppb28 at pci0 dev 24 function 2 "VMware PCIE" rev 0x01 > pci29 at ppb28 bus 29 > ppb29 at pci0 dev 24 function 3 "VMware PCIE" rev 0x01 > pci30 at ppb29 bus 30 > ppb30 at pci0 dev 24 function 4 "VMware PCIE" rev 0x01 > pci31 at ppb30 bus 31 > ppb31 at pci0 dev 24 function 5 "VMware PCIE" rev 0x01 > pci32 at ppb31 bus 32 > ppb32 at pci0 dev 24 function 6 "VMware PCIE" rev 0x01 > pci33 at ppb32 bus 33 > ppb33 at pci0 dev 24 function 7 "VMware PCIE" rev 0x01 > pci34 at ppb33 bus 34 > isa0 at piixpcib0 > isadma0 at isa0 > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo > pckbc0 at isa0 port 0x60/5 > pckbd0 at pckbc0 (kbd slot) > pckbc0: using irq 1 for kbd slot > wskbd0 at pckbd0: console keyboard, using wsdisplay0 > pms0 at pckbc0 (aux slot) > pckbc0: using irq 12 for aux slot > wsmouse0 at pms0 mux 0 > pcppi0 at isa0 port 0x61 > spkr0 at pcppi0 > lpt0 at isa0 port 0x378/4 irq 7 > npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16 > fdc0 at isa0 port 0x3f0/6 irq 6 drq 2 > fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec > vscsi0 at root > scsibus3 at vscsi0: 256 targets > softraid0 at root > scsibus4 at softraid0: 256 targets > root on sd0a (1a644d282d120fac.a) swap on sd0b dump on sd0b > carp211: state transition: BACKUP -> MASTER
