>Synopsis: Wherever an ICMP echo request is sent to the other end of the
>4in6 over Ipsec-over-IPv6, once every twenty requests is given a proper reply.
>Category: system user library i386
>Environment:
System : OpenBSD 5.6
Details : OpenBSD 5.6-stable (GENERIC) #0: Tue Dec 9 03:10:49 UTC
2014
[email protected]:/usr/src/sys/arch/i386/compile/GENERIC
Architecture: OpenBSD.i386
Machine : i386
>Description:
Suppose I have two OpenBSD hosts, A and B, running 5.6-STABLE, and we
first build an IPsec tunnel (NOT transport) between A and B. and then we build
a 4in6 tunnel and route it inside the established IPsec tunnel. We then set up
static routes for the 4in6 tunnel and put the line "set skip on gif1" in
pf.conf. Now, whenever A pings B or B pings A, tcpdump shows that the other end
receives the reply but the ping program does not produce any output of replies.
>How-To-Repeat:
-----pf.conf on A---BEGIN-----
skip_if = "{ lo, gif1 }"
set skip on $skip_if
block return
pass
addr_ip4_wan = "<REDACTED>"
net_rfc1918 = "{ 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }"
block in on egress proto tcp to port 6000:6010
block in on egress inet proto { tcp, udp } to port 1194
block in on egress inet6 from 2001:470:1f05:4e2::/64
block in on egress inet6 to 2001:470:d:dd2::/64
match out on egress inet from $net_rfc1918 nat-to egress
pass in on egress inet6 to 64:ff9b::/96 af-to inet from $addr_ip4_wan
pass inet proto icmp all icmp-type { echoreq, unreach } keep state
pass inet6 proto ipv6-icmp all icmp6-type { echoreq, unreach } keep state
ssh_if = "{ egress, vlan0 }"
block in on egress inet proto tcp to port ssh
pass in on $ssh_if proto tcp to port ssh
ipsec_if = "{ egress, gif }"
match on $ipsec_if proto udp from port { isakmp, ipsec-nat-t } \
scrub ( no-df random-id )
match on $ipsec_if proto udp to port { isakmp, ipsec-nat-t } \
scrub ( no-df random-id )
match on $ipsec_if proto esp scrub ( no-df random-id )
pass on enc keep state ( if-bound )
pass in on $ipsec_if proto udp to port { isakmp, ipsec-nat-t } keep state
pass in on $ipsec_if proto esp
block in on egress proto udp to port l2tp
block in on egress proto tcp to port pptp
match on pppx inet proto tcp scrub ( reassemble tcp max-mss )
match on pppx inet scrub ( no-df random-id )
block in on egress proto { tcp, udp } to port smtp
block in on egress proto { tcp, udp } to port submission
-----pf.conf on A---END-----
-----ipsec.conf on A---BEGIN-----
ike active esp tunnel from fc00::/16 to fd00::/16 \
local 2001:470:c:dd2::2 peer 2001:470:1f08:ebe::2 \
main auth hmac-sha2-256 enc aes-128 group modp2048 lifetime 3600 \
quick auth hmac-sha1 enc aes-128 group modp2048 lifetime 1200 \
psk test
-----ipsec.conf on A---END-----
-----vlan1 and gif1 on A---BEGIN-----
vlan1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 72:36:4e:fa:d9:63
priority: 0
vlan: 2 parent interface: em0
groups: vlan
status: active
inet6 fe80::7036:4eff:fefa:d963%vlan1 prefixlen 64 scopeid 0x10
inet6 fc00::1 prefixlen 16
gif1: flags=28051<UP,POINTOPOINT,RUNNING,MULTICAST,NOINET6> mtu 1280
priority: 0
groups: gif
tunnel: inet6 fc00::1 -> fd00::1
inet 172.18.0.2 --> 0.0.0.0 netmask 0xfffffffc
-----vlan1 and gif1 on A---END-----
-----pf.conf on B---BEGIN-----
set skip on lo
block return
pass
block return in on ! lo0 proto tcp to port 6000:6010
set skip on gif1
addr_ip4_wan = "<REDACTED>"
net_rfc1918 = "{ 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 }"
block in on egress inet proto { tcp, udp } to port 1194
pass inet proto icmp all icmp-type { echoreq, unreach } keep state
pass inet6 proto ipv6-icmp all icmp6-type { echoreq, unreach } keep state
match out on egress inet from $net_rfc1918 nat-to egress
pass in on egress inet6 to 64:ff9b::/96 af-to inet from $addr_ip4_wan
ssh_if = "{ egress, vlan }"
pass in on $ssh_if proto tcp to port ssh
block in on egress inet proto tcp to port ssh
match on egress proto udp from port { isakmp, ipsec-nat-t } \
scrub ( no-df random-id )
match on egress proto udp to port { isakmp, ipsec-nat-t } \
scrub ( no-df random-id )
match on egress proto esp scrub ( no-df random-id )
pass on enc keep state ( if-bound )
pass in on egress proto udp to port { isakmp, ipsec-nat-t } keep state
pass in on egress proto esp
block in on egress proto udp to port l2tp
block in on egress proto tcp to port pptp
match on pppx proto tcp scrub ( reassemble tcp )
match on pppx inet scrub ( no-df random-id )
-----pf.conf on B---END-----
-----ipsec.conf on B---BEGIN-----
ike active esp tunnel from fd00::/16 to fc00::/16 \
local 2001:470:1f08:ebe::2 peer 2001:470:c:dd2::2\
main auth hmac-sha2-256 enc aes-128 group modp2048 lifetime 3600 \
quick auth hmac-sha1 enc aes-128 group modp2048 lifetime 1200 \
psk test
-----ipsec.conf on B---END-----
-----vlan1 and gif1 on B---BEGIN-----
vlan1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 9e:9e:2b:3b:ba:6d
priority: 0
vlan: 2 parent interface: em0
groups: vlan
inet6 fe80::9c9e:2bff:fe3b:ba6d%vlan1 prefixlen 64 scopeid 0x10
inet6 fd00::1 prefixlen 16
gif1: flags=28051<UP,POINTOPOINT,RUNNING,MULTICAST,NOINET6> mtu 1280
priority: 0
groups: gif
tunnel: inet6 fd00::1 -> fc00::1
inet 172.18.0.1 --> 0.0.0.0 netmask 0xfffffffc
-----vlan1 and gif1 on B---END-----
Routing is done on both A and B with "route add 172.18.0.0/30 172.18.0.1".
tcpdump is behaving rather weird here. Since the payload inside the 4in6 is
actually an IPv4 packet. The next header field in the encapsulating IPv6 packet
should bear a value of 4. However, "tcpdump -i enc0 'proto 4'" on neither side
show any sign of packets. The packet capturing mentioned above is done with
"tcpdump i enc0 'proto 41'" instead.
>Fix:
Based on the erratic behavior of tcpdump, I figure this is a problem on
the side of pf. And since 4in6 WITHOUT IPsec gives NO problems, this can
further be attributed to the IPsec implementation. ONLY by adding "set skip on
{ gif1, enc0 }" does this problem go away. Also, this happens on a
Linux/StrongSWAN-to-OpenBSD 4in6 encapsulated in IPsec-over-IPv6 payloads. See
StrongSWAN issue 802 for further details. If this is further reproducible on a
Linux-to-Linux scenario, we can then conclude that something's wrong with the
current standard of processing IPsec packets for cross-IP transition across
implementations.
dmesg:
OpenBSD 5.6-stable (GENERIC) #0: Tue Dec 9 03:10:49 UTC 2014
[email protected]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: QEMU Virtual CPU version 0.15.0 ("GenuineIntel" 686-class) 2.14 GHz
cpu0:
FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,LONG,SSE3,CX16,POPCNT,LAHF,PERF
real mem = 133709824 (127MB)
avail mem = 119103488 (113MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 06/23/99, BIOS32 rev. 0 @ 0xff046, SMBIOS
rev. 2.4 @ 0x7fffef0 (10 entries)
bios0: vendor Bochs version "Bochs" date 01/01/2007
bios0: Bochs Bochs
acpi0 at bios0: rev 0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP SSDT APIC HPET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 11, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 1
acpihpet0 at acpi0: 100000000 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
bios0: ROM list: 0xc0000/0x8c00 0xc9000/0x600 0xc9800/0x2400
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0
wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
atapiscsi0 at pciide0 channel 1 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <QEMU, QEMU DVD-ROM, 0.15> ATAPI 5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, DMA mode 2
uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 1 int 11
piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 1 int 10
iic0 at piixpm0
iic0: addr 0x1c 0f=00 words 00=3b90 01=3b90 02=3b90 03=3b90 04=3b90 05=3b90
06=3b90 07=3b90
iic0: addr 0x1d 0f=00 words 00=3b90 01=3b90 02=3b90 03=3b90 04=3b90 05=3b90
06=3b90 07=3b90
iic0: addr 0x4c 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 words
00=3b90 01=3b90 02=3b90 03=3b90 04=3b90 05=3b90 06=3b90 07=3b90
iic0: addr 0x4e 00=00 01=00 02=00 03=00 04=00 05=00 06=00 07=00 08=00 words
00=3b90 01=3b90 02=3b90 03=3b90 04=3b90 05=3b90 06=3b90 07=3b90
vga1 at pci0 dev 2 function 0 "Cirrus Logic CL-GD5446" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
siop0 at pci0 dev 3 function 0 "Symbios Logic 53c895A" rev 0x00: apic 1 int 11,
using 8K of on-board RAM
scsibus2 at siop0: 16 targets, initiator 7
siop0: bad offset in siop_sdp (17)
sd0 at scsibus2 targ 0 lun 0: <QEMU, QEMU HARDDISK, 0.15> SCSI3 0/direct fixed
sd0: 3072MB, 512 bytes/sector, 6291456 sectors
em0 at pci0 dev 4 function 0 "Intel 82540EM" rev 0x03: apic 1 int 11, address
9e:9e:2b:3b:ba:6d
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: density unknown
fd1 at fdc0 drive 1: density unknown
usb at uhci0 not configured
nvram: invalid checksum
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (1bc2bad5054a3481.a) swap on sd0b dump on sd0b
clock: unknown CMOS layout
usbdevs:
usbdevs: no USB controllers found
