>Synopsis: Simple GRE tunnel does not work without eg. ipsec - arpresolve:
>route without link local address.
>Category: kernel
>Environment:
System : OpenBSD 4.9
Details : OpenBSD 4.9 (GENERIC) #671: Wed Mar 2 07:09:00 MST 2011
[email protected]:/usr/src/sys/arch/i386/compile/GENERIC
Architecture: OpenBSD.i386
Machine : i386
>Description:
Hello,
I Simple GRE tunnel between two OpenBSD 4.9 hosts does not work when
e.g. IPSEC tunnel is not established. Dmesg logs message "arpresolve:
<IP>: route without link local address".
The below example with configuration attached, precisely describes the problem.
"Test 2" presented below does not work but should.
The test configuration consists of two OpenBSD 4.9 hosts being run in KVM:
- obsd49 (IP 192.168.10.184 set on pcn0 ethernet interface)
- obsd49c (IP 192.168.10.186 set on pcn0 ethernet interface)
**********************************************
*** GRE tunnel configuration on both hosts ***
sysctl net.inet.ip.forwarding=1
sysctl net.inet.gre.allow=1
sysctl net.inet.gre.wccp=1
sysctl net.inet.esp.enable=1
obsd49:~# ifconfig gre0 tunnel 192.168.10.184 192.168.10.186 up
obsd49:~# ifconfig gre0 inet 10.10.184.1 10.10.186.1 netmask 0xffffffff
obsd49c:~# ifconfig gre0 tunnel 192.168.10.186 192.168.10.184 up
obsd49c:~# ifconfig gre0 inet 10.10.186.1 10.10.184.1 netmask 0xffffffff
* Now the GRE tunnel is set. Below the applied configuration --
gre0 interfaces and routing tables of both hosts are presented below:
obsd49:~# ifconfig gre0
gre0: flags=9011<UP,POINTOPOINT,LINK0,MULTICAST> mtu 1476
priority: 0
groups: gre
physical address inet 192.168.10.184 --> 192.168.10.186
inet6 fe80::50fe:d1aa:5495:3ee1%gre0 -> prefixlen 64 scopeid 0x5
inet 10.10.184.1 --> 10.10.186.1 netmask 0xffffffff
obsd49c:~# ifconfig gre0
gre0: flags=9011<UP,POINTOPOINT,LINK0,MULTICAST> mtu 1476
priority: 0
groups: gre
physical address inet 192.168.10.186 --> 192.168.10.184
inet6 fe80::c8bb:7345:e737:f9d5%gre0 -> prefixlen 64 scopeid 0x5
inet 10.10.186.1 --> 10.10.184.1 netmask 0xffffffff
obsd49:~# netstat -rn -f inet
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Prio Iface
default 192.168.10.1 UGS 2 68 - 8 pcn0
10.10.186.1 10.10.184.1 UH 0 0 - 4 gre0
127/8 127.0.0.1 UGRS 0 0 33200 8 lo0
127.0.0.1 127.0.0.1 UH 2 0 33200 4 lo0
192.168.10/24 link#1 UC 3 0 - 4 pcn0
192.168.10.1 00:0e:0c:4a:78:60 UHLc 1 0 - 4 pcn0
192.168.10.184 127.0.0.1 UGHS 0 0 33200 8 lo0
224/4 127.0.0.1 URS 0 0 33200 8 lo0
obsd49c:~# netstat -rn -f inet
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Prio Iface
default 192.168.10.1 UGS 0 61 - 8 pcn0
10.10.184.1 10.10.186.1 UH 0 0 - 4 gre0
127/8 127.0.0.1 UGRS 0 0 33200 8 lo0
127.0.0.1 127.0.0.1 UH 1 0 33200 4 lo0
192.168.10/24 link#1 UC 3 0 - 4 pcn0
192.168.10.1 00:0e:0c:4a:78:60 UHLc 1 0 - 4 pcn0
224/4 127.0.0.1 URS 0 0 33200 8 lo0
* PF rules on both hosts -- no traffic is blocked
# cat /etc/pf.conf
set skip on gre0
set skip on pcn0
pass quick
*************
*** Tests ***
*************
************
** Test 1 **
** ping - connection between hosts obsd49 and obsd49c (OK) **
obsd49:~# ping 192.168.10.186
PING 192.168.10.186 (192.168.10.186): 56 data bytes
64 bytes from 192.168.10.186: icmp_seq=0 ttl=255 time=1.063 ms
--- 192.168.10.186 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.063/1.063/1.063/0.000 ms
obsd49:~# tcpdump -i pcn0 -nve host 192.168.10.186
tcpdump: listening on pcn0, link-type EN10MB
21:36:21.955696 52:54:00:10:91:84 ff:ff:ff:ff:ff:ff 0806 42: arp who-has
192.168.10.186 tell 192.168.10.184
21:36:21.956131 52:54:00:10:91:86 52:54:00:10:91:84 0806 60: arp reply
192.168.10.186 is-at 52:54:00:10:91:86
21:36:21.956234 52:54:00:10:91:84 52:54:00:10:91:86 0800 98: 192.168.10.184 >
192.168.10.186: icmp: echo request (id:ab64 seq:0) (ttl 255, id 23164, len 84)
21:36:21.956507 52:54:00:10:91:86 52:54:00:10:91:84 0800 98: 192.168.10.186 >
192.168.10.184: icmp: echo reply (id:ab64 seq:0) (ttl 255, id 21762, len 84)
**********************
** Test 2 (PROBLEM) **
** ping using GRE encapsulated addresses (does not work) **
obsd49:~# ping 10.10.186.1
PING 10.10.186.1 (10.10.186.1): 56 data bytes
--- 10.10.186.1 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
* The echo request packet is visible on gre0 interface:
obsd49:~# tcpdump -i gre0 -nve
tcpdump: listening on gre0, link-type NULL
21:42:11.516240 10.10.184.1 > 10.10.186.1: icmp: echo request (id:6e69 seq:0)
(ttl 255, id 47704, len 84)
1 packets received by filter
0 packets dropped by kernel
* No traffic is visible on pcn0 traffic:
obsd49:~# tcpdump -i pcn0 -nve host 192.168.10.186
tcpdump: listening on pcn0, link-type EN10MB
52 packets received by filter
0 packets dropped by kernel
* After sending one ping packet dmesg logs the following problem:
obsd49:~# tail -n 1 /var/log/messages
Oct 20 21:42:03 obsd49 /bsd: arpresolve: 192.168.10.186: route without link
local address
***************************************************
*** The workaround is to add VPN connection between hosts obsd49 and obsd49c ***
** Adding VPN connection to the test hosts **
obsd49:~# cat /etc/ipsec.conf
ike esp from { 192.168.10.184 } to { 192.168.10.186 } local 192.168.10.184 peer
192.168.10.186 main auth hmac-sha1 enc aes group modp1024 quick auth hmac-sha1
enc aes group modp1024 psk "<shared_key>"
obsd49c:~# cat /etc/ipsec.conf
ike esp from { 192.168.10.186 } to { 192.168.10.184 } local 192.168.10.186 peer
192.168.10.184 main auth hmac-sha1 enc aes group modp1024 quick auth hmac-sha1
enc aes group modp1024 psk "<shared_key>"
* Running isakmpd daemon and loading ipsec configuration on both hosts:
# isakmpd -K -vv
# ipsecctl -f /etc/ipsec.conf
* Establishing VPN connection results in the following encap routing table:
obsd49:~# netstat -rn -f encap
Routing tables
Encap:
Source Port Destination Port Proto
SA(Address/Proto/Type/Direction)
192.168.10.186/32 0 192.168.10.184/32 0 0
192.168.10.186/esp/use/in
192.168.10.184/32 0 192.168.10.186/32 0 0
192.168.10.186/esp/require/out
obsd49c:~# netstat -rn -f encap
Routing tables
Encap:
Source Port Destination Port Proto
SA(Address/Proto/Type/Direction)
192.168.10.184/32 0 192.168.10.186/32 0 0
192.168.10.184/esp/use/in
192.168.10.186/32 0 192.168.10.184/32 0 0
192.168.10.184/esp/require/out
******************
** Test 3 and 4 **
** GRE tunnel esablished on (encapsulated in) IPSEC VPN works fine (OK) **
obsd49:~# ping 192.168.10.186
PING 192.168.10.186 (192.168.10.186): 56 data bytes
64 bytes from 192.168.10.186: icmp_seq=0 ttl=255 time=1.052 ms
--- 192.168.10.186 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.052/1.052/1.052/0.000 ms
obsd49:~# ping 10.10.186.1
PING 10.10.186.1 (10.10.186.1): 56 data bytes
64 bytes from 10.10.186.1: icmp_seq=0 ttl=255 time=1.033 ms
--- 10.10.186.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.033/1.033/1.033/0.000 ms
* Below there are presented packets sent during the test and
captured from the interfaces gre0, enc0 and pcn0:
obsd49:~# tcpdump -i gre0 -nve
tcpdump: listening on gre0, link-type NULL
21:53:30.263620 10.10.184.1 > 10.10.186.1: icmp: echo request (id:031c seq:0)
(ttl 255, id 14595, len 84)
21:53:30.264491 10.10.186.1 > 10.10.184.1: icmp: echo reply (id:031c seq:0)
(ttl 255, id 19881, len 84)
2 packets received by filter
0 packets dropped by kernel
obsd49:~# tcpdump -i enc0 -n
tcpdump: listening on enc0, link-type ENC
21:53:20.420061 (authentic,confidential): SPI 0x16274e90: 192.168.10.184 >
192.168.10.186: icmp: echo request (encap)
21:53:20.420901 (authentic,confidential): SPI 0x8edc11be: 192.168.10.186 >
192.168.10.184: icmp: echo reply (encap)
21:53:30.263661 (authentic,confidential): SPI 0x6175f14b: 10.10.184.1 >
10.10.186.1: icmp: echo request (gre encap) (encap)
21:53:30.264462 (authentic,confidential): SPI 0xe6800abc: 10.10.186.1 >
10.10.184.1: icmp: echo reply (gre encap) (encap)
4 packets received by filter
0 packets dropped by kernel
obsd49:~# tcpdump -i pcn0 -nve host 192.168.10.186
tcpdump: listening on pcn0, link-type EN10MB
21:53:20.420249 52:54:00:10:91:84 52:54:00:10:91:86 0800 166: esp
192.168.10.184 > 192.168.10.186 spi 0x16274e90 seq 1 len 132 (ttl 64, id 57418,
len 152)
21:53:20.420774 52:54:00:10:91:86 52:54:00:10:91:84 0800 166: esp
192.168.10.186 > 192.168.10.184 spi 0x8edc11be seq 1 len 132 (ttl 64, id 38502,
len 152)
21:53:30.263802 52:54:00:10:91:84 52:54:00:10:91:86 0800 182: esp
192.168.10.184 > 192.168.10.186 spi 0x6175f14b seq 1 len 148 (ttl 64, id 2769,
len 168)
21:53:30.264345 52:54:00:10:91:86 52:54:00:10:91:84 0800 182: esp
192.168.10.186 > 192.168.10.184 spi 0xe6800abc seq 1 len 148 (ttl 64, id 34658,
len 168)
>How-To-Repeat:
The above configuration describes the problem. Just attempt
to set the simplest GRE tunnel between hosts and try to ping
encapsulated addresses. Echo request is visible on the sender gre0,
but no traffic is visible on the ethernet interface. The message
"arpresolve: <IP>: route without link local address" is logged in
/var/log/messages.
>Fix:
I have no fix. The ananlogue configuration using GIF interfaces
(IP-in-IP)
works fine. Only GRE does not work. The workaround is to set the IPSEC
VPN
connection and encapsulate GRE in IPSEC as it was described in the
"Description:" section.
Bests regards,
Pawel Wieleba
