Hello,
I configured the route source IP as you recommended (I hope the right
way):
# ip -4 r s table all
8.8.8.8 via 10.198.54.1 dev eth0
10.198.54.0/24 dev eth0 proto kernel scope link src 10.198.54.208 metric
100
95.115.254.165 via 10.198.54.1 dev eth0
195.210.28.0/24 dev vti1 scope link src 95.115.254.161
broadcast 10.198.54.0 dev eth0 table local proto kernel scope link src
10.198.54.208
local 10.198.54.208 dev eth0 table local proto kernel scope host src
10.198.54.208
broadcast 10.198.54.255 dev eth0 table local proto kernel scope link src
10.198.54.208
local 95.115.254.161 dev vti1 table local proto kernel scope host src
95.115.254.161
broadcast 127.0.0.0 dev lo table local proto kernel scope link src
127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src
127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src
127.0.0.1
but still it doesn't work. I think this is not a routing issue (but
maybe I am wrong). Now I tried the reverse flow.
I started to ping 95.115.254.161 (my vpn endpoint IP) from 195.210.28.63
(I have configured static route to this net on my vpn endpoint). I can
see that icmp echo request arrive to vpn server at 95.115.254.165 (I
have access to this server), then is forwarded via ipsec tunnel to my
endpoint. I see this on eth0 on my vpn endpoint:
# tcpdump -i eth0 -n udp
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144
bytes
23:06:43.155626 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2b8),
length 136
23:06:44.155487 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2b9),
length 136
23:06:45.155547 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2ba),
length 136
23:06:46.155465 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bb),
length 136
23:06:47.155299 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bc),
length 136
23:06:48.155729 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bd),
length 136
For me, it seems that these are encrypted icmp echo requests coming from
195.210.28.63 (where ping command is running).
But those packets are not forwarded to vti1. Instead they are counted as
errors:
# ip -s tunnel show
vti1: ip/ip remote 95.115.254.165 local 10.198.54.208 ttl inherit
key 38
RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
0 0 889 0 0 0
TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
0 0 0 0 0 0
As you can see, there is 0 packets transmitted (I am only receiving
ping) and lot of received errors (those are ICMP requests I suppose).
What do you think about this? My amateur opinion is, that there is issue
with passing decrypted packets to the vti1 device and not the routing
issue, because packets are not even passed to vti1 device.
I achieve the same result with TCP instead of ICMP (trying to telnet to
95.115.254.161, not pinging 95.115.254.161).
Thank you for the reply
BR,
Miroslav
On 2017-11-22 21:55, Noel Kuntze wrote:
Hi,
Responses are in line.
On 22.11.2017 20:22, Miroslav Hostinsky wrote:
Hello,
I am not sure if I understand you, but source IP of the outgoing
packets on vti1 are right for me (it was always 95.115.254.161).
# ip a s vti1
5: vti1@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue
state UNKNOWN qlen 1
link/ipip 10.198.54.208 peer 95.115.254.165
inet 95.115.254.161/32 scope global vti1
valid_lft forever preferred_lft forever
No. That is just an IP that is bound to the interface. It has nothing
to do with the source IP of any packets. The kernel does not choose
the IP based on an IP that is bound to an interface.
It does it based on the routing table and the routing decision.
current ipv4 routing table is following (I simplified my
configuration, removed second physical interface, removed default
route):
# ip -4 r s table all
8.8.8.8 via 10.198.54.1 dev eth0
10.198.54.0/24 dev eth0 proto kernel scope link src 10.198.54.208
metric 100
95.115.254.165 via 10.198.54.1 dev eth0
195.210.28.0/24 dev vti1 scope link
broadcast 10.198.54.0 dev eth0 table local proto kernel scope link src
10.198.54.208
local 10.198.54.208 dev eth0 table local proto kernel scope host src
10.198.54.208
broadcast 10.198.54.255 dev eth0 table local proto kernel scope link
src 10.198.54.208
local 95.115.254.161 dev vti1 table local proto kernel scope host src
95.115.254.161
broadcast 127.0.0.0 dev lo table local proto kernel scope link src
127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src
127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src
127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link
src 127.0.0.1
I wonder what the kernel now does, because it has no way to infer the
recommended source IP for traffic to 0.0.0.0/0.
There is the static route 195.210.28.0/24 via vti1. If I do “ping
195.210.28.63” I can see using tcpdump on vti1 following output:
# tcpdump -i vti1 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on vti1, link-type RAW (Raw IP), capture size 262144 bytes
20:08:07.776307 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
id 1098, seq 480, length 64
20:08:08.776218 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
id 1098, seq 481, length 64
20:08:09.776228 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
id 1098, seq 482, length 64
20:08:10.776196 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
id 1098, seq 483, length 64
I guess it falls back to the primary IP of the interface that the
traffic is sent out of.
I can see there, that source IP is 95.115.254.161, so right IP
address. This flow is running over IKEv2 tunnel.
The IKE protocol in use is irrelevant.
If I do tcpdump on underlying eth0 device:
# tcpdump -i eth0 -n udp
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144
bytes
20:09:31.778294 IP 10.198.54.208.ipsec-nat-t >
95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x234),
length 136
20:09:31.792601 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24c),
length 136
20:09:32.131698 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24d),
length 104
20:09:32.778291 IP 10.198.54.208.ipsec-nat-t >
95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x235),
length 136
20:09:32.792787 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24e),
length 136
20:09:33.778217 IP 10.198.54.208.ipsec-nat-t >
95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x236),
length 136
20:09:33.792943 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24f),
length 136
What eth0 does is irrelevant, too.
I guess that 10.198.54.208->95.115.254.165 is ICMP echo request (which
I see on vti1 when doing tcpdump) and 95.115.254.165->10.198.54.208 is
ICMP echo reply (this I only guess, but there is no other traffic than
this ICMP flow).
It can also be ICMP errors. What traffic passes the other peer?
For reference I am pasting statusall outout:
# strongswan statusall
Status of IKE charon daemon (strongSwan 5.5.3, Linux
3.10.0-693.5.2.el7.x86_64, x86_64):
uptime: 18 minutes, since Nov 22 19:59:23 2017
malloc: sbrk 2859008, mmap 0, used 562720, free 2296288
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0,
scheduled: 2
loaded plugins: charon aes des rc2 sha2 sha1 md4 md5 random nonce
x509 revocation constraints acert pubkey pkcs1 pkcs8 pkcs12 pgp dnskey
sshkey pem openssl gcrypt fips-prf gmp curve25519 xcbc cmac hmac ctr
ccm gcm curl attr kernel-netlink resolve socket-default farp stroke
vici updown eap-identity eap-md5 eap-gtc eap-mschapv2 eap-tls eap-ttls
eap-peap xauth-generic xauth-eap xauth-pam xauth-noauth dhcp unity
Listening IP addresses:
10.198.54.208
95.115.254.161
Connections:
alconet: %any...ipsec.alconet.sk <http://ipsec.alconet.sk>
IKEv2, dpddelay=30s
alconet: local: [bmanovic] uses EAP authentication with EAP
identity '%any'
alconet: remote: [ipsec.alconet.sk <http://ipsec.alconet.sk>]
uses public key authentication
alconet: child: dynamic === 0.0.0.0/0 ::/0 TUNNEL,
dpdaction=restart
Security Associations (1 up, 0 connecting):
alconet[1]: ESTABLISHED 18 minutes ago,
10.198.54.208[bmanovic]...95.115.254.165[ipsec.alconet.sk
<http://ipsec.alconet.sk>]
alconet[1]: IKEv2 SPIs: aac4fe8159ef3f03_i* d183908ac6976b91_r,
rekeying disabled
alconet[1]: IKE proposal:
AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/CURVE_25519
alconet{1}: INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs:
c69deaf7_i c39fd86f_o, IPCOMP CPIs: 9858_i 7f79_o
alconet{1}: AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 91560
bytes_o (1090 pkts, 1s ago), rekeying disabled
alconet{1}: 95.115.254.161/32 === 0.0.0.0/0
As I wrote above, I simplified my configuration, but the result is the
same. It doesn't work.
Do you still think, that this is that source IP issue?
Not anymore, but I'm sure some software gets the source IP wrong,
because the hints are (still) wrong.
Ping might just work, because it does special things.
Kind regards
Noel
Thank you for your help,
BR,
Miroslav
On 22 Nov 2017, at 19:22, Noel Kuntze
<[email protected]
<mailto:[email protected]>> wrote:
Hi,
The route over the VTI device does not indicate a preferred source
IP, so the kernel takes the one from the default route, which is in
term deferred from the route to 10.198.54.0/24,
which is 10.198.54.208. I'm pretty sure running tcpdump on the VTI
device confirms that. So the source IP is wrong, as I wrote before.
You need to set the virtual IP as source IP in your updown script.
Kind regards
Noel
On 22.11.2017 18:49, Miroslav Hostinsky wrote:
Hello Noel,
I do not know what you exactly mean, but the source IP send over VTI
interface is the same as configured on VTI interface (95.115.254.161
in this case). As I wrote, I can see that ICMP echo request & reply
are delivered to this IPSEC endpoint machine (but I only suppose,
because packets are encrypted when sniffing using tcpdump, but there
is no other ICMP traffic). It seems that encrypted echo-reply is
delivered to the machine, but "kernel/ipsec stack" is not able to
properly "route" to the VTI device.
Actually my IPSEC/*swan knowledge is not very good so sorry if my
answer are dumb.
I am attaching logs/configs as you requested.
Thank you,
BR,
Miroslav
On 2017-11-22 17:41, Noel Kuntze wrote:
Hello Miroslav,
I suspect that the policy lookup for the received packets fail.
Check
what the source of the packets is that you send over the vti
device.
Anyway, please provide the full list of information from the
HelpRequests[1] page.
[1]
https://wiki.strongswan.org/projects/strongswan/wiki/HelpRequests
Kind regards
Noel
On 22.11.2017 16:47, Miroslav Hostinsky wrote:
Hello,
I have an issue configuring StrongSwan with VTI interface as
roadwarrior. This is my configuration:
ipsec.conf:
config setup
conn %default
keyexchange=ikev2
ikelifetime=60m
keylife=20m
rekeymargin=3m
rekey=no
dpdaction=restart
dpddelay=30s
compress=yes
auto=start
conn acnnet
leftupdown=/usr/local/sbin/ipsec-notify.sh
left=%defaultroute
leftauth=eap
leftsourceip=%config4,%config6
rightauth=pubkey
rightsubnet=0.0.0.0/0,::/0
eap_identity=%identity
leftid=bman
right=mailer.domena.sk <http://mailer.domena.sk>
[email protected] <mailto:[email protected]>
mark=28
VTI interface is configured using lefupdown script (real commands
executed):
ip tunnel add vti1 local 85.105.254.225 remote 185.210.28.63 mode
vti key 28 ikey 28
ip link set vti1 up
ip addr add 192.168.228.10 dev vti1
ip route add 74.99.179.0/24 dev vti1
sysctl -w net.ipv4.conf.vti1.disable_policy=1
It seems that outgoing connection via vti1 interface is working
(outgoing ICMP echo request to subnet 74.99.179.0/24 ). But I am
unable to receive ICMP echo reply. Using tcpdump I can clearly
see, that IPSEC encrypted ICMP echo reply is returning via
physical interface, but not via vti1.
I found, that, TX bytes is correctly counted via vti1, but RX
shows errors (it seems that each ICMP echo reply packet is counted
as +1 error):
# ip -s tunnel show
vti1: ip/ip remote 185.210.28.63 local 85.105.254.225 ttl
inherit key 28
RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
0 0 805 0 0 0
TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
401 68170 0 0 0 0
ip_vti0: ip/ip remote any local any ttl inherit nopmtudisc key
0
RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
0 0 0 0 0 0
TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
0 0 0 0 0 0
It seems that, RX Errors on vti1 are currently missing ICMP echo
reply packets. But is counted as RX errors, not RX received
packets.
Do you have any idea what's wrong?
I am using Centos 7.4 with strongswan-5.5.3-1.el7.x86_64 from
EPEL. A tried with same result on Archlinux (kernel 4.9 and
strongswan 5.6.0).
Route installation is disabled in charon.conf.
Normal connection using Virtual IP is working great.
Thank you very much for any help.
BR,
Miroslav
--
Miroslav Hostinsky
