Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Neale, Thank you so much for patiently explaining. I initially added next-hop like this - ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218 But it still got dropped with ip4-arp: ARP requests sent. Then I read your message again, and realised that the ARP response really doesn't matter. So I set a static ARP entry like so - set ip neighbor lan0.218 172.30.0.6 c8:5b:76:d9:7f:9c That thankfully, changed the nodes visited. Unfortunately, it still fails with an error like below - ipsec6-no-such-tunnel. I understand that it must be some mistake in setting up the tunnel, but once again, I am not sure how to proceed. The packet trace is at the end. Here is the way I set-up the tunnel: ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 integ-key integ-alg sha1-96 tunnel-src tunnel-dst ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 integ-key integ-alg sha1-96 tunnel-src tunnel-dst ipsec spd add 1 set interface ipsec spd lan0.218 1 ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 ipsec policy add spd 1 priority 10 outbound action protect sa 10 local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - 10.6.255.255 ipsec policy add spd 1 priority 10 inbound action protect sa 20 local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - 10.6.255.255 ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218 set ip neighbor lan0.218 172.30.0.6 24:6e:96:9c:e5:df *Packet trace:* 00:01:35:388393: dpdk-input lan0 rx queue 0 buffer 0x1389c3: current data 0, length 102, buffer-pool 1, ref-count 1, totlen-nifb 0, trace handle 0x10 ext-hdr-valid l4-cksum-computed l4-cksum-correct PKT MBUF: port 0, nb_segs 1, pkt_len 102 buf_len 2176, data_len 102, ol_flags 0x181, data_off 128, phys_addr 0xe2e27140 packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0 Packet Offload Flags PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 Packet Types RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN fragment id 0x91cf, flags DONT_FRAGMENT ICMP echo_request checksum 0x15bb 00:01:35:388395: ethernet-input frame: flags 0x3, hw-if-index 1, sw-if-index 1 IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 00:01:35:388396: ip4-input ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN fragment id 0x91cf, flags DONT_FRAGMENT ICMP echo_request checksum 0x15bb 00:01:35:388397: ip4-lookup fib 0 dpo-idx 6 flow hash: 0x ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN fragment id 0x91cf, flags DONT_FRAGMENT ICMP echo_request checksum 0x15bb 00:01:35:388397: ip4-rewrite tx_sw_if_index 4 dpo-idx 6 : ipv4 via 172.30.0.6 lan0.218: mtu:9000 246e969ce5dfb4969118ebbc81da0800 flow ha sh: 0x : 246e969ce5dfb4969118ebbc81da0800455491cf40003f01e891ac1e 0020: 00020a060b22080015bb4fa504f9a1a5b95e67cf0c00 00:01:35:388397: ipsec4-output-feature spd 1 policy 2 00:01:35:388398: esp4-encrypt esp: sa-index 0 spi 1000 (0x03e8) seq 11 sa-seq-hi 0 crypto aes-cbc-128 integrity sha1-96 00:01:35:388401: punt-dispatch reason: [2] ipsec6-no-such-tunnel 00:01:35:388403: drop punt-dispatch: No registrations Thanks again, Muthu On Tue, May 12, 2020 at 12:24 AM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > Your lan0.218 has address in the 172.16.0.5/16, set the next hop to the > peer on that link in that subnet. If you don’t have one, in this case, you > can use any address… the problem is that IPSec SPD runs as an output > feature, but we don’t run features for packets that hit the glean (i.e. I > need to ARP) adjacency. So the route needs to resolve via a neighbour > adjacency. It doesn’t have to be a complete adj, hence you don’t need an > ARP response, so any address will do. > > > > /neale > > > > *From: * on behalf of Muthu Raj < > muthuraj.muth...@gmail.com> > *Date: *Monday 11 May 2020 at 19:22 > *To: *"Neale Ranns (nranns)" > *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com>, "vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > Thanks
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Filip, I have traced the packet. Here is how I setup the tunnel || <===> || ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 integ-key integ-alg sha1-96 tunnel-src tunnel-dst ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 integ-key integ-alg sha1-96 tunnel-src tunnel-dst ipsec spd add 1 set interface ipsec spd lan0.218 1 ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 ipsec policy add spd 1 priority 10 outbound action protect sa 10 local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - 10.6.255.255 ipsec policy add spd 1 priority 10 inbound action protect sa 20 local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - 10.6.255.255 ip route add 10.6.0.0/16 via lan0.218 vpp# show int address lan0 (up): lan0.218 (up): L3 172.30.0.5/16 local0 (up): wan0 (up): wan0.67 (up): L3 Corresponding config on remote side. Trace: 00:32:40:825694: dpdk-input lan0 rx queue 0 buffer 0x13e1d1: current data 0, length 74, buffer-pool 1, ref-count 1, totlen-nifb 0, trace handle 0x11 ext-hdr-valid l4-cksum-computed l4-cksum-correct PKT MBUF: port 0, nb_segs 1, pkt_len 74 buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr 0xe05874c0 packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0 Packet Offload Flags PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 Packet Types RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712830: ethernet-input frame: flags 0x3, hw-if-index 1, sw-if-index 1 IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 00:32:40:712831: ip4-input ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712831: ip4-lookup fib 0 dpo-idx 3 flow hash: 0x ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712832: ip4-glean ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712833: ip4-drop ICMP: 172.30.0.2 -> 10.6.11.34 tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN fragment id 0xc4a2, flags DONT_FRAGMENT ICMP echo_request checksum 0x7ddd 00:32:40:712833: error-drop rx:lan0.218 00:32:40:712834: drop ip4-glean: ARP requests sent I believe for some reason the policy match is not happening. And I am not able to find examples of IPv4 over IPv6 tunnel, so I might be making a mistake in setting up the tunnel itself. Let me know if you need more info, and thank you so much for looking into this. Much appreciated. Thanks, Muthu On Mon, May 11, 2020 at 3:41 PM Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco) wrote: > Hi Muthu, > > you can enable packet tracing [1] and see what nodes are visited. > > Filip > > [1] > https://wiki.fd.io/view/VPP/How_To_Use_The_Packet_Generator_and_Packet_Tracer#Step_3._Examine_the_setup_script > VPP/How To Use The Packet Generator and Packet Tracer - fd.io > <https://wiki.fd.io/view/VPP/How_To_Use_The_Packet_Generator_and_Packet_Tracer#Step_3._Examine_the_setup_script> > Introduction. The VPP platform includes packet generation and packet > tracing facilities. The following example shows steps that you might > typically use to run a debug version of the vpp executable file, generate > packets, and to analyze results. > wiki.fd.io > > -- > *From:* Muthu Raj > *Sent:* Thursday, May 7, 2020 7:04 PM > *To:* Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco) < > fteh...@cisco.com> > *Cc:* vpp-dev@lists.fd.io > *Subject:* Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > Hello Filip, > > I just tried adding a loopback interface and setting the SPD on that > interface. > Here is how I tried it: > > create loopback interface > set int state loop0 up > set interface ip table loop0 0 > set int ip address loop0 11.11.11.11/32 > ipse
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Neale, The 10.6.0.0/16 is actually on the remote side, and should flow over the IPv6 IPsec tunnel. I was hoping by adding route via the lan0.218 interface, the spd will kick in and send over the tunnel. I guess I'm missing something very glaring, so would appreciate pointers. The source of the traced packet is actually another node, that sends to the lan0.218 IP for the remote subnet destination. Thanks, Muthu On Mon, May 11, 2020, 9:41 PM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > Your lan0 interface is not point-2-point, so your route needs a nexthop; > >ip route add 10.6.0.0/16 via 172.30.x.y lan0.218 > > > > x and y can be anything, apart from x=0 and y=5. > > > > /neale > > > > *From: * on behalf of Muthu Raj < > muthuraj.muth...@gmail.com> > *Date: *Monday 11 May 2020 at 17:39 > *To: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com> > *Cc: *"vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Filip, > > > > I have traced the packet. > > > > Here is how I setup the tunnel > > > > || <===> || Subnet=10.6.0.0/16> > > > > ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec spd add 1 > set interface ipsec spd lan0.218 1 > ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 > ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 > ipsec policy add spd 1 priority 10 outbound action protect sa 10 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > > ip route add 10.6.0.0/16 via lan0.218 > > > > vpp# show int address > lan0 (up): > lan0.218 (up): > L3 172.30.0.5/16 > local0 (up): > wan0 (up): > wan0.67 (up): > L3 > > > > Corresponding config on remote side. > > Trace: > > > > 00:32:40:825694: dpdk-input > lan0 rx queue 0 > buffer 0x13e1d1: current data 0, length 74, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x11 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 74 > buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr > 0xe05874c0 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712830: ethernet-input > frame: flags 0x3, hw-if-index 1, sw-if-index 1 > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > 00:32:40:712831: ip4-input > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712831: ip4-lookup > fib 0 dpo-idx 3 flow hash: 0x > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712832: ip4-glean > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712833: ip4-drop > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712833: error-drop > rx:lan0.218 > 00:32:40:712834: drop > ip4-glean: ARP requests sent > > > > I believe for some reason
[vpp-dev] Troubleshooting IPSec in VPP
Hello, I am trying out IPSec on VPP, and used the wiki[1] to create an IPSec tunnel between an AWS instance(remote) and my home. The tunnel was established successfully, and when pinging an IP on the remote side, the icmp req flows over the tunnel, is seen by the remote box, and responded back as well. I also see that the packets indeed end up reaching my home VPP instance - however, they do not reach the last hop. When I run show int, the ipip0 interface does not show the rx counter at all, and when running `show errors` I do not see the counter for the `ipsec4-tun-input` node either. Neither do I see the `esp4-encrypt-tun' counter. My preliminary guess is that it has something to do with the fact that on AWS we cannot see the public IP inside the instance and so that cannot be assigned to the interface itself, so probably the ESP packets are generated with source as the private IP address corresponding to the public IP. With strongswan, we specify an explicit sourceip parameter, like in the snippet below left=1.2.3.4 leftid=1.2.3.4 leftsubnet=172.16.0.0/16 right=4.5.6.7 #AWS public IP rightsourceip=10.6.82.34 #AWS private IP for that public IP, seen inside the instance. rightsubnet=10.6.0.0/16 I am attaching the ikev2 sa as seen from both sides. How would I fix this issue? Any help is appreciated very much. Thanks in advance. This is from the home side. I've changed the IPs on home and remote side. The private IP addresses have been left as it is. vpp# show ikev2 sa iip 1.2.3.4 ispi d8607eea97ac12a9 rip 4.5.6.7 rspi d6726c2768b2420 encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048 nonce i:eb01e6ef107ba7018679bd239e25d4557f2465323caf0d3213b453ca59af3deb r:1d9cb8f11cd69d4b2f73b182028d8aa8854a49bb3c99797f3994575c2994154c SK_d1eee29fff1ff234f1452006a79a7e27787e83331b29954300a70a9d6061f2fde SK_a i:7f86a547c2d9cb2a4035e4926ca6e23c745c6c8c r:04a71f139f2076058ceafb9be73eb359e43bc308 SK_e i:281c47cd100f69a3425031667150d3054124ff887d77a4a1f43fd7dece7486fc r:3f72f8e973ee62962dc9dffd64d80af9e83993acbcd3690adf85044a23310409 SK_p i:79c096024c45499bd43b5d716c56e5152252c433b112195201dd5c4c23a1f1c7 r:fb7e3b35d57b2987bf61f04858a4afaeee10045c6001594f9f2e505b94d950d8 identifier (i) fqdn vpp.aws identifier (r) fqdn vpp.home child sa 0: encr:aes-cbc-256 integ:sha1-96 esn:yes spi(i) 147e7a05 spi(r) de36dcbc SK_e i:31e22be618e3fe60faf935759e75fdc699f743486dd18f07de8b78747d10d229 r:30b10195fdb1cd5b7384a2db92d5a51fd9fab7f6fc7db775957e3dc862d72532 SK_a i:f98f3539966a66afec330c7cdf85fbe2794e01d3 r:9504182eb614d90aa8fe742122ec9d98c1b6e224 traffic selectors (i): 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 - 65535 traffic selectors (r): 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535 iip 1.2.3.4 ispi d8607eea97ac12a9 rip 4.5.6.7 rspi d6726c2768b2420 Here is the AWS side vpp# show ikev2 sa iip 1.2.3.4 ispi a72ae3cef809725c rip rspi b8b7b8ef09266a6d encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048 nonce i:d3e4299761fd93edd3df16456cb0ca9f717f67e57155fa7cb4cd0b9a1d371019 r:e9a33a33b901366438e262d225a418e9489839415562d3e3673107e0d81d830f SK_d7e4f795db87a02c5b4d5ea738945521473f5e449b783f3ac4b954be7716b7909 SK_a i:13639a11b6e96e65dd38d095a87fc1b5ceefdc6b r:97c96809563dfe39c3d2762c1ff1bf0a8fbc3576 SK_e i:114661a058686bd4362d8515ce83a7d7de098af11b08084c407ad51843316135 r:d812542cfa988e6c302fc52d848fb2d7b7321d6c3e77ee04134338a21c0ccba8 SK_p i:a65ea61c70b3cb749dedc205b7715b4c278a4bc630c6508d89a55a00cd00a2cd r:9e23352bac4d21f6f0d2ec8de82e556db3ddaba0ade0c4d664a020da3986d17b identifier (i) fqdn vpp.home identifier (r) fqdn vpp.aws child sa 0: encr:aes-cbc-256 integ:sha1-96 esn:yes spi(i) 31c649f8 spi(r) 967b11c4 SK_e i:6a1b5898746bc922af1beba021768cd6417a0e8a4c555e5544781fee302cf633 r:2035a8be8fae47c284cef445381cef487bcd670bddc31558109c0303bc0f5399 SK_a i:da119e539529803a3d2a883c01a825211c782bd2 r:2330bc2dd9eb3741e3df649bcc3f7e5320fba512 traffic selectors (i): 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 - 65535 traffic selectors (r): 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535 iip 1.2.3.4 ispi a72ae3cef809725c rip rspi b8b7b8ef09266a6d iip 1.2.3.4 ispi d8607eea97ac12a9 rip rspi d6726c2768b2420 encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048 nonce i:eb01e6ef107ba7018679bd239e25d4557f2465323caf0d3213b453ca59af3deb r:1d9cb8f11cd69d4b2f73b182028d8aa8854a49bb3c99797f3994575c2994154c SK_d1eee29fff1ff234f1452006a79a7e27787e83331b29954300a70a9d6061f2fde SK_a i:7f86a547c2d9cb2a4035e4926ca6e23c745c6c8c r:04a71f139f2076058ceafb9be73eb359e43bc308 SK_e
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
checksum 0xf51f dscp CS0 ecn NON_ECN fragment id 0x6e42, flags DONT_FRAGMENT ICMP echo_request checksum 0x15b1 20:48:33:279242: ip4-rewrite tx_sw_if_index 3 dpo-idx 6 : ipv4 via 11.11.11.10 loop0: mtu:9000 next:4 246e969ce5 dfdead0800 flow hash: 0x : 246e969ce5dfdead080045546e4240003f01f61f0a062121ac1e 0020: 0002080015b178da0011bd5ec60f05001011 20:48:33:279244: ipsec4-output-feature spd 1 policy 2 20:48:33:279246: esp4-encrypt esp: sa-index 0 spi 1000 (0x03e8) seq 13028 sa-seq-hi 0 crypto aes-cbc-128 inte grity sha1-96 20:48:33:279255: ip6-load-balance fib 3 dpo-idx 3 flow hash: 0x IPSEC_ESP: 2001::1 -> 2001::2 tos 0x00, flow label 0x0, hop limit 254, payload length 132 20:48:33:279258: ip6-rewrite tx_sw_if_index 1 adj-idx 3 : ipv6 via fe80::1043:12ff:fec4:2197 wan0: mtu:9000 next :3 124312c4219712273c81f8f386dd flow hash: 0x : 124312c4219712273c81f8f386dd608432fd26001f18254c9301e507 0020: 7624cf3f7184260410c1001003e832e42112 0040: 6acf95626ff015f514ce23230e5cab93ba24df0ade533f87b96bc02856a7f0e7 0060: df4922f832163a63346bb518f1308c757bbda288b750bd2cae26732a 20:48:33:279258: wan0-output wan0 IP6: 12:27:3c:81:f8:f3 -> 12:43:12:c4:21:97 IPSEC_ESP: 2001::1 -> 2001::2 tos 0x00, flow label 0x0, hop limit 253, payload length 132 20:48:33:279260: wan0-tx wan0 tx queue 0 buffer 0x8ee28: current data -64, length 186, buffer-pool 0, ref-count 1, trace han dle 0x0 ext-hdr-valid l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 l3-hdr-offset 14 PKT MBUF: port 1, nb_segs 1, pkt_len 186 buf_len 2176, data_len 186, ol_flags 0x0, data_off 64, phys_addr 0xb3fb8a80 packet_type 0x10 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0xed0f5862 fdir.hi 0x0 fdir.lo 0xed0f5862 Packet Types RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers IP6: 12:27:3c:81:f8:f3 -> 12:43:12:c4:21:97 IPSEC_ESP: 2001::1 -> 2001::2 tos 0x00, flow label 0x0, hop limit 253, payload length 132 Let me know if you need more info. Thanks, Muthu On Wed, May 13, 2020 at 9:15 PM Muthu Raj via lists.fd.io wrote: > Hi Neale, > > Thanks a lot for looking into this and providing a fix, much appreciated. > > I have applied the fix, and that solved the issue on the sender side. > The packet successfully reached the other side. However, the issue is on > the other side now. > > The packet trace looks like this: (I've put dummy local and remote IPv6 > IPs) > > Packet 12 > > 02:06:48:110231: dpdk-input > wan0 rx queue 0 > buffer 0x11e885: current data 0, length 74, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0xb >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 1, nb_segs 1, pkt_len 74 > buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr > 0xe27a21c0 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 1, nb_segs 1, pkt_len 190 > buf_len 2176, data_len 190, ol_flags 0x181, data_off 128, phys_addr > 0xe279a300 > packet_type 0x41 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 67 > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV6 (0x0040) IPv6 packet without extension headers > IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67 > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 02:06:48:074127: ethernet-input > frame: flags 0x3, hw-if-index 2, sw-if-index 2 > IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67 > 02:06:48:074128: ip6-input > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 02:06:48:074128: ip6-lookup > fib 0 dpo-idx 8 flow hash: 0x > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 02:06:48:074129: ip6-local > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 02:06:48:074130: ip6-punt > IPSEC_ESP: 2001::2 -> 2001::1 > tos 0x28, flow label 0x0, hop limit 238, payload length
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Neale, Sorry about the trace. The match in the SPD is against SA 20’s tunnel addresses not the policy’s > local/remote range. Thanks for clarifying this. I was confused here. I created the SA and policy with this in mind and got it to work successfully. Thanks a lot for your help. I will spend some time this coming week and try to get a small write up onto https://wiki.fd.io/ It may be of help to someone. Muthu On Thu, May 14, 2020 at 8:52 PM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > The tracing is not great, but what you see indicates a miss in the SPD. > > The match in the SPD is against SA 20’s tunnel addresses not the policy’s > local/remote range. > > > > /neale > > > > > > *From: *Muthu Raj > *Date: *Thursday 14 May 2020 at 15:51 > *To: *"muthuraj.muth...@gmail.com" > *Cc: *"Neale Ranns (nranns)" , "Filip Tehlar -X > (ftehlar - PANTHEON TECH SRO at Cisco)" , " > vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > So I've since tried out setting SPD on the interface with the IPv6 > address, and even though I am not able to ping the interface, I see that it > does receive and process packets (which I had erroneously assumed it did > not when it became unpingable). > > > > > > I added a new SPD and added a policy like so > > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range - remote-ip-range - > > > > > This is what the trace looks like: > > > > Packet 10 > > 01:02:05:902414: dpdk-input > lan0 rx queue 0 > buffer 0x13daa8: current data 0, length 96, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x9 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 96 > buf_len 2176, data_len 96, ol_flags 0x181, data_off 128, phys_addr > 0xe2f6aa80 > packet_type 0x211 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > 01:02:05:868297: ethernet-input > frame: flags 0x3, hw-if-index 2, sw-if-index 2 > IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67 > 01:02:05:868299: ip6-input > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868299: ipsec6-input-feature > IPSEC_ESP: sa_id 0 spd 2 policy 0 spi 1000 (0x03e8) seq 13693 > 01:02:05:868300: ip6-lookup > fib 0 dpo-idx 8 flow hash: 0x > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868301: ip6-local > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868301: ip6-punt > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868302: error-punt > rx:wan0.67 > 01:02:05:868302: punt > ip6-input: valid ip6 packets > > > > > > IPSEC_ESP: sa_id 0 spd 2 policy 0 spi 1000 (0x03e8) seq 13693 > > > > Here, the spd 2 actually does not have any policy in the 0 index. > > here is what show ipsec spd 2 looks like: > > > > ip6-inbound-protect: >[4] priority 100 action protect type ip6-inbound-protect protocol any > sa 20 > local addr range 2001::2 - 2001::2 port range 0 - 65535 > remote addr range 2001::1 - 2001::1 port range 0 - 65535 > packets 0 bytes 0 > > > > Is there a mistake in the way SPD has been added? > > Or is something else the issue? > > > > Here is trace as seen by the sender: > > > > Packet 1 > > > 20:48:33:279228: dpdk-input > eth0 rx queue 0 > buffer 0x8ee28: current data 0, length 98, buffer-pool 0, ref-count 1, > totlen-nifb 0, trace handle 0x0 > ext-hdr-valid > l4-cksum-computed l4-cksum-correct > PKT MBUF: port 1, nb_segs 1, pkt_len 98 > buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr > 0xb3fb8a80 > packet_type 0x10 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0xed0f5862 fdir.hi 0x0 fdir.lo 0xed0f5862 > Packet Types > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: 12:6a:27:79:97:3c -> 12:71:86:3a:04:7b > ICMP: 10.6.33.33 -> 172.30.0.2 > tos 0x00, ttl 64, length 84, checksum 0xf51f dscp CS0 ecn NON_ECN > fragment id 0x6e42, flags DONT_FRAGMENT > ICMP echo_request checksum 0x15b1 > 20:48:33:279234: ethernet-input > frame: flags
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
interface, that gets all the packets destined for wan0.67 (may be through an l2 bridge?) and SPD should be there. Am I in the right path? If yes, could you direct me to documentation regarding oneway L2 bridging to a virtual / loopback interface? Thanks so much for looking into this, once again. Thanks, Muthu On Wed, May 13, 2020 at 2:54 PM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > I tried your 4over6 config, it doesn’t work… here’s the fix: > > https://gerrit.fd.io/r/c/vpp/+/27019 > > > > /neale > > > > *From: *Muthu Raj > *Date: *Monday 11 May 2020 at 21:36 > *To: *"Neale Ranns (nranns)" > *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com>, "vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > Thank you so much for patiently explaining. I initially added next-hop > like this - > > ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218 > > > But it still got dropped with ip4-arp: ARP requests sent. > > > > Then I read your message again, and realised that the ARP response really > doesn't matter. So I set a static ARP entry like so - > > set ip neighbor lan0.218 172.30.0.6 c8:5b:76:d9:7f:9c > > > > That thankfully, changed the nodes visited. Unfortunately, it still fails > with an error like below - ipsec6-no-such-tunnel. > > I understand that it must be some mistake in setting up the tunnel, but > once again, I am not sure how to proceed. > > > > The packet trace is at the end. > > > > Here is the way I set-up the tunnel: > > > > ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec spd add 1 > set interface ipsec spd lan0.218 1 > ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 > ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 > ipsec policy add spd 1 priority 10 outbound action protect sa 10 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218 > > set ip neighbor lan0.218 172.30.0.6 24:6e:96:9c:e5:df > > > > *Packet trace:* > > > 00:01:35:388393: dpdk-input > lan0 rx queue 0 > buffer 0x1389c3: current data 0, length 102, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x10 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 102 > buf_len 2176, data_len 102, ol_flags 0x181, data_off 128, phys_addr > 0xe2e27140 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN > fragment id 0x91cf, flags DONT_FRAGMENT > ICMP echo_request checksum 0x15bb > 00:01:35:388395: ethernet-input > frame: flags 0x3, hw-if-index 1, sw-if-index 1 > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > 00:01:35:388396: ip4-input > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN > fragment id 0x91cf, flags DONT_FRAGMENT > ICMP echo_request checksum 0x15bb > 00:01:35:388397: ip4-lookup > fib 0 dpo-idx 6 flow hash: 0x > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN > fragment id 0x91cf, flags DONT_FRAGMENT > ICMP echo_request checksum 0x15bb > 00:01:35:388397: ip4-rewrite > tx_sw_if_index 4 dpo-idx 6 : ipv4 via 172.30.0.6 lan0.218: mtu:9000 > 246e969ce5dfb4969118ebbc81da0800 flow ha > sh: 0x > : > 246e969ce5dfb4969118ebbc81da0800455491cf40003f01e891ac1e > 0020: 00020a060b22080015bb4fa504f9a1a5b95e00
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hi Neale, Thanks for responding so quickly. It might sound a little bit pedestrian, but what would be the next hop in this case? Would it be the IPv6 IP (over which the tunnel is established) ? I have added it through the lan0.218 interface with the assumption that the sa will kick in and send over the tunnel. On Mon, May 11, 2020, 10:26 PM Neale Ranns (nranns) wrote: > > > > > *From: *Muthu Raj > *Date: *Monday 11 May 2020 at 18:42 > *To: *"Neale Ranns (nranns)" > *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com>, "vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > The 10.6.0.0/16 is actually on the remote side, and should flow over the > IPv6 IPsec tunnel. > > I was hoping by adding route via the lan0.218 interface, the spd will kick > in and send over the tunnel. I guess I'm missing something very glaring, so > would appreciate pointers. > > > > You’re missing next-hop for the route. > > > > /neale > > > > The source of the traced packet is actually another node, that sends to > the lan0.218 IP for the remote subnet destination. > > > > > > Thanks, > > Muthu > > > > On Mon, May 11, 2020, 9:41 PM Neale Ranns (nranns) > wrote: > > > > Hi Muthu, > > > > Your lan0 interface is not point-2-point, so your route needs a nexthop; > >ip route add 10.6.0.0/16 via 172.30.x.y lan0.218 > > > > x and y can be anything, apart from x=0 and y=5. > > > > /neale > > > > *From: * on behalf of Muthu Raj < > muthuraj.muth...@gmail.com> > *Date: *Monday 11 May 2020 at 17:39 > *To: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com> > *Cc: *"vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Filip, > > > > I have traced the packet. > > > > Here is how I setup the tunnel > > > > || <===> || Subnet=10.6.0.0/16> > > > > ipsec sa add 10 spi 1000 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec sa add 20 spi 1001 esp crypto-key crypto-alg aes-cbc-128 > integ-key integ-alg sha1-96 tunnel-src tunnel-dst > > ipsec spd add 1 > set interface ipsec spd lan0.218 1 > ipsec policy add spd 1 priority 100 inbound action bypass protocol 50 > ipsec policy add spd 1 priority 100 outbound action bypass protocol 50 > ipsec policy add spd 1 priority 10 outbound action protect sa 10 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 - > 10.6.255.255 > > ip route add 10.6.0.0/16 via lan0.218 > > > > vpp# show int address > lan0 (up): > lan0.218 (up): > L3 172.30.0.5/16 > local0 (up): > wan0 (up): > wan0.67 (up): > L3 > > > > Corresponding config on remote side. > > Trace: > > > > 00:32:40:825694: dpdk-input > lan0 rx queue 0 > buffer 0x13e1d1: current data 0, length 74, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x11 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 74 > buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr > 0xe05874c0 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218 > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2, flags DONT_FRAGMENT > ICMP echo_request checksum 0x7ddd > 00:32:40:712830: ethernet-input > frame: flags 0x3, hw-if-index 1, sw-if-index 1 > IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218 > 00:32:40:712831: ip4-input > ICMP: 172.30.0.2 -> 10.6.11.34 > tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN > fragment id 0xc4a2,
Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind NAT (AWS instance)
Hello, I have just added an use case over at https://wiki.fd.io/view/VPP/IPSec_and_IKEv2#IPSec_between_VPP_peers.2C_tunneling_IPv4_over_IPv6 It is pretty bare bones for now, but I hope to continue to improve it. Feel free to point out mistakes if there are any. I also will try to write a longer version explaining more things (more like capturing what neale explained to me) with traces in the VPP user docs. Thanks Neale, Filip and everyone. On Fri, May 15, 2020 at 3:11 PM Neale Ranns (nranns) wrote: > > > Hi Muthu, > > > > *From: * on behalf of Muthu Raj < > muthuraj.muth...@gmail.com> > *Date: *Friday 15 May 2020 at 09:20 > *To: *"Neale Ranns (nranns)" > *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" < > fteh...@cisco.com>, "vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > Sorry about the trace. > > > > Not your fault at all I was commenting that the trace VPP produced was > not clear in indicating the miss. > > > > The match in the SPD is against SA 20’s tunnel addresses not the policy’s > local/remote range. > > > > Thanks for clarifying this. I was confused here. > > > > I created the SA and policy with this in mind and got it to work > successfully. > > Thanks a lot for your help. > > > > Glad to hear it. > > > > I will spend some time this coming week and try to get a small write up > onto https://wiki.fd.io/ > > It may be of help to someone. > > > > I’m sure it will be. Thanks you! > > > > /neale > > > > Muthu > > > > On Thu, May 14, 2020 at 8:52 PM Neale Ranns (nranns) > wrote: > > > > Hi Muthu, > > > > The tracing is not great, but what you see indicates a miss in the SPD. > > The match in the SPD is against SA 20’s tunnel addresses not the policy’s > local/remote range. > > > > /neale > > > > > > *From: *Muthu Raj > *Date: *Thursday 14 May 2020 at 15:51 > *To: *"muthuraj.muth...@gmail.com" > *Cc: *"Neale Ranns (nranns)" , "Filip Tehlar -X > (ftehlar - PANTHEON TECH SRO at Cisco)" , " > vpp-dev@lists.fd.io" > *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer > behind NAT (AWS instance) > > > > Hi Neale, > > > > So I've since tried out setting SPD on the interface with the IPv6 > address, and even though I am not able to ping the interface, I see that it > does receive and process packets (which I had erroneously assumed it did > not when it became unpingable). > > > > > > I added a new SPD and added a policy like so > > ipsec policy add spd 1 priority 10 inbound action protect sa 20 > local-ip-range - remote-ip-range - > > > > > This is what the trace looks like: > > > > Packet 10 > > 01:02:05:902414: dpdk-input > lan0 rx queue 0 > buffer 0x13daa8: current data 0, length 96, buffer-pool 1, ref-count 1, > totlen-nifb 0, trace handle 0x9 >ext-hdr-valid >l4-cksum-computed l4-cksum-correct > PKT MBUF: port 0, nb_segs 1, pkt_len 96 > buf_len 2176, data_len 96, ol_flags 0x181, data_off 128, phys_addr > 0xe2f6aa80 > packet_type 0x211 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > 01:02:05:868297: ethernet-input > frame: flags 0x3, hw-if-index 2, sw-if-index 2 > IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67 > 01:02:05:868299: ip6-input > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868299: ipsec6-input-feature > IPSEC_ESP: sa_id 0 spd 2 policy 0 spi 1000 (0x03e8) seq 13693 > 01:02:05:868300: ip6-lookup > fib 0 dpo-idx 8 flow hash: 0x > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868301: ip6-local > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868301: ip6-punt > IPSEC_ESP: 2001::1 -> 2001::2 > tos 0x28, flow label 0x0, hop limit 238, payload length 132 > 01:02:05:868302: error-punt > rx:wan0.67 > 01:02:05:868302: punt > ip6-input: valid ip6 packets > > > > > > IPSEC_ESP: sa_id 0 spd 2 policy 0 spi 1000 (0x03e8) seq 13693 > > > > Here, the spd 2 actually does not have any policy in the 0 index. > > here is what show ipsec spd 2 looks like: > > > > ip6-inbound-protect:
Re: [vpp-dev] Observing multiple VRRP Routers acting as master while testing Master/Back-up functionality using vrrp plugin
Hello Amit, state Master flags: preempt yes accept *no* unicast no Your clue lies here. Check https://vpp.flirble.org/master/d7/d40/vrrp_8c_source.html#l00182 and https://vpp.flirble.org/master/dc/dfb/vrrp__cli_8c.html#a2fd76fa6d5cd9ddfef75af8f0d12e016 HTH. Muthu On Sat, Jun 13, 2020 at 2:01 PM Amit Mehra wrote: > Hi, > > I am trying to test Master/Backup functionality on 2 VPP nodes, each > running it's own VRRP plugin and I am observing 2 VRRP routers acting as > Masters at the same time. Please find below the configuration that i am > using for my testing:- > > On VVP Node 1:- > set interface mtu 1500 GigabitEthernet13/0/0 > set interface ip address GigabitEthernet13/0/0 10.20.37.118/24 > set interface state GigabitEthernet13/0/0 up > vrrp vr add GigabitEthernet13/0/0 vr_id 1 priority 255 10.20.37.118 > vrrp proto start GigabitEthernet13/0/0 vr_id 1 > > This becomes a Master, as it is the address owner and has the priority as > 255. > > On VPP Node2:- > set interface mtu 1500 GigabitEthernet13/0/0 > set interface ip address GigabitEthernet13/0/0 10.20.37.109/24 > set interface state GigabitEthernet13/0/0 up > vrrp vr add GigabitEthernet13/0/0 vr_id 1 priority 200 accept_mode > 10.20.37.118 > vrrp proto start GigabitEthernet13/0/0 vr_id 1 > > This becomes a back-up and has priority 200. > > Note:- 10.20.37.118 is the IP that will be pinged from external > machine(which would be having same subnet as 10.20.37.xx) and that is why > have configured this IP on VR (VVP instance 2) with "accept_mode" ON. > > So, VR on VPP-1 comes up as Master and VR on VPP-2 comes up as Back-up. > Now, if i kill VPP-1, then VR on VPP-2 becomes a Master and IP 10.20.37.118 > also gets added to the VPP interface. Please find the output of show int > addr below > > vpp# show int addr > GigabitEthernet13/0/0 (up): > L3 10.20.37.109/24 > L3 10.20.37.118/24 > GigabitEthernet1b/0/0 (dn): > local0 (dn): > > vpp# show vrrp vr > [0] sw_if_index 1 VR ID 1 IPv4 >state Master flags: preempt yes accept yes unicast no >priority: configured 200 adjusted 200 >timers: adv interval 100 master adv 100 skew 21 master down 321 >virtual MAC 00:00:5e:00:01:01 >addresses 10.20.37.118 >peer addresses >tracked interfaces > > Now, if i bring VPP-1 up, the VR on VVP-1 also becomes a Master and VR on > VPP-2 also remains in Master State.At this moment, both the VRs i.e on > VPP-1 and VPP-2 are acting as Masters. Please find the output of show vrrp > vr from both the VPP instances:- > > From VPP-1 > vpp# show vrrp vr > [0] sw_if_index 1 VR ID 1 IPv4 >state Master flags: preempt yes accept no unicast no >priority: configured 255 adjusted 255 >timers: adv interval 100 master adv 0 skew 0 master down 0 >virtual MAC 00:00:5e:00:01:01 >addresses 10.20.37.118 >peer addresses >tracked interfaces > > vpp# show int addr > GigabitEthernet13/0/0 (up): > L3 10.20.37.118/24 > GigabitEthernet1b/0/0 (dn): > local0 (dn): > > On VPP-2 > vpp# show vrrp vr > [0] sw_if_index 1 VR ID 1 IPv4 >state Master flags: preempt yes accept yes unicast no >priority: configured 200 adjusted 200 >timers: adv interval 100 master adv 100 skew 21 master down 321 >virtual MAC 00:00:5e:00:01:01 >addresses 10.20.37.118 >peer addresses >tracked interfaces > > vpp# show int addr > GigabitEthernet13/0/0 (up): > L3 10.20.37.109/24 > L3 10.20.37.118/24 > GigabitEthernet1b/0/0 (dn): > local0 (dn): > > Is this a known issue or am i missing something in my configuration? How > to overcome this issue of multiple VRRP routers acting as Master at the > same time? > > Regards > Amit > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16719): https://lists.fd.io/g/vpp-dev/message/16719 Mute This Topic: https://lists.fd.io/mt/74854562/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
