Hi Xingyeping, My customer use case if gre-over-ipsec but we want to use mGRE only because we don't want to create too many GRE tunnels which is the case if there are too many peers connecting with VPP (for a GRE p2p case)
I saw your example, it looks like you are using GRE with manually keyed IPSec (ipsec sad and spd are manually chosen). But in our case we are using IKEv2 stack which creates SA. Did you test GRE-over-IPSec using IKEv2 way of SA creation and not manual IPSec SA? On Tue, Mar 23, 2021 at 11:49 AM xingyep...@163.com <xingyep...@163.com> wrote: > Hi： > I use Gre over IPsec（not mgre），it's ok。There is only gre tunnel， no ipip > tunnel： > vpp1 : > > set interface state GigabitEthernet0/b/0 up > > set interface ip address GigabitEthernet0/b/0 10.1.1.3/24 > > create gre tunnel src 10.1.1.3 dst 10.1.1.4 > > set interface state gre0 up > > set interface unnumbered gre0 use GigabitEthernet0/b/0 > > > > create host-interface name vpp1out > > set interface state host-vpp1out up > > set interface ip address host-vpp1out 10.10.1.2/24 > > ip route add 10.10.2.0/24 via gre0 > > > > ipsec sa add 10 spi 2001 esp crypto-alg aes-cbc-128 crypto-key > 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key > 4339314b55523947594d6d3547666b45764e6a58 > > ipsec tunnel protect gre0 sa-in 10 sa-out 10 > > > vpp2： > > create host-interface name vpp2out > > set interface state host-vpp2out up > > set interface ip address host-vpp2out 10.10.2.2/24 > > > > set interface state GigabitEthernet0/b/0 up > > set interface ip address GigabitEthernet0/b/0 10.1.1.4/24 > > create gre tunnel src 10.1.1.4 dst 10.1.1.3 > > set interface state gre0 up > > set interface unnumbered gre0 use GigabitEthernet0/b/0 > > ip route add 10.10.1.0/24 via gre0 > > > > ipsec sa add 10 spi 2001 esp crypto-alg aes-cbc-128 crypto-key > 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key > 4339314b55523947594d6d3547666b45764e6a58 > > ipsec tunnel protect gre0 sa-in 10 sa-out 10 > > *Router-A Trace：* > > > > > > DBGvpp# trace add af-packet-input 10 > > > DBGvpp# show trace > > ------------------- Start of thread 0 vpp_main ------------------- > > No packets in trace buffer > > ------------------- Start of thread 1 vpp_wk_0 ------------------- > > No packets in trace buffer > > ------------------- Start of thread 2 vpp_wk_1 ------------------- > > Packet 1 > > > > 00:33:03:857975: af-packet-input > > af_packet: hw_if_index 5 next-index 4 > > tpacket2_hdr: > > status 0x20000001 len 98 snaplen 98 mac 66 net 80 > > sec 0x60137ac5 nsec 0x230ab2d4 vlan 0 vlan_tpid 0 > > 00:33:03:858004: ethernet-input > > IP4: 6a:2a:e7:05:19:8e -> 02:fe:ec:52:c8:c8 > > 00:33:03:858012: ip4-input > > ICMP: 10.10.1.1 -> 10.10.2.1 > > tos 0x00, ttl 64, length 84, checksum 0xb6b9 dscp CS0 ecn NON_ECN > > fragment id 0x6cda, flags DONT_FRAGMENT > > ICMP echo_request checksum 0x9bb3 > > 00:33:03:858024: ip4-lookup > > fib 0 dpo-idx 7 flow hash: 0x00000000 > > ICMP: 10.10.1.1 -> 10.10.2.1 > > tos 0x00, ttl 64, length 84, checksum 0xb6b9 dscp CS0 ecn NON_ECN > > fragment id 0x6cda, flags DONT_FRAGMENT > > ICMP echo_request checksum 0x9bb3 > > 00:33:03:858055: ip4-midchain > > tx_sw_if_index 4 dpo-idx 7 : ipv4 via 0.0.0.0 gre0: mtu:9000 next:7 > 4500000000000000fe2fa6c60a0101030a01010400000800 > > stacked-on entry:21: > > [@2]: ipv4 via 10.1.1.4 GigabitEthernet0/b/0: mtu:9000 next:4 > 525400280679525400b1983f0800 flow hash: 0x00000000 > > 00000000: > 4500006c00000000fe2fa65a0a0101030a01010400000800450000546cda4000 > > 00000020: > 3f01b7b90a0a01010a0a020108009bb37da50001c57a1360000000003ef80800 > > 00000040: > 00000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b > > 00000060: > 2c2d2e2f30313233343536370000000000000000000000000000000000000000 > > 00000080: > 0000000000000000000000000000000000000000000000000000000000000000 > > 000000a0: > 0000000000000000000000000000000000000000000000000000000000000000 > > 000000c0: > 0000000000000000000000000000000000000000000000000000000000000000 > > 000000e0: 00000000000000000000000000000000000000000000000000000000 > > 00:33:03:858064: esp4-encrypt-tun > > esp: sa-index 0 spi 2001 (0x000007d1) seq 4 sa-seq-hi 0 crypto > aes-cbc-128 integrity sha1-96 > > 00:33:03:858088: adj-midchain-tx > > adj-midchain:[7]:ipv4 via 0.0.0.0 gre0: mtu:9000 next:7 > 4500000000000000fe2fa6c60a0101030a01010400000800 > > stacked-on entry:21: > > [@2]: ipv4 via 10.1.1.4 GigabitEthernet0/b/0: mtu:9000 next:4 > 525400280679525400b1983f0800 > > 00:33:03:858090: ip4-rewrite > > tx_sw_if_index 3 dpo-idx 4 : ipv4 via 10.1.1.4 GigabitEthernet0/b/0: > mtu:9000 next:4 525400280679525400b1983f0800 flow hash: 0x00000000 > > 00000000: > 525400280679525400b1983f08004500009800000000fd32a72b0a0101030a01 > > 00000020: > 0104000007d10000000499d0d3e1eb5b77d7d723d0fd448e8023ee2ab5786d20 > > 00000040: > de9c788457480384ca18cf23a242192a9d83c87be32f1ddefad5e74b1fa5c08f > > 00000060: > 6f9fa089191cb7a0e6f4afb318976582f991575a60e3245a9b046e39becc5547 > > 00000080: > 94217ad37f3d55ac2b0258afb4b0ee738da60321f089d90708fdd73937d3c5ea > > 000000a0: > d07b29e97c650000000000000000000000000000000000000000000000000000 > > 000000c0: > 0000000000000000000000000000000000000000000000000000000000000000 > > 000000e0: 00000000000000000000000000000000000000000000000000000000 > > 00:33:03:858093: GigabitEthernet0/b/0-output > > GigabitEthernet0/b/0 > > IP4: 52:54:00:b1:98:3f -> 52:54:00:28:06:79 > > IPSEC_ESP: 10.1.1.3 -> 10.1.1.4 > > tos 0x00, ttl 253, length 152, checksum 0xa72b dscp CS0 ecn NON_ECN > > fragment id 0x0000 > > 00:33:03:858107: GigabitEthernet0/b/0-tx > > GigabitEthernet0/b/0 tx queue 2 > > buffer 0x9faf4: current data -48, length 166, buffer-pool 0, ref-count > 1, trace handle 0x2000000 > > l2-hdr-offset 0 l3-hdr-offset 14 > > PKT MBUF: port 65535, nb_segs 1, pkt_len 166 > > buf_len 2176, data_len 166, ol_flags 0x0, data_off 80, phys_addr > 0xa67ebd80 > > packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > > IP4: 52:54:00:b1:98:3f -> 52:54:00:28:06:79 > > IPSEC_ESP: 10.1.1.3 -> 10.1.1.4 > > tos 0x00, ttl 253, length 152, checksum 0xa72b dscp CS0 ecn NON_ECN > > fragment id 0x0000 > > > > > > > > > > > *Router-B Trace：* > > Packet 10 > > > > 00:32:55:147921: dpdk-input > > GigabitEthernet0/b/0 rx queue 0 > > buffer 0x89a85: current data 0, length 166, buffer-pool 0, ref-count 1, > totlen-nifb 0, trace handle 0x1000009 > > ext-hdr-valid > > l4-cksum-computed l4-cksum-correct > > PKT MBUF: port 2, nb_segs 1, pkt_len 166 > > buf_len 2176, data_len 166, ol_flags 0x0, data_off 128, phys_addr > 0xa646a1c0 > > packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > > IP4: 52:54:00:b1:98:3f -> 52:54:00:28:06:79 > > IPSEC_ESP: 10.1.1.3 -> 10.1.1.4 > > tos 0x00, ttl 253, length 152, checksum 0xa72b dscp CS0 ecn NON_ECN > > fragment id 0x0000 > > 00:32:55:147928: ethernet-input > > frame: flags 0x1, hw-if-index 3, sw-if-index 3 > > IP4: 52:54:00:b1:98:3f -> 52:54:00:28:06:79 > > 00:32:55:147932: ip4-input > > IPSEC_ESP: 10.1.1.3 -> 10.1.1.4 > > tos 0x00, ttl 253, length 152, checksum 0xa72b dscp CS0 ecn NON_ECN > > fragment id 0x0000 > > 00:32:55:147936: ip4-lookup > > fib 0 dpo-idx 10 flow hash: 0x00000000 > > IPSEC_ESP: 10.1.1.3 -> 10.1.1.4 > > tos 0x00, ttl 253, length 152, checksum 0xa72b dscp CS0 ecn NON_ECN > > fragment id 0x0000 > > 00:32:55:147943: ip4-local > > IPSEC_ESP: 10.1.1.3 -> 10.1.1.4 > > tos 0x00, ttl 253, length 152, checksum 0xa72b dscp CS0 ecn NON_ECN > > fragment id 0x0000 > > 00:32:55:147945: ipsec4-tun-input > > IPSec: remote:10.1.1.3 spi:2001 (0x000007d1) seq 5 sa 0 > > 00:32:55:147950: esp4-decrypt-tun > > esp: crypto aes-cbc-128 integrity sha1-96 pkt-seq 5 sa-seq 0 sa-seq-hi 0 > > 00:32:55:147964: ip4-input-no-checksum > > ICMP: 10.10.1.1 -> 10.10.2.1 > > tos 0x00, ttl 63, length 84, checksum 0xb7b7 dscp CS0 ecn NON_ECN > > fragment id 0x6cdc, flags DONT_FRAGMENT > > ICMP echo_request checksum 0xd748 > > 00:32:55:147965: ip4-lookup > > fib 0 dpo-idx 7 flow hash: 0x00000000 > > ICMP: 10.10.1.1 -> 10.10.2.1 > > tos 0x00, ttl 63, length 84, checksum 0xb7b7 dscp CS0 ecn NON_ECN > > fragment id 0x6cdc, flags DONT_FRAGMENT > > ICMP echo_request checksum 0xd748 > > 00:32:55:147969: ip4-rewrite > > tx_sw_if_index 5 dpo-idx 7 : ipv4 via 10.10.2.1 host-vpp2out: mtu:9000 > next:6 a6d09d34bf3602fe6c5a64340800 flow hash: 0x00000000 > > 00000000: > a6d09d34bf3602fe6c5a64340800450000546cdc40003e01b8b70a0a01010a0a > > 00000020: > 02010800d7487da50002c67a1360000000000162090000000000101112131415 > > 00000040: > 161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435 > > 00000060: > 3637010203040506062feb79f044a653a0fc6dc183dd00000000000000000000 > > 00000080: > 0000000000000000000000000000000000000000000000000000000000000000 > > 000000a0: > 0000000000000000000000000000000000000000000000000000000000000000 > > 000000c0: > 0000000000000000000000000000000000000000000000000000000000000000 > > 000000e0: 00000000000000000000000000000000000000000000000000000000 > > 00:32:55:147972: host-vpp2out-output > > host-vpp2out > > IP4: 02:fe:6c:5a:64:34 -> a6:d0:9d:34:bf:36 > > ICMP: 10.10.1.1 -> 10.10.2.1 > > tos 0x00, ttl 62, length 84, checksum 0xb8b7 dscp CS0 ecn NON_ECN > > fragment id 0x6cdc, flags DONT_FRAGMENT > > ICMP echo_request checksum 0xd748 > > > > > > ------------------------------ > xingyep...@163.com > > > *From:* Vijay Kumar <vjkumar2...@gmail.com> > *Date:* 2021-03-23 11:47 > *To:* Neale Ranns <ne...@graphiant.com> > *CC:* vpp-dev <vpp-dev@lists.fd.io> > *Subject:* Re: [vpp-dev] GRE-over-IPSec fails > Hi Neale, > > Could you let me know if you faced the mentioned problem anytime? > > For me only IPSec works fine, Only GRE also works fine. But when I > configure GRE-over-IPSec, the traffic is dropped at *esp4-decrypt-tun* > due to integrity check failure. > As there are two logical interfaces created at VPP (ipip0 and gre0) for > the peer, do I need to take care of something specially? As far as I know, > I haven't missed any config. > > > Regards, > Vijay Kumar N > > On Mon, Mar 22, 2021 at 11:31 PM Vijay Kumar via lists.fd.io <vjkumar2003= > gmail....@lists.fd.io> wrote: > >> Hi, >> >> I am trying a test case where-in I have an GRE P2MP (mGRE) tunnel on the >> VPP. The GRE peer is a strongswan VM that hosts both the GRE tunnel and >> IPSec SA. When I started ping traffic from SS, the traffic is dropped at >> esp4-decrypt-tun graph node due to integrity check failure. >> >> Has any one tested GRE-over-IPSec recently? If so can you pls share me a >> working config. If not please review the below config and let me know if I >> missed something >> >> *NOTE: -* >> If I have run only GRE test case, traffic is fine (no IPSec enabled). If >> I have only IPSec configured but no GRE then also traffic is fine. >> >> I am facing this issue only when both GRE and IPSec are enabled at the >> same time. >> >> Topology and config at SS and VPP >> ============================== >> Strongswan VM (20.20.99.215, gre peer 2.2.2.1, loopback 7.7.7.7) >> <=============> VPP cluster (20.20.99.99, gre peer 2.2.2.2, loopback >> 8.8.8.8) >> IPSec SA Traffic Selector (7.7.7.7/32 to 8.8.8.8/32) >> ike=aes256-sha256-modp2048! >> esp=aes256-sha1-noesn! >> >> >> Below is the VPP trace >> ================ >> 03:20:34:670201: dpdk-input >> VirtualFuncEthernet0/7/0 rx queue 0 >> buffer 0x4c6b91: current data 0, length 170, buffer-pool 0, ref-count >> 1, totlen-nifb 0, trace handle 0x1000000 >> ext-hdr-valid >> l4-cksum-computed l4-cksum-correct >> PKT MBUF: port 0, nb_segs 1, pkt_len 170 >> buf_len 2176, data_len 170, ol_flags 0x180, data_off 128, phys_addr >> 0xa3dae4c0 >> packet_type 0x691 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 >> rss 0x0 fdir.hi 0x0 fdir.lo 0x0 >> Packet Offload Flags >> PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid >> PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid >> Packet Types >> RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet >> RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without >> extension headers >> RTE_PTYPE_L4_NONFRAG (0x0600) Non-fragmented IP packet >> IP4: fa:16:3e:4b:6b:42 -> fa:16:3e:c2:b4:f4 802.1q vlan 1556 >> IPSEC_ESP: 20.20.99.215 -> 20.20.99.99 >> tos 0x00, ttl 64, length 152, checksum 0x5b33 dscp CS0 ecn NON_ECN >> fragment id 0xef9e, flags DONT_FRAGMENT >> 03:20:34:670208: ethernet-input >> frame: flags 0x3, hw-if-index 3, sw-if-index 3 >> IP4: fa:16:3e:4b:6b:42 -> fa:16:3e:c2:b4:f4 802.1q vlan 1556 >> 03:20:34:670214: ip4-input >> IPSEC_ESP: 20.20.99.215 -> 20.20.99.99 >> tos 0x00, ttl 64, length 152, checksum 0x5b33 dscp CS0 ecn NON_ECN >> fragment id 0xef9e, flags DONT_FRAGMENT >> 03:20:34:670218: ip4-lookup >> fib 1 dpo-idx 21 flow hash: 0x00000000 >> IPSEC_ESP: 20.20.99.215 -> 20.20.99.99 >> tos 0x00, ttl 64, length 152, checksum 0x5b33 dscp CS0 ecn NON_ECN >> fragment id 0xef9e, flags DONT_FRAGMENT >> 03:20:34:670220: ip4-local >> IPSEC_ESP: 20.20.99.215 -> 20.20.99.99 >> tos 0x00, ttl 64, length 152, checksum 0x5b33 dscp CS0 ecn NON_ECN >> fragment id 0xef9e, flags DONT_FRAGMENT >> 03:20:34:670222: ipsec4-tun-input >> IPSec: remote:20.20.99.215 spi:305419897 (0x12345679) seq 40 sa 1 >> 03:20:34:670225: esp4-decrypt-tun >> esp: crypto aes-cbc-256 integrity sha1-96 pkt-seq 40 sa-seq 0 sa-seq-hi >> 0 >> 03:20:34:670241: ip4-drop >> IP6_NONXT: 242.163.36.86 -> 70.168.225.19 >> version 1, header length 8 >> tos 0x34, ttl 245, length 22137, checksum 0x5156 (should be 0x972a) >> dscp unknown ecn NON_ECN >> fragment id 0x0000 offset 320 >> 03:20:34:670243: error-drop >> rx:ipip0 >> 03:20:34:670244: drop >> esp4-decrypt-tun: Integrity check failed >> >> >> vpp# show node counters >> Count Node Reason >> 25 esp4-encrypt-tun ESP pkts received >> 213 memif-input not ip packet >> 3 dpdk-input no error >> 136 arp-reply ARP replies sent >> 3 arp-reply IP4 source address not >> local to subnet >> 1 gre4-input no error >> 213 ip4-udp-lookup No error >> 42 esp4-decrypt-tun ESP pkts received >> 42 esp4-decrypt-tun Integrity check failed >> 25 esp4-encrypt-tun ESP pkts received >> 42 ipsec4-tun-input good packets received >> 11 ip4-local ip4 source lookup miss >> 3 ip4-local unknown ip protocol >> 3 ethernet-input unknown vlan >> vpp# >> >> >> >> >> >> >>
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18996): https://lists.fd.io/g/vpp-dev/message/18996 Mute This Topic: https://lists.fd.io/mt/81531694/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
