From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> on behalf of Benoit Ganne (bganne) via lists.fd.io <bganne=cisco....@lists.fd.io> Date: Thursday, 1 July 2021 at 10:38 To: Mechthild Buescher <mechthild.buesc...@ericsson.com>, vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] next-hop-table between two FIB tables results in punt and 'unknown ip protocol' I think the issue is the way you populate the route: ip route add 198.19.255.248/29 table 1 via 198.19.255.249 next-hop-table 4093
As 198.19.255.249 is the IP of host-Vpp2Host.4093, VPP interprets it as you want to deliver the packet locally instead of forwarding it. Try changing it to: ip route add 198.19.255.248/29 table 1 via 0.0.0.0 next-hop-table 4093 0.0.0.0/32 in any table is a drop. One cannot specify a route to be recursive via another network, i.e. via 0.0.0.0/0, VPP doesn’t support that. If the semantics you’re after is ‘do a second lookup in table 4093’ then you want ‘via ip4-lookup-in-table 4093’. But as Neale pointed out, if you only have a few routes in 4093, it would be even better to just do: ip route add 198.19.255.248/29 table 1 via host-Vpp2Host.4093 I suspect if you don’t do it this way, then a ping from another device in the 198.19.255.248/29 subnet won’t work, because the return path is not in table 1. /neale That would save you 1 additional fib lookup. Best ben > -----Original Message----- > From: Mechthild Buescher <mechthild.buesc...@ericsson.com> > Sent: jeudi 1 juillet 2021 09:39 > To: Benoit Ganne (bganne) <bga...@cisco.com>; vpp-dev@lists.fd.io > Subject: RE: next-hop-table between two FIB tables results in punt and > 'unknown ip protocol' > > Hi Ben, > > Sorry, I sent the wrong output for the fib table - with the 'next-hop- > table' configuration it looks as follows: > ipv4-VRF:0, fib_index:0, flow hash:[src dst sport dport proto flowlabel ] > epoch:0 flags:none locks:[default-route:1, ] > 0.0.0.0/0 fib:0 index:0 locks:2 > default-route refs:1 entry-flags:drop, src- > flags:added,contributing,active, > path-list:[0] locks:2 flags:drop, uPRF-list:0 len:0 itfs:[] > path:[0] pl-index:0 ip4 weight=1 pref=0 special: cfg-flags:drop, > [@0]: dpo-drop ip4 > > forwarding: unicast-ip4-chain > [@0]: dpo-load-balance: [proto:ip4 index:1 buckets:1 uRPF:0 to:[0:0]] > [0] [@0]: dpo-drop ip4 > : > ipv4-VRF:4093, fib_index:3, flow hash:[src dst sport dport proto flowlabel > ] epoch:0 flags:none locks:[CLI:2, adjacency:1, recursive-resolution:1, ] > 198.19.255.253/32 fib:3 index:170 locks:2 > adjacency refs:1 entry-flags:attached, src- > flags:added,contributing,active, cover:53 > path-list:[188] locks:2 uPRF-list:191 len:1 itfs:[14, ] > path:[224] pl-index:188 ip4 weight=1 pref=0 attached-nexthop: oper- > flags:resolved, > 198.19.255.253 host-Vpp2Host.4093 > [@0]: ipv4 via 198.19.255.253 host-Vpp2Host.4093: mtu:9000 next:4 > flags:[] a215f39524f302fee89a0ec381000ffd0800 > Extensions: > path:224 adj-flags:[refines-cover] > forwarding: unicast-ip4-chain > [@0]: dpo-load-balance: [proto:ip4 index:171 buckets:1 uRPF:191 > to:[4:384]] > [0] [@5]: ipv4 via 198.19.255.253 host-Vpp2Host.4093: mtu:9000 next:4 > flags:[] a215f39524f302fee89a0ec381000ffd0800 > ipv4-VRF:1, fib_index:4, flow hash:[src dst sport dport proto flowlabel ] > epoch:0 flags:none locks:[API:1, CLI:2, adjacency:1, recursive- > resolution:1, ] > 198.19.255.248/29 fib:4 index:66 locks:2 > CLI refs:1 src-flags:added,contributing,active, > path-list:[75] locks:20 flags:shared, uPRF-list:75 len:0 itfs:[] > path:[89] pl-index:75 ip4 weight=1 pref=0 recursive: oper- > flags:resolved, > via 198.19.255.249 in fib:3 via-fib:56 via-dpo:[dpo-load- > balance:57] > > forwarding: unicast-ip4-chain > [@0]: dpo-load-balance: [proto:ip4 index:67 buckets:1 uRPF:75 to:[0:0]] > [0] [@11]: dpo-load-balance: [proto:ip4 index:57 buckets:1 uRPF:65 > to:[4:384]] > [0] [@2]: dpo-receive: 198.19.255.249 on host-Vpp2Host.4093 > ipv4-VRF:10, fib_index:5, flow hash:[src dst sport dport proto flowlabel ] > epoch:0 flags:none locks:[CLI:2, ] > > > The output below is when we replaced the 'next-hop table' routing with > direct routing to /32 Ips. That direct routing is working but is regarded > as work-around as long as we don't find a better solution. > > BR/Mechthild > > -----Original Message----- > From: Mechthild Buescher > Sent: Wednesday, 30 June 2021 18:32 > To: Benoit Ganne (bganne) <bga...@cisco.com>; vpp-dev@lists.fd.io > Subject: RE: next-hop-table between two FIB tables results in punt and > 'unknown ip protocol' > > Hi Ben, > > Thanks for your fast reply. Here is the requested output (I skipped config > for other interfaces and VLANs) > > vppctl show int addr > NCIC-1-v1 (up): > NCIC-1-v1.1 (up): > L3 10.10.203.1/29 ip4 table-id 1 fib-idx 4 host-Vpp2Host (up): > host-Vpp2Host.4093 (up): > L3 198.19.255.249/29 ip4 table-id 4093 fib-idx 3 > local0 (dn): > > and: > vppctl sh ip fib 198.19.255.253 > pv4-VRF:0, fib_index:0, flow hash:[src dst sport dport proto flowlabel ] > epoch:0 flags:none locks:[default-route:1, ] > 0.0.0.0/0 fib:0 index:0 locks:2 > default-route refs:1 entry-flags:drop, src- > flags:added,contributing,active, > path-list:[0] locks:2 flags:drop, uPRF-list:0 len:0 itfs:[] > path:[0] pl-index:0 ip4 weight=1 pref=0 special: cfg-flags:drop, > [@0]: dpo-drop ip4 > > forwarding: unicast-ip4-chain > [@0]: dpo-load-balance: [proto:ip4 index:1 buckets:1 uRPF:0 to:[0:0]] > [0] [@0]: dpo-drop ip4 > ipv4-VRF:4093, fib_index:3, flow hash:[src dst sport dport proto flowlabel > ] epoch:0 flags:none locks:[CLI:2, adjacency:1, recursive-resolution:1, ] > 198.19.255.253/32 fib:3 index:189 locks:2 > adjacency refs:1 entry-flags:attached, src- > flags:added,contributing,active, cover:53 > path-list:[206] locks:2 uPRF-list:210 len:1 itfs:[14, ] > path:[241] pl-index:206 ip4 weight=1 pref=0 attached-nexthop: oper- > flags:resolved, > 198.19.255.253 host-Vpp2Host.4093 > [@0]: ipv4 via 198.19.255.253 host-Vpp2Host.4093: mtu:9000 next:4 > flags:[] e2de891fcccb02fe33d4dc0d81000ffd0800 > Extensions: > path:241 adj-flags:[refines-cover] > forwarding: unicast-ip4-chain > [@0]: dpo-load-balance: [proto:ip4 index:190 buckets:1 uRPF:210 > to:[2:192] via:[6:504]] > [0] [@5]: ipv4 via 198.19.255.253 host-Vpp2Host.4093: mtu:9000 next:4 > flags:[] e2de891fcccb02fe33d4dc0d81000ffd0800 > ipv4-VRF:1, fib_index:4, flow hash:[src dst sport dport proto flowlabel ] > epoch:0 flags:none locks:[API:1, CLI:2, adjacency:1, recursive- > resolution:1, ] > 198.19.255.253/32 fib:4 index:190 locks:2 > CLI refs:1 entry-flags:attached, src-flags:added,contributing,active, > path-list:[207] locks:2 flags:shared, uPRF-list:211 len:1 itfs:[14, ] > path:[243] pl-index:207 ip4 weight=1 pref=0 attached-nexthop: oper- > flags:resolved, cfg-flags:attached, > 198.19.255.253 host-Vpp2Host.4093 > [@0]: ipv4 via 198.19.255.253 host-Vpp2Host.4093: mtu:9000 next:4 > flags:[] e2de891fcccb02fe33d4dc0d81000ffd0800 > > forwarding: unicast-ip4-chain > [@0]: dpo-load-balance: [proto:ip4 index:191 buckets:1 uRPF:211 > to:[5:480]] > [0] [@5]: ipv4 via 198.19.255.253 host-Vpp2Host.4093: mtu:9000 next:4 > flags:[] e2de891fcccb02fe33d4dc0d81000ffd0800 > > BR/Mechthild > > -----Original Message----- > From: Benoit Ganne (bganne) <bga...@cisco.com> > Sent: Wednesday, 30 June 2021 18:16 > To: Mechthild Buescher <mechthild.buesc...@ericsson.com>; vpp- > d...@lists.fd.io > Subject: RE: next-hop-table between two FIB tables results in punt and > 'unknown ip protocol' > > From the trace output, it looks like VPP thinks 198.19.255.253 is one of > its interface address, and hence try to deliver it locally. As there is no > configured listener for TCP packets, it default to punting, and as there > is no punt rule it drops. > > Can you share the output of 'show int addr' and 'sh ip fib > 198.19.255.253'? > > Best > ben > > > -----Original Message----- > > From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Mechthild > > Buescher via lists.fd.io > > Sent: mercredi 30 juin 2021 18:06 > > To: vpp-dev@lists.fd.io > > Subject: [vpp-dev] next-hop-table between two FIB tables results in > > punt and 'unknown ip protocol' > > > > Hi all, > > > > > > > > we are using VPP with several FIB tables and when we use 'next-hop- > table' > > the ip4-lookup results somehow in 'unknown ip protocol'. Can you > > please help? > > > > > > > > Our setup: > > > > * 1 (out of 2) with VPP and a DPDK interface > > > > > > > > The VPP version is (both nodes): > > > > > > vpp# show version verbose > > > > Version: v21.06-rc2~6-gf377e9545 > > > > Compiled by: suse > > > > Compile host: SUSE > > > > Compile date: 2021-06-24T14:02:01 > > > > Compile location: /root/vpp-32298/vpp > > > > Compiler: GCC 7.5.0 > > > > Current PID: 22527 > > > > > > > > The VPP config uses the DPDK interface (both nodes): > > > > > > vpp# show hardware-interfaces NCIC-1-v1 > > > > Name Idx Link Hardware > > > > NCIC-1-v1 3 up NCIC-1-v1 > > > > Link speed: 40 Gbps > > > > RX Queues: > > > > queue thread mode > > > > 0 vpp_wk_0 (1) polling > > > > Ethernet address 72:a6:1e:ae:cd:f1 > > > > Intel iAVF > > > > carrier up full duplex mtu 9206 > > > > flags: admin-up pmd maybe-multiseg subif tx-offload > > intel-phdr-cksum rx-ip4-cksum int-unmaskable > > > > Devargs: > > > > rx: queues 1 (max 256), desc 1024 (min 64 max 4096 align 32) > > > > tx: queues 3 (max 256), desc 1024 (min 64 max 4096 align 32) > > > > pci: device 8086:154c subsystem 1028:0000 address 0000:17:0e.01 > > numa 0 > > > > max rx packet len: 9728 > > > > promiscuous: unicast off all-multicast on > > > > vlan offload: strip off filter off qinq off > > > > rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum qinq- > > strip > > > > outer-ipv4-cksum vlan-filter jumbo-frame > > scatter rss-hash > > > > rx offload active: ipv4-cksum jumbo-frame scatter > > > > tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum > > sctp- cksum > > > > tcp-tso outer-ipv4-cksum qinq-insert > > vxlan-tnl-tso > > > > gre-tnl-tso ipip-tnl-tso geneve-tnl-tso > > multi-segs > > > > mbuf-fast-free > > > > tx offload active: udp-cksum tcp-cksum multi-segs > > > > rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-sctp ipv4-other > > ipv4 > > > > ipv6-frag ipv6-tcp ipv6-udp ipv6-sctp > > ipv6-other > > ipv6 > > > > rss active: none > > > > tx burst function: iavf_xmit_pkts > > > > rx burst function: iavf_recv_scattered_pkts_vec_avx2 > > > > > > > > The VPP config is (there is a veth-pair configured on the host): > > > > > > > > create host-interface name Vpp2Host > > > > set interface state host-Vpp2Host up > > > > > > > > ip table add 4093 > > > > create sub-interfaces host-Vpp2Host 4093 > > > > set interface state host-Vpp2Host.4093 up > > > > set interface ip table host-Vpp2Host.4093 4093 > > > > set interface ip address host-Vpp2Host.4093 198.19.255.249/29 > > > > > > > > set interface state NCIC-1-v1 up > > > > ip table add 1 > > > > create sub-interfaces NCIC-1-v1 1 > > > > set interface state NCIC-1-v1.1 up > > > > set interface ip table NCIC-1-v1.1 1 > > > > set interface ip address NCIC-1-v1.1 10.10.203.3/29 > > > > ip route add 198.19.255.248/29 table 1 via 198.19.255.249 > > next-hop-table > > 4093 > > > > > > > > When a packet is received (curl -k -vv > > https://protect2.fireeye.com/v1/url?k=00957dfa-5f0e44be-00953d61- > 861d41abace8-dc5e93fd2e4adcdf&q=1&e=532639d0-8e37-4c7a-83d3- > 552d3a0d02a9&u=https%3A%2F%2F198.19.255.253%3A8443%2F... ) we see the > following trace on dpdk-input: > > > > NCIC-1-v1 rx queue 0 > > > > buffer 0x14296: current data 0, length 78, buffer-pool 0, ref-count > > 1, totlen-nifb 0, trace handle 0x1000016 > > > > ext-hdr-valid > > > > l4-cksum-computed l4-cksum-correct > > > > PKT MBUF: port 2, nb_segs 1, pkt_len 78 > > > > buf_len 2176, data_len 78, ol_flags 0x180, data_off 128, phys_addr > > 0x50a600 > > > > packet_type 0x191 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > > > > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > > > > Packet Offload Flags > > > > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > > > > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid > > > > Packet Types > > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > without extension headers > > > > RTE_PTYPE_L4_TCP (0x0100) TCP packet > > > > IP4: 86:ca:f3:a1:20:fc -> 1e:a0:ab:00:2a:ea 802.1q vlan 1 > > > > TCP: 172.17.41.8 -> 198.19.255.253 > > > > tos 0x00, ttl 64, length 60, checksum 0x7bc3 dscp CS0 ecn NON_ECN > > > > fragment id 0x23ce, flags DONT_FRAGMENT > > > > TCP: 41834 -> 8443 > > > > seq. 0xbd8844b4 ack 0x00000000 > > > > flags 0x02 SYN, tcp header: 40 bytes > > > > window 29200, checksum 0xa704 > > > > 18:41:55:850222: ethernet-input > > > > frame: flags 0x3, hw-if-index 3, sw-if-index 3 > > > > IP4: 86:ca:f3:a1:20:fc -> 1e:a0:ab:00:2a:ea 802.1q vlan 1 > > > > 18:41:55:850224: ip4-input > > > > TCP: 172.17.41.8 -> 198.19.255.253 > > > > tos 0x00, ttl 64, length 60, checksum 0x7bc3 dscp CS0 ecn NON_ECN > > > > fragment id 0x23ce, flags DONT_FRAGMENT > > > > TCP: 41834 -> 8443 > > > > seq. 0xbd8844b4 ack 0x00000000 > > > > flags 0x02 SYN, tcp header: 40 bytes > > > > window 29200, checksum 0xa704 > > > > 18:41:55:850225: ip4-lookup > > > > fib 4 dpo-idx 57 flow hash: 0x00000000 > > > > TCP: 172.17.41.8 -> 198.19.255.253 > > > > tos 0x00, ttl 64, length 60, checksum 0x7bc3 dscp CS0 ecn NON_ECN > > > > fragment id 0x23ce, flags DONT_FRAGMENT > > > > TCP: 41834 -> 8443 > > > > seq. 0xbd8844b4 ack 0x00000000 > > > > flags 0x02 SYN, tcp header: 40 bytes > > > > window 29200, checksum 0xa704 > > > > 18:41:55:850226: ip4-load-balance > > > > fib 4 dpo-idx 18 flow hash: 0x00000000 > > > > TCP: 172.17.41.8 -> 198.19.255.253 > > > > tos 0x00, ttl 64, length 60, checksum 0x7bc3 dscp CS0 ecn NON_ECN > > > > fragment id 0x23ce, flags DONT_FRAGMENT > > > > TCP: 41834 -> 8443 > > > > seq. 0xbd8844b4 ack 0x00000000 > > > > flags 0x02 SYN, tcp header: 40 bytes > > > > window 29200, checksum 0xa704 > > > > 18:41:55:850228: ip4-local > > > > TCP: 172.17.41.8 -> 198.19.255.253 > > > > tos 0x00, ttl 64, length 60, checksum 0x7bc3 dscp CS0 ecn > > NON_ECN > > > > fragment id 0x23ce, flags DONT_FRAGMENT > > > > TCP: 41834 -> 8443 > > > > seq. 0xbd8844b4 ack 0x00000000 > > > > flags 0x02 SYN, tcp header: 40 bytes > > > > window 29200, checksum 0xa704 > > > > 18:41:55:850230: ip4-punt > > > > TCP: 172.17.41.8 -> 198.19.255.253 > > > > tos 0x00, ttl 64, length 60, checksum 0x7bc3 dscp CS0 ecn > > NON_ECN > > > > fragment id 0x23ce, flags DONT_FRAGMENT > > > > TCP: 41834 -> 8443 > > > > seq. 0xbd8844b4 ack 0x00000000 > > > > flags 0x02 SYN, tcp header: 40 bytes > > > > window 29200, checksum 0xa704 > > > > 18:41:55:850231: error-punt > > > > rx:NCIC-1-v1.1 > > > > 18:41:55:850232: punt > > > > ip4-local: unknown ip protocol > > > > > > > > And the following is the relevant part in the FIB table: > > vpp# show ip fib table 1 > > > > ipv4-VRF:1, fib_index:4, flow hash:[src dst sport dport proto ] > > epoch:0 flags:none locks:[API:1, CLI:2, adjacency:1, > > recursive-resolution:1, ] > > > > : > > > > 198.19.255.248/29 > > > > unicast-ip4-chain > > > > [@0]: dpo-load-balance: [proto:ip4 index:67 buckets:1 uRPF:75 > > to:[1447:116264]] > > > > [0] [@11]: dpo-load-balance: [proto:ip4 index:57 buckets:1 uRPF:65 > > to:[6:576] via:[1447:116264]] > > > > [0] [@2]: dpo-receive: 198.19.255.249 on host-Vpp2Host.4093 > > > > > > > > The uRPF seems to be empty: > > vpp# show fib uRPF > > > > 65@uPRF-list:65 len:0 itfs:[] > > > > > > > > A ping was working fine but the curl command failed. Is there some > > additional configuration needed to allow ports via 'next-hop-table' or > > is this a bug in VPP? > > > > > > > > Any hint on how to solve this is appreciated. > > > > > > > > Thank you and best regards, > > > > > > > > Mechthild Buescher > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19674): https://lists.fd.io/g/vpp-dev/message/19674 Mute This Topic: https://lists.fd.io/mt/83895986/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
