Hoi, Does the same behavior happen with linux_cp_plugin and linux_nl_plugin enabled instead? I saw a change in linux_cp that will walk the fib and remove routes associated with down interfaces; and I think this is because FRR does not remove them while Bird does. Can you try with linux_cp/linux_nl please?
Relevant reading: https://gerrit.fd.io/r/c/vpp/+/35529 https://gerrit.fd.io/r/c/vpp/+/35530 https://gerrit.fd.io/r/c/vpp/+/35531 groet, Pim On Sun, Mar 26, 2023 at 5:33 PM Christopher Adigun <future...@gmail.com> wrote: > > *Hi,* > > *I am facing an issue with the LCP route update when a particular > interface changes state (i.e. when I manually shut it down and when I bring > it up).* > > *Before shutting interface down, below is the state both in FRR and Linux > table:* > > *FRR table:* > > ingress-node-vpp-58dcb69b5f-g9rzp:/# vtysh > > Hello, this is FRRouting (version 8.5_git). > Copyright 1996-2005 Kunihiro Ishiguro, et al. > > ingress-node-vpp-58dcb69b5f-g9rzp# show ip route > Codes: K - kernel route, C - connected, S - static, R - RIP, > O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, > T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR, > f - OpenFabric, > > - selected route, * - FIB route, q - queued, r - rejected, b - > backup > t - trapped, o - offload failure > > K>* 0.0.0.0/0 [0/0] via 169.254.1.1, eth0, 00:29:12 > C>* 10.0.0.142/32 is directly connected, eth0, 00:29:12 > O 10.0.5.0/24 [110/10] is directly connected, dpdk0, weight 1, 00:29:12 > C>* 10.0.5.0/24 is directly connected, dpdk0, 00:29:12 > > *O>* 10.0.9.0/24 <http://10.0.9.0/24> [110/40] via 192.168.100.2, gre0, > weight 1, 00:24:19 * via 192.168.100.6, gre1, weight > 1, 00:24:19* > K>* 169.254.1.1/32 [0/0] is directly connected, eth0, 00:29:12 > O 192.168.100.0/30 [110/10] is directly connected, gre0, weight 1, > 00:24:31 > C>* 192.168.100.0/30 is directly connected, gre0, 00:24:31 > O 192.168.100.4/30 [110/10] is directly connected, gre1, weight 1, > 00:24:33 > C>* 192.168.100.4/30 is directly connected, gre1, 00:24:33 > O>* 192.168.100.8/30 [110/20] via 192.168.100.2, gre0, weight 1, 00:24:19 > O>* 192.168.100.12/30 [110/20] via 192.168.100.6, gre1, weight 1, 00:24:19 > O>* 192.168.100.16/30 [110/30] via 192.168.100.2, gre0, weight 1, 00:24:19 > O>* 192.168.100.20/30 [110/30] via 192.168.100.6, gre1, weight 1, 00:24:19 > > *Linux Table:* > ingress-node-vpp-58dcb69b5f-g9rzp:/# ip r > default via 169.254.1.1 dev eth0 > 10.0.5.0/24 dev dpdk0 proto kernel scope link src 10.0.5.16 > > > *10.0.9.0/24 <http://10.0.9.0/24> nhid 22 proto ospf metric 20 > nexthop via 192.168.100.2 dev gre0 weight 1 nexthop via > 192.168.100.6 dev gre1 weight 1* > 169.254.1.1 dev eth0 scope link > 192.168.100.0/30 dev gre0 proto kernel scope link src 192.168.100.1 > 192.168.100.4/30 dev gre1 proto kernel scope link src 192.168.100.5 > 192.168.100.8/30 nhid 18 via 192.168.100.2 dev gre0 proto ospf metric 20 > 192.168.100.12/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > 192.168.100.16/30 nhid 18 via 192.168.100.2 dev gre0 proto ospf metric 20 > 192.168.100.20/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > > *When I manually shut gre0 down, it is removed from the nexthop:* > > ingress-node-vpp-58dcb69b5f-g9rzp:/# vtysh > > Hello, this is FRRouting (version 8.5_git). > Copyright 1996-2005 Kunihiro Ishiguro, et al. > > ingress-node-vpp-58dcb69b5f-g9rzp# > ingress-node-vpp-58dcb69b5f-g9rzp# > ingress-node-vpp-58dcb69b5f-g9rzp# show ip route > Codes: K - kernel route, C - connected, S - static, R - RIP, > O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, > T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR, > f - OpenFabric, > > - selected route, * - FIB route, q - queued, r - rejected, b - > backup > t - trapped, o - offload failure > > K>* 0.0.0.0/0 [0/0] via 169.254.1.1, eth0, 00:37:00 > C>* 10.0.0.142/32 is directly connected, eth0, 00:37:00 > O 10.0.5.0/24 [110/10] is directly connected, dpdk0, weight 1, 00:37:00 > C>* 10.0.5.0/24 is directly connected, dpdk0, 00:37:00 > *O>* 10.0.9.0/24 <http://10.0.9.0/24> [110/40] via 192.168.100.6, gre1, > weight 1, 00:00:06* > K>* 169.254.1.1/32 [0/0] is directly connected, eth0, 00:37:00 > O>* 192.168.100.0/30 [110/60] via 192.168.100.6, gre1, weight 1, 00:00:06 > O 192.168.100.4/30 [110/10] is directly connected, gre1, weight 1, > 00:32:21 > C>* 192.168.100.4/30 is directly connected, gre1, 00:32:21 > O>* 192.168.100.8/30 [110/50] via 192.168.100.6, gre1, weight 1, 00:00:06 > O>* 192.168.100.12/30 [110/20] via 192.168.100.6, gre1, weight 1, 00:32:07 > O>* 192.168.100.16/30 [110/40] via 192.168.100.6, gre1, weight 1, 00:00:06 > O>* 192.168.100.20/30 [110/30] via 192.168.100.6, gre1, weight 1, 00:32:07 > > ingress-node-vpp-58dcb69b5f-g9rzp:/# ip link set dev gre0 down > ingress-node-vpp-58dcb69b5f-g9rzp:/# > ingress-node-vpp-58dcb69b5f-g9rzp:/# > ingress-node-vpp-58dcb69b5f-g9rzp:/# > ingress-node-vpp-58dcb69b5f-g9rzp:/# ip r > default via 169.254.1.1 dev eth0 > 10.0.5.0/24 dev dpdk0 proto kernel scope link src 10.0.5.16 > *10.0.9.0/24 <http://10.0.9.0/24> nhid 23 via 192.168.100.6 dev gre1 proto > ospf metric 20* > 169.254.1.1 dev eth0 scope link > 192.168.100.0/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > 192.168.100.4/30 dev gre1 proto kernel scope link src 192.168.100.5 > 192.168.100.8/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > 192.168.100.12/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > 192.168.100.16/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > 192.168.100.20/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > > *But when I bring it up again, FRR shows that both gre0 and gre1 are > nexthops but Linux table (also in VPP fib) still shows only gre1:* > > ingress-node-vpp-58dcb69b5f-g9rzp:/# ip link set dev gre0 up > ingress-node-vpp-58dcb69b5f-g9rzp:/# > ingress-node-vpp-58dcb69b5f-g9rzp:/# > ingress-node-vpp-58dcb69b5f-g9rzp:/# > ingress-node-vpp-58dcb69b5f-g9rzp:/# > ingress-node-vpp-58dcb69b5f-g9rzp:/# ip r > default via 169.254.1.1 dev eth0 > 10.0.5.0/24 dev dpdk0 proto kernel scope link src 10.0.5.16 > *10.0.9.0/24 <http://10.0.9.0/24> nhid 23 via 192.168.100.6 dev gre1 proto > ospf metric 20* > 169.254.1.1 dev eth0 scope link > 192.168.100.0/30 dev gre0 proto kernel scope link src 192.168.100.1 > 192.168.100.4/30 dev gre1 proto kernel scope link src 192.168.100.5 > 192.168.100.8/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > 192.168.100.12/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > 192.168.100.16/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > 192.168.100.20/30 nhid 23 via 192.168.100.6 dev gre1 proto ospf metric 20 > ingress-node-vpp-58dcb69b5f-g9rzp:/# > ingress-node-vpp-58dcb69b5f-g9rzp:/# > ingress-node-vpp-58dcb69b5f-g9rzp:/# vtysh > > Hello, this is FRRouting (version 8.5_git). > Copyright 1996-2005 Kunihiro Ishiguro, et al. > > ingress-node-vpp-58dcb69b5f-g9rzp# > ingress-node-vpp-58dcb69b5f-g9rzp# > ingress-node-vpp-58dcb69b5f-g9rzp# show ip route > Codes: K - kernel route, C - connected, S - static, R - RIP, > O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, > T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR, > f - OpenFabric, > > - selected route, * - FIB route, q - queued, r - rejected, b - > backup > t - trapped, o - offload failure > > K>* 0.0.0.0/0 [0/0] via 169.254.1.1, eth0, 00:34:25 > C>* 10.0.0.142/32 is directly connected, eth0, 00:34:25 > O 10.0.5.0/24 [110/10] is directly connected, dpdk0, weight 1, 00:34:25 > C>* 10.0.5.0/24 is directly connected, dpdk0, 00:34:25 > > *O>* 10.0.9.0/24 <http://10.0.9.0/24> [110/40] via 192.168.100.2, gre0, > weight 1, 00:00:17 * via 192.168.100.6, gre1, weight > 1, 00:00:17* > K>* 169.254.1.1/32 [0/0] is directly connected, eth0, 00:34:25 > O 192.168.100.0/30 [110/10] is directly connected, gre0, weight 1, > 00:00:24 > C>* 192.168.100.0/30 is directly connected, gre0, 00:00:24 > O 192.168.100.4/30 [110/10] is directly connected, gre1, weight 1, > 00:29:46 > C>* 192.168.100.4/30 is directly connected, gre1, 00:29:46 > O>* 192.168.100.8/30 [110/20] via 192.168.100.2, gre0, weight 1, 00:00:17 > O>* 192.168.100.12/30 [110/20] via 192.168.100.6, gre1, weight 1, 00:29:32 > O>* 192.168.100.16/30 [110/30] via 192.168.100.2, gre0, weight 1, 00:00:17 > O>* 192.168.100.20/30 [110/30] via 192.168.100.6, gre1, weight 1, 00:29:32 > > *startup.conf:* > > unix { > nodaemon > log /tmp/vpp.log > full-coredump > gid vpp > interactive > cli-listen /run/vpp/cli.sock > exec /etc/vpp/init.conf > } > cpu { > main-core CORE1 > corelist-workers CORE2 > } > memory { > main-heap-size 2G > } > api-trace { > on > } > buffers { > buffers-per-numa 128000 > default data-size 2048 > page-size default-hugepage > } > dpdk { > dev 0000:00:06.0 {name dpdk0} > dev 0000:00:07.0 {name dpdk1} > } > api-segment { > gid vpp > } > plugins { > path /usr/lib/x86_64-linux-gnu/vpp_plugins/ > plugin ping_plugin.so { disable } > plugin linux_cp_plugin.so { disable } > plugin linux_nl_plugin.so { disable } > plugin lcpng_if_plugin.so { enable } > plugin lcpng_nl_plugin.so { enable } > plugin dpdk_plugin.so { enable } > } > lcpng { > lcp-sync > lcp-auto-subint > } > logging { > default-log-level info > default-syslog-log-level crit > class linux-cp/if { rate-limit 10000 level debug syslog-level debug } > class linux-cp/nl { rate-limit 10000 level debug syslog-level debug } > } > > *Init conf:* > > create gre tunnel src 10.0.6.16 dst 10.0.6.17 instance 0 > create gre tunnel src 10.0.6.16 dst 10.0.6.18 instance 1 > lcp create gre0 host-if gre0 tun > lcp create gre1 host-if gre1 tun > lcp create dpdk0 host-if dpdk0 > set interface ip address gre0 192.168.100.1/30 > set interface ip address gre1 192.168.100.5/30 > set interface state gre0 up > set interface state gre1 up > set interface mtu 9001 dpdk0 > set interface mtu 9001 dpdk1 > set interface ip address dpdk0 10.0.5.16/24 > set interface ip address dpdk1 10.0.6.16/24 > set interface state dpdk0 up > set interface state dpdk1 up > > What could be wrong? > > Thanks > > > > > > -- Pim van Pelt <p...@ipng.nl> PBVP1-RIPE - http://www.ipng.nl/
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#22766): https://lists.fd.io/g/vpp-dev/message/22766 Mute This Topic: https://lists.fd.io/mt/97863718/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
