On Tue, Dec 14, 2021 at 08:35:48PM +0100, Nico Schottelius wrote: > > Hello, > > every year or then bird is putting me into the Invalid NEXT_HOP > message.
Hello Yes, this is kind of confusing error message (as i noted in response to Simon Ruderich). > TL;DR: > Why does bird on router1+router2 refuse the route > 2a0a:e5c0:0:12:b01a:5ae3:1bd4:1e00/122 via > 2a0a:e5c0::225:90ff:fe1e:3e62 > > even though router1+router2 know how to reach 2a0a:e5c0::/64 via > fe80::20d:b9ff:fe57:2f91 by means of ospf? Note that in gateway-recursive mode, the fact of whether reachability is known is not relevant for 'Invalid NEXT_HOP' message. If it is unreachable, then the route would be accepted as 'unreachable' and changes to reachable if reachability is changed. It is usually some other issue like zero or local IP address. > In detail: > > router1, router2 are peered to apu-router1,apu-router2 via OSPF + BGP. > > apu-router1,apu-router2 are peered to a set of kubernetes hosts. > > The goal is to have router1 + router2 import the routes sent by the > kubernetes hosts: > > router1 router2---------| > | \ | | > | \ | | > | \ | | > apu-router1 apu-router2 | > . | . | > |-------------------------- > . . > [ kubernetes cluster via apu-routers ] > > > The problem: router1+router reject the routes with: > > Dec 14 20:33:51 router1 daemon.err bird: > apu_router1_place5_ungleich_ch_v6: Invalid NEXT_HOP attribute > > The setup: > > router1, router2, apu-router1, apu-router2 = ASN209898 > kubernetes hosts = ASN65533 > kubernetes peers with apu-routers only. So the BGP link between kubernetes and APU-ROUTER is EBGP, while between APU-ROUTER and ROUTER is IBGP? I expect it is in multihop / gateway-recursive mode, as it is default for IBGP? > The routes: > Kubernetes announces parts of 2a0a:e5c0:0:12::/64 and > 2a0a:e5c0:0:13::/64, for instance the route > 2a0a:e5c0:0:12:b01a:5ae3:1bd4:1e00/122. > > Kubernetes nodes live in 2a0a:e5c0::/64. > > apu-routers have a leg in 2a0a:e5c0::/64, via eth1.2. They reach the > cluster directly. They have the routes. > > routers1+2 receive the route for 2a0a:e5c0::/64 via ospf: > > bird> show route 2a0a:e5c0::/64 > Table master6: > 2a0a:e5c0::/64 unicast [ospf6 17:08:18.515] * I (150/20) [0.0.0.47] > via fe80::20d:b9ff:fe57:2f91 on bond0.8 > The apu-routers: > - They import the route [0] Could you show 'show route all' on apu-router for failed routes to see their BGP_NEXT_HOP attribute? And also ideally tcpdump output to see BGP_NEXT_HOP as sent from apu-router to router? > - They export the route to the routers [1] Could you also show 'show protocol all' for the session from apu-router to the kubernetes hosts? > The routers: > - print 4x the Invalid NEXT_HOP attribute, once per exported > kubernetes network > - They ignore the 4 routes [2] > > Question: why does bird on the routers not accept the routes? Or is > there a different problem I am not seeing? Aside from that, shouldn't > bird on the apu-routers set itself as nexthop for the kubernetes routes? Not, because sending it to routerx over IBGP link, where BGP_NEXT_HOP is kept unmodified by default (unless 'next hop self' option is used). It might be an issue related to IPv6 dual next-hops (global and link-local), where global is empty. -- Elen sila lumenn' omentielvo Ondrej 'Santiago' Zajicek (email: [email protected]) OpenPGP encrypted e-mails preferred (KeyID 0x11DEADC3, wwwkeys.pgp.net) "To err is human -- to blame it on a computer is even more so."
