[apologies in advance for top-posting]
bgpd(8) is excellent in many ways, and I am SO very grateful it exists. (Thank
you Henning, Claudio, Peter and everyone else who has contributed to it over
the years! It has straight-up saved my bacon a couple of times.)
But one feature it does not yet AFAIK have is Additional Paths
("additional-path", or just "add-path") [1]. This is where a BGP speaker
advertises not only its "best" routes, but *all* its routes, to its peers. The
FIB remains unchanged, but the RIB can grow very large in a well-connected AS.
Since each router now knows all the available paths through every other router,
convergence is - at least in theory - sped up quite dramatically, and you
mostly avoid the "hole" described. It's not a perfect solution but it works
very well.
If you're brave enough, at least some versions/forks of Bird/Quagga/Zebra have
support for add-path. I wouldn't recommend running these on OpenBSD, generally
speaking - bgpd(8) is more appropriate 99.999% of the time - but you might find
it worthwhile, depending on your needs. Be particularly careful of any routing
software that lacks the ability to affect kernel routes - unless you're just
running a route reflector, that will change your design *significantly*.
Or, as Stuart said, running a "proper" IGP like OSPF could bridge some of the
gaps you might see. YMMV.
-Adam
P.S. From what I heard a few years ago, OpenBSD isn't completely ignoring
add-path, it's just
complex/difficult/time-consuming/unfunded/low-priority/pick-your-favourite-reason.
[1] https://datatracker.ietf.org/doc/html/rfc7911
-----Original Message-----
From: owner-m...@openbsd.org <owner-m...@openbsd.org> On Behalf Of Sebastian
Benoit
Sent: Monday, November 29, 2021 3:38 PM
To: misc@openbsd.org
Subject: Re: bgpd, announce to ibgp from 2 routers, prefixes only show up from 1
Stuart Henderson(s...@spacehopper.org) on 2021.11.13 00:11:08 +0000:
> I have a pair of -current routers running bgpd (let's call them rtr-a
> and rtr-b) on a subnet which also has some vpn gateways and firewalls.
>
> These routers provide a carp address which the vpn gateways are using
> as default route. There are some networks behind the vpn gateways (a
> /32 to accept incoming vpn connections and some other prefixes that
> vpn clients are numbered from).
>
> rtr-a and rtr-b have static routes to those networks, and they have
> network statements in bgpd.conf to announce them to their ibgp peers
> ("network 172.24.232.0/21 set nexthop XXX" etc) so the paths are
> reachable from the rest of the network. (This is replacing an existing
> setup using ospf, trying to remove routing protocols from machines
> that don't really need them).
>
> It is working but something seems a little odd - the paths are
> announced from both routers briefly and show up on the rest of the
> network from both rtr-a and rtr-b. But after a few seconds, rtr-b
> receives these paths from rtr-a, and then rtr-b stops announcing them
> itself. (they stop showing in "bgpctl sh rib out" on rtr-b; "bgpctl sh
> nex" does correctly identify the associated nexthops as connected/UP).
>
> Is this expected/correct behaviour?
It is expected: once rtr-b receives the route from rtr-a, it will run the route
decision process on it. IF both routers are configured identically except for
the router-id, one of the routes will be prefered at either the "oldest path"
or the "lowest bgp id" criteria.
As only one route is a best route, that one will be annouced to the neighbors.
However this is IBGP. In a set of IBGP connected routers, a router will not
announce a route to other IBGP peers that it received from on a IBGP session.
Thus, rtr-b will stop announcing that route.
When rtr-a goes down, the session is shut down or the prefix is filtered, bgpd
wont see the "better" route anymore and announce its own instead.
> I'd prefer to have them announced from both rtr-a and rtr-b, so
> there's no blackhole period if rtr-a is restarted while rtr-b figures
> out that it should start announcing them, etc. (No need for tracking
> carp state in this case, I'm not using stateful pf rules on the traffic
> involved).
This is a place where ospf might give you faster failover, especiall y with the
redistribute ... depend on ... syntax.
> If rtr-b stops seeing the prefixes from rtr-a (either by taking down
> the ibgp session, or by filtering) I see the announcements from both
> rtr-a and rtr-b again. So the obvious workaround is to filter, but I
> thought I'd ask first in case it's something that is better handled by
> code changes rather than config.
