On Tue, May 05, 2015 at 02:47:28PM +0100, Paul Jakma wrote: > On Tue, 5 May 2015, David Lamparter wrote: > > > The R bit is intended to be based on wallclock time. While setting the > > R bit doesn't break anything per se, setting the R bit is not intended > > by the spec when bringing up a new session. > > No, *route selection* is meant to be deferred until the EoR comes in, or > the timer expires.
Route selection, and by way of not having any selected routes, all outbound route advertisement. > The RFC is quiet about when to unset the R-bit. There are a number of > reasonable interpretations on that. Using the timer is one, but I don't > think it's a good one, see below. "When set (value 1), this bit indicates that the BGP speaker has restarted" does imply that it's not intended to be set on a newly created session 500 days after starting the bgp speaker. > > In particular, if bringing up a new session from a "good" router > > (non-restarted recently) to a restarted router, setting the R bit will > > cause extraneous incoming update noise from the restarted router to the > > good router. > > It's the other way around. Not setting the R-bit when we have restarted is > what leads to extra noise and timer + runtime dependent behaviour (which > can vary). I don't follow. > If R-bit is *not* set, the other side will wait for EoR. > > > (But if the R-bit is set for new sessions, this will inadvertedly > > trigger the deadlock-avoidal mechanism and send all updates as they > > arrive, causing potentially many incoming updates for each individual > > net.) > > Right, which is what you want if you have in fact restarted. Funny that you agree, because my above paragraph is actually wrong. I should stop looking at BGP when having been awake for 20 hours :D Receiving the R bit set means that you shouldn't wait for any EoRs from that router, not that updates must be sent immediately or anything. It simply excludes the session from the set of sessions for which the set-and "all done" is evaluated. A restarting box will, while it's in startup mode (= for a limited time), look at the "all done?" boolean. That boolean moves to true when either: - the timer is up (i.e. we don't want to wait anymore) - all sessions fall in one of these three: - not sent GR capability, thus don't send EoR - have sent GR capab + have sent EoR - have sent GR capab + have sent R bit Until "all done?" is true, not route selection happens and no outbound routes are advertised - not even to non-GR boxes or GR boxes with R bit. > If you restart, your RIB comes up empty, you want the remotes to send > their routes ASAP. So you need to set the R-bit. Well, you always want updates ASAP ;) - this is not what the R bit is for. The R-bit is to tell the peer that you're minding your own business and it might be an extra minute until an EoR shows up from you. What you really *want* here is not run the route selection processes 4 or 5 times as much as you need, just once after you actually have all of the data. > Well, yes, I am well aware the point is deadlock removal. > > The point is to ensure that deferring route selection is done only when it > is useful, where one side genuinely has restarted and the other not. > I.e. we want XOR on the restart-state. For 1-1 or 0-0 R-bit, there's no > point anyone waiting (and obviously deadlock prone), so the point is to > ensure that it only happens for 0-1 or 1-0. Which ensures you never get > both sides deferring. > > With a timer on the restart-bit (and remember, it is configurable on each > side), you can get 2 sides restarting (A-B = 1-1) but the session coming > up after the timer has expired on one (A-B = 1-0). So now you've got 1 > side pointlessly waiting for the other. > > Further, those speakers could also be received routes from other routers > that didn't restart, and so have a fully-up-to-date RIB: > > C-A-B-D > 0-1-0-0 > > So C is sending routes to A, D and B are sending routes to each other, A > is waiting on C (correctly) and on D (incorrectly - D had restarted). I don't get it, is this a linear C<->A<->B<->D peering scenario, or is this an everyone-with-everyone? Why would A wait on D, it doesn't even have a session to D? Also, the R-bit is pairwise, you could be sending 1 to one session and 0 to another (which came up later for example). > For the above example the reality is: > > C-A-B-D > 0-1-1-0 > > What should happen is that A and B each wait on C and D to send their full > RIBs, and also update each other (no wait). Once C and D are done sending, > they send EoR, and /now/ finally A and B can send their best routes to C > and D. > > That should happen deterministically with this patch. And it'd happen in a situation where a solid 500day router is configured with a new session to a router that is currently down, and then when the router comes up in GR we send R=1... > You could have a longer chain of this, where it should be 0-1-1-1-1-1-0, > but you get 0-1-0-1-0-1-0 and you now a series of arbitrary bottle-necks > to routes propogating, with routers waiting on others unnecessarily. The 0 routers would be propagating their updates immediately since that's what they said: R=0 "i'm operating normally, you can expect an EoR from me in reasonable time". And the 1 routers would go to R=0 after their timer expires, which on wallclock time would happen simultaneously? > Without this patch it is dependent on timer and timing issues. Different > things could happen on different days. I hate that kind of behaviour > myself. > > The comment about the EoR being per-AFI/SAFI and R-bit global is an aside. > If you're trying to deterministically determine when to clear R-bit > according to the actual connection state, then it would have been nice if > R-bit and EoR were at same level. Ah, I see what you mean. Well, you *can* handle "all done?" on a per-SAFI level, i.e. start selecting and sending updates when all peers on a particular AFI/SAFI have indicated !GR || R=0 || EoR(AFI/SAFI). Also, you can send R=0 and send "false" EoRs, which has the same effect as R=1 for the particular AFI/SAFI you sent the EoR for. (In fact, the R bit is redundant - you could as well just send EoRs directly first thing, and it'd have the same effect.) > > All in all, the old code seems correct - and beautiful in its > > simplicity. While the new code isn't technically wrong and won't cause > > persistent issues (as far as I can see), there is the downside of above > > scenario where it causes extra load - and I can see no upside? > > It's state-indeterministic. It depends on (invisible remote) configured > timer and runtime-variability (e.g. load induced issues), which I find > annoying and nasty. I don't see the fallout yet, probably because I didn't understand your above ABCD example. -David _______________________________________________ Quagga-dev mailing list [email protected] https://lists.quagga.net/mailman/listinfo/quagga-dev
