On Mon, May 18, 2020 at 8:42 AM <bruno.decra...@orange.com> wrote: > Hello Gyan, > > > > Nice to know that fast LSP flooding is important to you. > > You are right that there are existing code and knobs to improve IGP > convergence in general, and in particular to the specific implementation > you are referring to (I’d bet that its first letter start with C).. > > Focusing on efficient flooding, which is the subject of this thread, > “fast-flood x” is indeed a nice name. AFAIR, the feature has been > introduced approximately 15 years ago and was indeed an improvement. Many > more have been introduced as part of the fast IGP convergence project that > I’d been running in this company, resulting in very significant and much > needed improvement on that implementation. > > Despite the name “fast flood”, I don’t think that anyone is claiming that > it’s solving the broad issue of efficient LSP flooding. > > LSP-pacing may also be configured in order to influence the delay between > two consecutively sent LSP. > > > > One question, which you have likely faced when enabling these knobs, is > the right values to configure, depending on the receiver (’s capability) > If you have multiple implementation on the sender side –with different ways > to implement ‘fast flooding’- and different platforms (‘s capability) on > the receiver side, combined with various usages (e.g. from 2 to 50 IGP > adjacencies, with or without BGP/RSVP-TE/multicast….) the matrix is quickly > getting significant. Your vendor of the receiver implementation may help > you; or not (significantly). > Gyan> I completely understand the issue and that makes sense that with the variables thrown out there with a mixed vendor environment with a variety of platforms and sender flooding capability, and the receiver receiving the flooded LSP capability to be processed during its local SPF and overrunning the receivers. So this is a real world worst case scenario which can happen with flow control and how to deal with the resulting slow convergence and sub optimal routing with micro loops. Got it.
Thank you for the explanation. > With IS-IS, the problem is exacerbated because the protocol is typically > vital to the network and all its usages and customers. So nobody wants to > be held accountable if the parameters turned to be ‘non-optimal’ over the > next decade. Especially as routing ‘incidents’ may already happen with > default configuration. > > Gyan> Default change are painful for operators but unfortunately they do > happen and throw off any tuning or standard template deployed. > > > > Sending fast is “easy”, but what we need is an effective communication > between the two entities involved: one sender and a receiver; sometimes > referred to as goodput. RFC 2914 “Congestion Control Principles” [1] may > be an introduction to this. > > e.g. you can probably speak English to me at 200-250 words per minutes but > I can assure you that it would be more effective to speak to me at a lower > rate, because of limitations on the receiver side. > > That’s similar for machines in general and IS-IS in particular, especially > since IS-IS is not that fast to recover from lost LSPs. > > The problem is not new and referred to as flow control. From [2] “In data > communications <https://en.wikipedia.org/wiki/Data_communications>, *flow > control* is the process of managing the rate of data transmission between > two nodes to prevent a fast sender from overwhelming a slow receiver. It > provides a mechanism for the receiver to control the transmission speed, so > that the receiving node is not overwhelmed with data from transmitting > node.” > > > > That’s what we are working on. > > Gyan> Makes sense. > > Kind regards, > > Bruno > > > > [1] https://tools.ietf.org/html/rfc2914 > > [2] https://en.wikipedia.org/wiki/Flow_control_(data) > > > > > > > > *From:* Gyan Mishra [mailto:hayabusa...@gmail.com] > *Sent:* Sunday, May 17, 2020 11:07 AM > *To:* DECRAENE Bruno TGI/OLN > *Cc:* Christian Hopps; Les Ginsberg (ginsberg); lsr@ietf.org > *Subject:* Re: [Lsr] Flooding across a network > > > > > > Am reading through this thread late but want to chime in on the discussion. > > > > Router Isis > > fast-flood x > > > > The concept of ISIS fast flooding has been around for decades and is > critical for ISIS LSDB synchronization to so all nodes have all prefixes to > avoid micro loops and within the network. > > > > The concept of fast flood feature is basically to flood the number of LSPs > that need to be flooded before starting the local SPF. So with fast flood > enabled all nodes in the domain wait till the last LSP is flooded before it > runs its local SPF. By fast flooding it reduces the overall number of SPFs > executed and allows the LSDB to be synchronized by all nodes in the domain. > > > > Let’s say a link went down or came up topology change. The router should > at a minimum flood at least that LSP that triggered the SPF before running > its local SPF. Fast flooding is recommended by all vendors as it improves > convergence times and limits duration of LSDB inconsistencies during which > time micro loops form. > > > > In case where a few routers don’t have fast flood enabled, those few > router in slow mode may have to run multiple SPFs before completely being > synchronized. I think that question where a mix of slow and fast exist > versus all slow or all fast you may have mixed unpredictable behaviors that > you really would have to test out in a lab or live network. > > > > In my experience I have always had all nodes running fast flood mode as > the feature has been around for a long time and recommended for optimal > convergence. > > > > There are many other parameters with ISIS that come into play to optimize > convergence below which may vary by vendor but can improve convergence and > minimize micro loops below: > > > > One of them is setting all P2P routed links to circuit type point-to-point > to avoid DIS election. > > > > Both ISIS and OSPF have the iSPF incremental SPF so only the changed part > of the tree is updated and not the entire tree to save on SPF processing. > > > > With short spf interval and spf delay the local spf May start before the > lsp that triggered the spf is flooded. > > > > Lsp pacing can be used to speed up end to end flooding. > > > > Hello padding TLV 8 for MTU defection to help with convergence so max > number of LSPs can be sent during flooding. > > > > Increase lsp lifetime to maximum to reduce control traffic so cpu can be > optimized to refresh lsp’s > > > > Reduce the frequency of periodic lsp flooding of topology to reduce link > utilization by Isis > > > > Ignore lsp errors instead of purging them > > > > Carful balance of event processing and throttling of events to maximize > convergence times > > > > Prefix prioritization on critical prefixes such as MPLS FEC binding. > > > > LFA and RLFA for pre computed backup paths > > > > BFD for ms failure detection > > > > Overall Isis as far as scalability far exceeds ospf with stability with > domains with a very large number of nodes in the 100s and high number of > adjacencies per node. > > > > As far as stability and convergence both ospf and Isis are equivalent from > my experience. > > > > Kind regards > > > > Gyan > > Verizon > > > > On Thu, May 7, 2020 at 12:03 PM <bruno.decra...@orange.com> wrote: > > Les, > > > > > > *From:* Les Ginsberg (ginsberg) [mailto:ginsb...@cisco.com] > *Sent:* Thursday, May 7, 2020 4:55 PM > *To:* DECRAENE Bruno TGI/OLN > *Cc:* lsr@ietf.org; Christian Hopps > *Subject:* RE: [Lsr] Flooding across a network > > > > Bruno – > > > > I have specifically used an example where “microloop avoidance” is not > applicable. So I did not want to use the term “microloop” but rather used > “loop” so as not to suggest that “microloop avoidance” is a potential > solution for the sub-optimal behavior. > > [Bruno] I’m not sure what you mean exactly by “microloop avoidance”. > > On my side, I mean “Loop avoidance using Segment Routing” [1] which _*is*_ > applicable. (Note that I’m not saying that all _*implementations*_ cover > all cases.) > > [1] > https://tools.ietf.org/html/draft-bashandy-rtgwg-segment-routing-uloop-08 > > > > > > Hope you can appreciate that point. > > > > It would be easy enough to include more nodes in the topology which only > support slow flooding. The end result would be the same. > > [Bruno] The question is whether the activation of fast flooding on one > (some) node(s)/IGP adjacency(es) may result in delaying the LSDB > synchronization network wide. It’s not about the number of slow nodes. > > > > You seem to assume that we have a majority of fast nodes, and some > remaining slow nodes. In which cases I agree that in some cases the > overall/network wide behavior may be a slow LSDB sync. But not slower than > with slow nodes only. > > I have kept the example simple in the hopes we could more easily agree > that what I describe can happen when not all nodes support faster flooding > – which is the only point I am trying to make. > > [Bruno] If your point is that while we have slow nodes in the network, in > some cases the network wide behavior can be “slow”, somewhere between “all > nodes are fast” and “all nodes are slow”, then I agree with you. > > > > However what you said is: “when only some nodes in the network support > faster flooding […] it prolongs the period of LSDB inconsistency..” > > In that sentence, I disagree with “prolongs” which according to 2 > dictionaries means “lasting longer”. > > https://dictionary.cambridge.org/fr/dictionnaire/anglais/prolong > > https://www.merriam-webster.com/dictionary/prolong > > > > Because, at least to me, this reads as “The introduction of fast-flooding > nodes in the network (may) increase the period of LSDB inconsistency.” > > > > Whether the ratio of Fast/Slow nodes is large or small or about the same > doesn’t eliminate the possibility that the same behavior could be seen – > though it might alter the location of the topology change which would be > problematic. > > > > From an operator’s POV, I am pretty sure that what you really care about > is whether packets get successfully forwarded or not. > > [Bruno] Yes, but let’s not combine multiple (complex) problems. > > Here I’d like we focus on LSP flooding and LSDB synchronization across the > network. Plus this is exactly the point that you raised: “increase the > period of LSDB inconsistency.” > > > > I am demonstrating that it isn’t safe to assume forwarding behavior will > be optimal when not all nodes support fast flooding. > > [Bruno] Which is very different from your original point “when only some > nodes in the network support faster flooding […] it prolongs the period of > LSDB inconsistency.” > > > > --Bruno > > > > Les > > > > > > *From:* bruno.decra...@orange.com <bruno.decra...@orange.com> > *Sent:* Thursday, May 07, 2020 2:18 AM > *To:* Les Ginsberg (ginsberg) <ginsb...@cisco.com> > *Cc:* lsr@ietf.org; Christian Hopps <cho...@chopps.org> > *Subject:* RE: [Lsr] Flooding across a network > > > > Les, > > > > > From: Les Ginsberg (ginsberg) [mailto:ginsb...@cisco.com > <ginsb...@cisco.com>] > > > > > > Bruno - > > > > > > I am sorry it has been so difficult for us to understand each other. I > am trying my best. > > +1 +1 > > > > > Look at it this way: > > > > > > You are the customer. 😊 > > > I am the vendor. > > > > I'm not sure what (technical) point you are trying to make. > > > > Coming back to your statement : “when only some nodes in the network > support faster flooding […] it prolongs the period of LSDB inconsistency > .” > > Your example is flawed on two points: > > - Your performance indicator is "micro loop duration", while we are > talking about the duration of LSP flooding across the network. So the > metric should be "LSP flooding time" (or "period of LSDB inconsistency") > > - Your example is about one node doing slower flooding, while we are > interested in the case when one node support faster flooding. (It's quite > clear that if one node is slower, the end to end flooding time may be > longer. The point you are raising is when one node is doing faster flooding) > > > > Can you fix your example on those 2 points? > > > > Thanks > > --Bruno > > > > > > > The failure scenario I describe below happens and you notice that all > Northbound destinations loop for 35 seconds whenever fast flooding is > enabled. > > > I think you are going to complain about this - to me. 😊 > > > > > > And I am going to tell you that this is a consequence of enabling fast > flooding in the presence of a node which does not support it. Your options > to reduce the period of looping will be: > > > > > > 1)Upgrade the slow node to support faster flooding > > > 2)Disable fast flooding > > > 3)Redesign your network > > > > > > Les > > > > > > > -----Original Message----- > > > > From: bruno.decra...@orange.com <bruno.decra...@orange.com> > > > > Sent: Wednesday, May 06, 2020 10:10 AM > > > > To: Christian Hopps <cho...@chopps.org> > > > > Cc: Les Ginsberg (ginsberg) <ginsb...@cisco.com>; lsr@ietf.org > > > > Subject: RE: [Lsr] Flooding across a network > > > > > > > > > From: Christian Hopps [mailto:cho...@chopps.org <cho...@chopps.org>] > > > > > > > > > > Bruno persistence has made me realize something fundamental here. > > > > > > > > > > The minute the LSP originator changes the LSP and floods it you have > LSDB > > > > inconsistency. > > > > > > > > Exactly my point. Thank you Chris. > > > > I would even say: "The minute the LSP originator changes the LSP then > you > > > > have LSDB inconsistency." But no big deal if there is disagreement on > this > > > > detail. > > > > > > > > > That is going to last until the last node in the network has updated > it's LSDB. > > > > > > > > Absolutely. > > > > So the faster we flood, the shorter the LSBD inconsistency. > > > > > > > > Now IMO, even if a single/few nodes flood faster, there is a chance of > > > > shortening the LSDB inconsistency. But in all cases, I don't see how > this could > > > > make the LSDB inconsistency longer. > > > > > > > > > > > > > Les is pointing out that LSDB inconsistency can be bad in certain > > > > circumstances e.g., if a critical node is slow and thus inconsistent. > > > > > > > > > > I believe the right way to fix this is a simple one, help the > operator flag the > > > > broken router software/hardware for replacement, but otherwise IS-IS > > > > should just try to do the best job it can do to which is to flood > around the > > > > problem (i.e., flood as optimally as possible). > > > > > > > > +1 > > > > On a side note, I would not call a router flooding slowly as "broken".. > I find it > > > > understandable that in a given network there are different type of > routers > > > > (core vs aggregation), different roles (P having 50 IGP adjacencies > with 50 PEs > > > > vs PE having only 2 IGP adjacencies with 2 P), different hardware > > > > generations, different software, different vendors with different > > > > perspectives/markets. > > > > > > > > Thank you Chris. > > > > > > > > --Bruno > > > > > > > > > > Thanks, > > > > > Chris. > > > > > [as WG member] > > > > > > > > > > > > > > > > On May 6, 2020, at 10:33 AM, bruno.decra...@orange.com wrote: > > > > > > > > > > > > Les, > > > > > > > > > > > > From: Les Ginsberg (ginsberg) [mailto:ginsb...@cisco.com > <ginsb...@cisco.com>] > > > > > > Sent: Wednesday, May 6, 2020 4:14 PM > > > > > > To: DECRAENE Bruno TGI/OLN > > > > > > Cc: lsr@ietf.org > > > > > > Subject: RE: Flooding across a network > > > > > > > > > > > > Bruno – > > > > > > > > > > > > I am somewhat at a loss to understand your comments. > > > > > > The example is straightforward and does not need to consider FIB > update > > > > time nor the ordering of prefix updates on different nodes. > > > > > > [Bruno] The example is straightforward but you are referring to > FIB and IP > > > > packets forwarding as per those FIBs. > > > > > > I’d like we focus on LSP flooding and LSDB consistency. > > > > > > > > > > > > Consider the state of Node B and Node D at various time points > from the > > > > trigger event. > > > > > > > > > > > > T+ 2 seconds: > > > > > > ----------------- > > > > > > B has received all LSP Updates. It triggers an SPF and for all > Northbound > > > > destinations previously reachable via C it installs paths via D. > > > > > > Let’s assume it take 5 seconds to update the forwarding plane. > > > > > > > > > > > > D has received 40 of the 1000 LSP updates. It triggers an SPF and > finds > > > > that all Northbound destinations are reachable via B-C. It makes no > changes > > > > to the forwarding plane. > > > > > > > > > > > > T+7 seconds > > > > > > ----------------- > > > > > > B has completed FIB updates. Traffic to all Northbound > destinations is > > > > being forwarded via D. > > > > > > > > > > > > D has now received 140 of the 1000 LSP updates. Entries in its > forwarding > > > > plane for Northbound destinations still point to B. > > > > > > > > > > > > We have a loop. > > > > > > > > > > > > T + 30 seconds > > > > > > -------------------- > > > > > > D has now received 600 of the 1000 LSP updates. Still no changes > to its > > > > forwarding plane. > > > > > > Traffic to Northbound destinations is still looping. > > > > > > > > > > > > T+ 50 seconds > > > > > > ------------------- > > > > > > D has finally received all 1000 LSP updates.. > > > > > > It triggers (another) SPF and calculates paths to Northbound > destinations > > > > via E. It begins to update its forwarding plane. > > > > > > Let’s assume this will take 5 seconds.. > > > > > > > > > > > > T + 55 seconds > > > > > > -------------------- > > > > > > D has completed forwarding plane updates – no more looping. > > > > > > > > > > > > That is all I am trying to illustrate. > > > > > > > > > > > > If you want to start arguing that node protecting LFAs + microloop > > > > avoidance could help (NOTE I explicitly took those out of the example > for > > > > simplicity) – it is easy enough to change the example to include > multiple node > > > > failures or a node failure plus some northbound link failures on other > nodes. > > > > > > [Bruno] I’m not talking about LFA/FRR. And with regards to > microloops > > > > avoidance, some algorithms can handle any graph transition so including > > > > multiple node failures. > > > > > > > > > > > > But again, let’s stick to LSP flooding and LSDB consistency. (you > are the > > > > one speaking about microloops in the forwarding plane). > > > > > > > > > > > > The point here is to look at the impact of long-lived LSDB > inconsistency > > > > which results when some nodes support flooding an order of magnitude > > > > faster flooding than other nodes – which is what you asked me to > clarify. > > > > > > [Bruno] No. I asked you to clarify why having a node with faster > flooding > > > > could prolongs the period of LSDB inconsistency. > > > > > > > > > > > > Again, with you own words: “when only some nodes in the network > > > > support faster flooding the behavior of the whole network may not be > > > > "better" when faster flooding is enabled because it prolongs the > period of > > > > LSDB inconsistency.” > > > > > > And with less words: “when only some nodes in the network support > > > > faster flooding […] it prolongs the period of LSDB inconsistency.” > > > > > > > > > > > > --Bruno > > > > > > > > > > > > Les > > > > > > > > > > > > > > > > > > > > > > > > From: bruno.decra...@orange.com <bruno.decra...@orange.com> > > > > > > Sent: Wednesday, May 06, 2020 6:21 AM > > > > > > To: Les Ginsberg (ginsberg) <ginsb...@cisco.com> > > > > > > Cc: lsr@ietf.org > > > > > > Subject: RE: Flooding across a network > > > > > > > > > > > > Les, > > > > > > > > > > > > From: Les Ginsberg (ginsberg) [mailto:ginsb...@cisco.com > <ginsb...@cisco.com>] > > > > > > Sent: Wednesday, May 6, 2020 1:35 AM > > > > > > To: DECRAENE Bruno TGI/OLN; l...@ietf..org > > > > > > Subject: RE: Flooding across a network > > > > > > > > > > > > Bruno - > > > > > > > > > > > > Seems like it was not too long ago that we were discussing this in > person. > > > > Ahhh...the good old days... > > > > > > [Bruno] Indeed, may be not to the point of concluding. Indeed. > > > > > > > > > > > > First, let's agree that the interesting case does not involve 1 or > even a > > > > small number of LSPs. For those cases flooding speed does not matter. > > > > > > The interesting cases involve a large number of LSPs (hundreds or > > > > thousands). And in such cases LFA/microloop avoidance techniques are > not > > > > applicable. > > > > > > > > > > > > Take the following simple topology: > > > > > > > > > > > > | | ... | | > > > > > > +---+ +---+ > > > > > > | C | | E | > > > > > > +---+ +---+ > > > > > > | | 1000 > > > > > > +---+ +---+ > > > > > > | B |-------------| D | > > > > > > +---+ 1000 +---+ > > > > > > | | > > > > > > | | > > > > > > \ / > > > > > > \ / > > > > > > \ / > > > > > > \ / > > > > > > +---+ > > > > > > | A | > > > > > > +---+ > > > > > > > > > > > > There is a topology northbound of C and E (not shown) and a > topology > > > > southbound of A (not shown). > > > > > > Cost on all links is 10 except B-D and D-E where cost is high. > > > > > > > > > > > > C is a node with 1000 neighbors. > > > > > > When all links are up, shortest path for all northbound > destinations is via > > > > C. > > > > > > All nodes in the network support fast flooding except for Node D. > > > > > > Let’s say fast flooding is 500 LSPs/second and slow flooding (Node > D) is 20 > > > > LSPs/seconds. > > > > > > If Node C fails we have 1000 LSPs to flood. > > > > > > All nodes except for D can receive these in 2 seconds (plus > internode > > > > delay time). > > > > > > D can receive LSPs in 50 seconds. > > > > > > > > -- <http://www.verizon.com/> *Gyan Mishra* *Network Solutions A**rchitect * *M 301 502-134713101 Columbia Pike *Silver Spring, MD
_______________________________________________ Lsr mailing list Lsr@ietf.org https://www.ietf.org/mailman/listinfo/lsr
