Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
> On Jun 10, 2020, at 2:10 PM, Tony Przygienda wrote: > > > > On Wed, Jun 10, 2020 at 11:04 AM Christian Hopps wrote: > > > > > > I also suggest to look up why in PNNI we ended up introducing a special "L1 > > equivalent" computation to the peer group leader to validate that it was > > actually reachable for correct operation (especially hierarchy > > negotiations) on peer group egresses which in a sense was like but worse > > than virtual links operationally IME. I was suggesting then to keep an SVC > > from PGL to every border for that reason since a partition of a PGL led > > otherwise to the peer groups looking like the same thing for a while with > > highly undesirable operational effects (my memory doesn't reach that far > > really but we had at least discussions then to implement proprietary > > solution in Fore to have such SVCs for more stable deployments). In more > > abstract terms, flooding is extremely good to quickly "route around > > failures" when e'ones state is completely decentralized but is simply not a > > great mechanism if you have to talk to an entity couple hops away in a > > stable manner, especially if this entity hold state you ne ed for correct operation when talking to your peers. > > > So we understand L2 tunnels from leaf to FR are "real" tunnels and have > nothing to do with virtual links. > > As to the data plane, nope, you don't end up with any "virtual links" I can > recognize unless you choose to call redistribution "virtual links" as well, > maybe look @ the presentation again. That's exactly it! Excellent. :) Safely using redistributed reachability to provide transit w/o tunnels is exactly what OSPF virtual links are, so that's why I'm saying "A solution without tunnels is also possible by judicious scoping of reachability information between the levels." will reduce to the same thing, if you want it to actually work generically. Perhaps this will become more obvious when that part of the draft is fleshed out. Thanks, Chris. > > -- tony > ___ > Lsr mailing list > Lsr@ietf.org > https://www.ietf.org/mailman/listinfo/lsr ___ Lsr mailing list Lsr@ietf.org https://www.ietf.org/mailman/listinfo/lsr
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
On Wed, Jun 10, 2020 at 11:04 AM Christian Hopps wrote: > > ... > > > > I also suggest to look up why in PNNI we ended up introducing a special > "L1 equivalent" computation to the peer group leader to validate that it > was actually reachable for correct operation (especially hierarchy > negotiations) on peer group egresses which in a sense was like but worse > than virtual links operationally IME. I was suggesting then to keep an SVC > from PGL to every border for that reason since a partition of a PGL led > otherwise to the peer groups looking like the same thing for a while with > highly undesirable operational effects (my memory doesn't reach that far > really but we had at least discussions then to implement proprietary > solution in Fore to have such SVCs for more stable deployments). In more > abstract terms, flooding is extremely good to quickly "route around > failures" when e'ones state is completely decentralized but is simply not a > great mechanism if you have to talk to an entity couple hops away in a > stable manner, especially if this entity hold state you need for correct > operation when talking to your peers. > > So we understand L2 tunnels from leaf to FR are "real" tunnels and have nothing to do with virtual links. As to the data plane, nope, you don't end up with any "virtual links" I can recognize unless you choose to call redistribution "virtual links" as well, maybe look @ the presentation again. -- tony ___ Lsr mailing list Lsr@ietf.org https://www.ietf.org/mailman/listinfo/lsr
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
> On Jun 10, 2020, at 1:56 PM, Tony Przygienda wrote: > > > > On Wed, Jun 10, 2020 at 10:29 AM Christian Hopps wrote: > > > > > > > > > Hmmm, AFAIR the implementation of OSPF virtual links was having no tunnel > > at all (and that's how I remember I implemented it then). I cite > > Correct, that's why I'm suggesting that any solution without tunnels is going > to reduce to OSPF virtual links. > > > Good to hear you implemented virtual w/o tunnels. > > But then FR is not a solution "without tunnels" so the whole logic of "it's > not the same hence it's the same for me" escapes me completely now and I let > it rest. > > > > If it seems like I'm "carrying the torch" it's b/c as I understand the drafts > currently, the area proxy seems like a much simpler solution (KISS), and I > haven't yet been able to figure out what the benefits of using the flood > reflector method instead would bring. If those benefits were more obvious I'd > probably like it too. :) > > > looking @ the amount of protocol extensions in terms of TLVs, flooding > scoping, configuration etc necessary in either case and involved > fork-lifting, new-constructs necessary in operational tooling I dare say you > hold a somewhat unique "opinion" IMO ... > > As to simplicity you see and since your mental model seems fixed around > virtual links concept, Tony, I think you have misunderstood me at some point. When asked about the requirement of tunnels you said that they were not required, and that a solution could be done with carefully scoped reachability, this was also mentioned in your second presentation. *ALL* I've been talking about WRT virtual links is that when you hint at a non-tunnel solution in your draft and presentation involving reachability, that your going to end up with OSPF virtual links. That is all. Thanks, Chris. [as WG member] > I also suggest to look up why in PNNI we ended up introducing a special "L1 > equivalent" computation to the peer group leader to validate that it was > actually reachable for correct operation (especially hierarchy negotiations) > on peer group egresses which in a sense was like but worse than virtual links > operationally IME. I was suggesting then to keep an SVC from PGL to every > border for that reason since a partition of a PGL led otherwise to the peer > groups looking like the same thing for a while with highly undesirable > operational effects (my memory doesn't reach that far really but we had at > least discussions then to implement proprietary solution in Fore to have such > SVCs for more stable deployments). In more abstract terms, flooding is > extremely good to quickly "route around failures" when e'ones state is > completely decentralized but is simply not a great mechanism if you have to > talk to an entity couple hops away in a stable manner, especially if this > entity hold state you need for correct operation when talking to your peers. > > -- tony > ___ Lsr mailing list Lsr@ietf.org https://www.ietf.org/mailman/listinfo/lsr
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
On Wed, Jun 10, 2020 at 10:29 AM Christian Hopps wrote: > > > > > > > > > Hmmm, AFAIR the implementation of OSPF virtual links was having no > tunnel at all (and that's how I remember I implemented it then). I cite > > Correct, that's why I'm suggesting that any solution without tunnels is > going to reduce to OSPF virtual links. > > Good to hear you implemented virtual w/o tunnels. But then FR is not a solution "without tunnels" so the whole logic of "it's not the same hence it's the same for me" escapes me completely now and I let it rest. > > > If it seems like I'm "carrying the torch" it's b/c as I understand the > drafts currently, the area proxy seems like a much simpler solution (KISS), > and I haven't yet been able to figure out what the benefits of using the > flood reflector method instead would bring. If those benefits were more > obvious I'd probably like it too. :) > > looking @ the amount of protocol extensions in terms of TLVs, flooding scoping, configuration etc necessary in either case and involved fork-lifting, new-constructs necessary in operational tooling I dare say you hold a somewhat unique "opinion" IMO ... As to simplicity you see and since your mental model seems fixed around virtual links concept, I also suggest to look up why in PNNI we ended up introducing a special "L1 equivalent" computation to the peer group leader to validate that it was actually reachable for correct operation (especially hierarchy negotiations) on peer group egresses which in a sense was like but worse than virtual links operationally IME. I was suggesting then to keep an SVC from PGL to every border for that reason since a partition of a PGL led otherwise to the peer groups looking like the same thing for a while with highly undesirable operational effects (my memory doesn't reach that far really but we had at least discussions then to implement proprietary solution in Fore to have such SVCs for more stable deployments). In more abstract terms, flooding is extremely good to quickly "route around failures" when e'ones state is completely decentralized but is simply not a great mechanism if you have to talk to an entity couple hops away in a stable manner, especially if this entity hold state you need for correct operation when talking to your peers. -- tony ___ Lsr mailing list Lsr@ietf.org https://www.ietf.org/mailman/listinfo/lsr
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
> On Jun 10, 2020, at 12:40 PM, Tony Przygienda wrote: > > > > On Wed, Jun 10, 2020 at 4:27 AM Christian Hopps wrote: > > > > On Jun 9, 2020, at 10:01 PM, Tony Przygienda wrote: > > > > Chris (addressing in WG member context you declared), I reply tersely since > > we will put more work into the draft once it's adopted (for which I think > > you saw a good amount of support in two threads already). > > > > I deferred from your email since the chain-terrasteam topology you're > > showing is simply not what we are dealing in any operational, successful > > networks today AFAIK frankly and I saw lots of "assume complexity" and > > "dislikes" in your email which I didn't read as technical arguments but > > mental attitudes. Likes or dislikes and assumptions are fine but we should > > probably focus on existing network/customer technical + operational > > arguments & requirements when building solutions and now what you or we > > like first. > > Both Area Proxy and Flood Reflector are proposing to use L1 areas as transit > to connect L2, isn't that chaining? It seemed like a decent way to help > visualize the proposals along with some numbers, perhaps you have something > better... > > The Area Proxy draft is making everything L2 and using the L1 areas to > redefine the advertised topology information to allow it to scale. Because > everything is ultimately L2 nothing changes in the data plane to provide this > transit. > > The flood reflector draft is keeping the L1-only abstraction so it has to > provide for transit some other way. > > > So trying to extract the technical point you seem to be making inbetween > > all that > > > > a) I see how you can try to have a mental model of "virtual links". What we > > suggest are not virtual links (I implemented VL @ least once but it's so > > long ago I forgot pretty much all the details so had to look stuff up :-) > > Virtual links in OSPF were "magic bits on LSAs" that kind of computed "SPF > > reachability through the area to change SPF" edge-to-edge and the > > asynchronicity of all that flooding-being-tunnels-being-SPF was playing > > havoc on us @ this time of 300MHz CPUs + frankly, the complexity of that > > was not needed @ this time just as partition healing was never implemented > > in ISIS.. That's why it never went anywhere much, my take, others may > > correct. Saying "virtual links are bad" and "this is virtual links to me so > > it's bad" is simply a "strawman fallacy" to me frankly. This draft suggests > > (but as I wrote Bruno as answer to his fairly deep email) to run proper > > flooding over proper tunnels (we run routing over tunnels all the time in > > all kind of scenarios be it BGP proper or SD-WAN or overlays obviously) but if you choose FRs to be one hop away you can get away without any "L2 tunneled adjacencies", that's deployment choice. > > If you are not using tunnels, but are still trying to provide transit through > the L1 area for L2, that is exactly what OSPF virtual links are doing. Part > of making those work is the advertisement of reachability into the area from > another, changes to router advertisements (to indicate if the area is transit > capable), and changes to the SPF calculation to modify route choices based on > whether they are based on virtual links or not. > > The complexity of OSPF virtual links is there to make them work, right? I'm > not trying to make any argument, strawman or otherwise; I'm just trying to > understand what is being proposed as a replacement for the full-mesh of > tunnels. > > It's nice if it reduces to OSPF virtual links b/c we then have an example of > how to actually implement it, and years of experience to understand it. If > that highlights that getting the non-tunneled choice right isn't easy, well I > guess that's important, right? > > > Hmmm, AFAIR the implementation of OSPF virtual links was having no tunnel at > all (and that's how I remember I implemented it then). I cite Correct, that's why I'm suggesting that any solution without tunnels is going to reduce to OSPF virtual links. :) > > " > The InterfaceUp event occurs for > a virtual link when its corresponding routing table entry becomes > reachable. Conversely, the InterfaceDown event occurs when its > routing table entry becomes unreachable. In other words, the > virtual link's viability is determined by the existence of an > intra-area path, through the Transit area, between the two > endpoints. > > " > and that was largely the problem, flooding+SPF+routing table was basically > the adjacency keep up and very flappy/circular instead of proper stable > tunnel with hellos. Did you implement and run virtual links in OSPF over > tunnels? which type? Well I had to implement them (virtual links, not tunnels, since that's just how OSPF virtual links work) back in the day when I added the non-backbone code to the second GateD
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
ok, let's not drag vendor specific stuff in. I shouldn't have brought it up I guess, outside the scope of IETF threads ... thanks --- tony On Wed, Jun 10, 2020 at 10:23 AM wrote: > > Tony, > > > On Jun 10, 2020, at 9:40 AM, Tony Przygienda wrote: > > You do seem to be carrying as WG member a hot torch for area-proxy for > some reason, that's fine with me, frankly, I had extensive discussions with > customers when DriveNet was being proposed to them (which AFAIS is > basically area-proxy) and the solution is intriguing but it did not cut > lots of requirements of large customers and there are a lot of unresolved > issues operationally with an approach like that. > > > > Drivenets, as I understand it, is an attempt to physically deaggregate a > multi-chassis fabric down to the chip level using a proprietary > chip-set-specific cell based interlink protocol. However, it retains a > single control plane and as such looks like a single IS-IS system. It has > no relationship whatsoever to area proxy. > > Regards, > Tony > > > > ___ Lsr mailing list Lsr@ietf.org https://www.ietf.org/mailman/listinfo/lsr
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
Tony, > On Jun 10, 2020, at 9:40 AM, Tony Przygienda wrote: > > You do seem to be carrying as WG member a hot torch for area-proxy for some > reason, that's fine with me, frankly, I had extensive discussions with > customers when DriveNet was being proposed to them (which AFAIS is basically > area-proxy) and the solution is intriguing but it did not cut lots of > requirements of large customers and there are a lot of unresolved issues > operationally with an approach like that. Drivenets, as I understand it, is an attempt to physically deaggregate a multi-chassis fabric down to the chip level using a proprietary chip-set-specific cell based interlink protocol. However, it retains a single control plane and as such looks like a single IS-IS system. It has no relationship whatsoever to area proxy. Regards, Tony ___ Lsr mailing list Lsr@ietf.org https://www.ietf.org/mailman/listinfo/lsr
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
On Wed, Jun 10, 2020 at 4:27 AM Christian Hopps wrote: > > > > On Jun 9, 2020, at 10:01 PM, Tony Przygienda > wrote: > > > > Chris (addressing in WG member context you declared), I reply tersely > since we will put more work into the draft once it's adopted (for which I > think you saw a good amount of support in two threads already). > > > > I deferred from your email since the chain-terrasteam topology you're > showing is simply not what we are dealing in any operational, successful > networks today AFAIK frankly and I saw lots of "assume complexity" and > "dislikes" in your email which I didn't read as technical arguments but > mental attitudes. Likes or dislikes and assumptions are fine but we should > probably focus on existing network/customer technical + operational > arguments & requirements when building solutions and now what you or we > like first. > > Both Area Proxy and Flood Reflector are proposing to use L1 areas as > transit to connect L2, isn't that chaining? It seemed like a decent way to > help visualize the proposals along with some numbers, perhaps you have > something better... > > The Area Proxy draft is making everything L2 and using the L1 areas to > redefine the advertised topology information to allow it to scale. Because > everything is ultimately L2 nothing changes in the data plane to provide > this transit. > > The flood reflector draft is keeping the L1-only abstraction so it has to > provide for transit some other way. > > > So trying to extract the technical point you seem to be making inbetween > all that > > > > a) I see how you can try to have a mental model of "virtual links". What > we suggest are not virtual links (I implemented VL @ least once but it's so > long ago I forgot pretty much all the details so had to look stuff up :-) > Virtual links in OSPF were "magic bits on LSAs" that kind of computed "SPF > reachability through the area to change SPF" edge-to-edge and the > asynchronicity of all that flooding-being-tunnels-being-SPF was playing > havoc on us @ this time of 300MHz CPUs + frankly, the complexity of that > was not needed @ this time just as partition healing was never implemented > in ISIS.. That's why it never went anywhere much, my take, others may > correct. Saying "virtual links are bad" and "this is virtual links to me so > it's bad" is simply a "strawman fallacy" to me frankly. This draft suggests > (but as I wrote Bruno as answer to his fairly deep email) to run proper > flooding over proper tunnels (we run routing over tunnels all the time in > all kind of scenarios be it BGP proper or SD-WAN or overlays obviously) but > if you choose FRs to be one hop away you can get away without any "L2 > tunneled adjacencies", that's deployment choice. > > If you are not using tunnels, but are still trying to provide transit > through the L1 area for L2, that is exactly what OSPF virtual links are > doing. Part of making those work is the advertisement of reachability into > the area from another, changes to router advertisements (to indicate if the > area is transit capable), and changes to the SPF calculation to modify > route choices based on whether they are based on virtual links or not. > > The complexity of OSPF virtual links is there to make them work, right? > I'm not trying to make any argument, strawman or otherwise; I'm just trying > to understand what is being proposed as a replacement for the full-mesh of > tunnels. > > It's nice if it reduces to OSPF virtual links b/c we then have an example > of how to actually implement it, and years of experience to understand it. > If that highlights that getting the non-tunneled choice right isn't easy, > well I guess that's important, right? > > Hmmm, AFAIR the implementation of OSPF virtual links was having no tunnel at all (and that's how I remember I implemented it then). I cite " The InterfaceUp event occurs for a virtual link when its corresponding routing table entry becomes reachable. Conversely, the InterfaceDown event occurs when its routing table entry becomes unreachable. In other words, the virtual link's viability is determined by the existence of an intra-area path, through the Transit area, between the two endpoints. " and that was largely the problem, flooding+SPF+routing table was basically the adjacency keep up and very flappy/circular instead of proper stable tunnel with hellos. Did you implement and run virtual links in OSPF over tunnels? which type? And having run good amount of routing over tunnels I am still to see all those dragons you are summoning. GRE, incl soft GRE is very widely deployed and works well as are other tunnel types I worked with. A valid point of discussion is how many adjacencies you can keep up to the edge, my experience is hundreds which for all practical purposes is a very good scale but otherwise we can relax the draft and build a "flood reflector hierarchy", I'm open to that if it's a real concern. Ultimately running
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
> On Jun 9, 2020, at 10:01 PM, Tony Przygienda wrote: > > Chris (addressing in WG member context you declared), I reply tersely since > we will put more work into the draft once it's adopted (for which I think you > saw a good amount of support in two threads already). > > I deferred from your email since the chain-terrasteam topology you're showing > is simply not what we are dealing in any operational, successful networks > today AFAIK frankly and I saw lots of "assume complexity" and "dislikes" in > your email which I didn't read as technical arguments but mental attitudes. > Likes or dislikes and assumptions are fine but we should probably focus on > existing network/customer technical + operational arguments & requirements > when building solutions and now what you or we like first. Both Area Proxy and Flood Reflector are proposing to use L1 areas as transit to connect L2, isn't that chaining? It seemed like a decent way to help visualize the proposals along with some numbers, perhaps you have something better... The Area Proxy draft is making everything L2 and using the L1 areas to redefine the advertised topology information to allow it to scale. Because everything is ultimately L2 nothing changes in the data plane to provide this transit. The flood reflector draft is keeping the L1-only abstraction so it has to provide for transit some other way. > So trying to extract the technical point you seem to be making inbetween all > that > > a) I see how you can try to have a mental model of "virtual links". What we > suggest are not virtual links (I implemented VL @ least once but it's so long > ago I forgot pretty much all the details so had to look stuff up :-) Virtual > links in OSPF were "magic bits on LSAs" that kind of computed "SPF > reachability through the area to change SPF" edge-to-edge and the > asynchronicity of all that flooding-being-tunnels-being-SPF was playing havoc > on us @ this time of 300MHz CPUs + frankly, the complexity of that was not > needed @ this time just as partition healing was never implemented in ISIS.. > That's why it never went anywhere much, my take, others may correct. Saying > "virtual links are bad" and "this is virtual links to me so it's bad" is > simply a "strawman fallacy" to me frankly. This draft suggests (but as I > wrote Bruno as answer to his fairly deep email) to run proper flooding over > proper tunnels (we run routing over tunnels all the time in all kind of > scenarios be it BGP proper or SD -WAN or overlays obviously) but if you choose FRs to be one hop away you can get away without any "L2 tunneled adjacencies", that's deployment choice. If you are not using tunnels, but are still trying to provide transit through the L1 area for L2, that is exactly what OSPF virtual links are doing. Part of making those work is the advertisement of reachability into the area from another, changes to router advertisements (to indicate if the area is transit capable), and changes to the SPF calculation to modify route choices based on whether they are based on virtual links or not. The complexity of OSPF virtual links is there to make them work, right? I'm not trying to make any argument, strawman or otherwise; I'm just trying to understand what is being proposed as a replacement for the full-mesh of tunnels. It's nice if it reduces to OSPF virtual links b/c we then have an example of how to actually implement it, and years of experience to understand it. If that highlights that getting the non-tunneled choice right isn't easy, well I guess that's important, right? > b) Generally we may seem a bit muddled between different types of "tunnels" > and "tunnels are bad" and "lots of tunnels". The draft talks about 2 types of > tunnels and it seemed to be written clearly enough to distinguish that easily > based on feedback I got so far > > i) control plane tunnels are proper L2 entities (again, if your FR is one hop > away from leafs then you don't need any tunneling but can run normal L2 > adjacencies which I hope are not too scary; whole thing is really equivalent > to BGP RR, do you put it in path, do you want to run multi-hop and how > confident are you in your lower level infra not dropping TCP "tunnels" under > you; every day's business since years really). So, no, there is no magic and > hidden complexity and whatever not, you may or may not use auto-discovery the > draft provides (you can just build something completely statically configured > and in fact several customers told me they'd prefer it that way just as they > don't auto-discover RRs normally) to build bunch of tunnels towards your 2-3 > FRs from your edges and you're in business after L2 adjacency comes up. Looks > like any old ISIS + some optional TLVs you can ignore that indicate for some > smart future folks to know it's a FR adjacnecy and not "real L2 adjaqcency". N o fork-lifting of whole cluster, no fork-lifting routers
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
Chris (addressing in WG member context you declared), I reply tersely since we will put more work into the draft once it's adopted (for which I think you saw a good amount of support in two threads already). I deferred from your email since the chain-terrasteam topology you're showing is simply not what we are dealing in any operational, successful networks today AFAIK frankly and I saw lots of "assume complexity" and "dislikes" in your email which I didn't read as technical arguments but mental attitudes. Likes or dislikes and assumptions are fine but we should probably focus on existing network/customer technical + operational arguments & requirements when building solutions and now what you or we like first. So trying to extract the technical point you seem to be making inbetween all that a) I see how you can try to have a mental model of "virtual links". What we suggest are not virtual links (I implemented VL @ least once but it's so long ago I forgot pretty much all the details so had to look stuff up :-) Virtual links in OSPF were "magic bits on LSAs" that kind of computed "SPF reachability through the area to change SPF" edge-to-edge and the asynchronicity of all that flooding-being-tunnels-being-SPF was playing havoc on us @ this time of 300MHz CPUs + frankly, the complexity of that was not needed @ this time just as partition healing was never implemented in ISIS. That's why it never went anywhere much, my take, others may correct. Saying "virtual links are bad" and "this is virtual links to me so it's bad" is simply a "strawman fallacy" to me frankly. This draft suggests (but as I wrote Bruno as answer to his fairly deep email) to run proper flooding over proper tunnels (we run routing over tunnels all the time in all kind of scenarios be it BGP proper or SD-WAN or overlays obviously) but if you choose FRs to be one hop away you can get away without any "L2 tunneled adjacencies", that's deployment choice. b) Generally we may seem a bit muddled between different types of "tunnels" and "tunnels are bad" and "lots of tunnels". The draft talks about 2 types of tunnels and it seemed to be written clearly enough to distinguish that easily based on feedback I got so far i) control plane tunnels are proper L2 entities (again, if your FR is one hop away from leafs then you don't need any tunneling but can run normal L2 adjacencies which I hope are not too scary; whole thing is really equivalent to BGP RR, do you put it in path, do you want to run multi-hop and how confident are you in your lower level infra not dropping TCP "tunnels" under you; every day's business since years really). So, no, there is no magic and hidden complexity and whatever not, you may or may not use auto-discovery the draft provides (you can just build something completely statically configured and in fact several customers told me they'd prefer it that way just as they don't auto-discover RRs normally) to build bunch of tunnels towards your 2-3 FRs from your edges and you're in business after L2 adjacency comes up. Looks like any old ISIS + some optional TLVs you can ignore that indicate for some smart future folks to know it's a FR adjacnecy and not "real L2 adjaqcency". No fork-lifting of whole cluster, no fork-lifting routers outside a cluster, no single point of failure under some fancy new name, barely any protocol changes (in fact I didn't see any proposal to run anything more minimal than that except maybe very simple version of TTZ which however has too many L2 adjacencies exposed @ any reasonable cluster size to really solve the problem of amount of L2 information sloshed around) ii) data plane tunnels. The draft basically explains that for the solution to work in a simple fashion,full-mesh of data forwarding tunnels can be established (which are NOT visible in L2) as shortcuts that allow to utilize all paths through L1 and that will work fine since it doesn't spill into L2. You want to run L1 adjacencies over those tunnels if you care whether they are up but you could do just BFD e.g. and use them as forwarding next-hops in the computation without them being visible in L1 ISIS. The other option is to not use such a data plane mesh and use reachability instead and we can explain that further in detail after adoption and we get more people talking through that etc. or you can look @ my preso @ last IETF where I kind of quickly ran through that (and it seemed relatively obvious to me how it works). In summary, we chose to do real work rather than polilsh optional points in individual drafts because, frankly, customers are not interested all that much often whether IETF WG feels like working on it while they have a pressing problem and need solutions in a timely manner. And AFAIR the chairs guided the group multiple times towards "ignore the problem, of no importance" and now with a certain urgency want to have everything @ the same time. So, I look forward to this draft being adopted as WG item given the support seen
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
Hi, Given that the flood reflector authors have asked the chairs to do a call for adoption, might someone from that group be able to talk to a couple of the points/questions from my mail? I think it would help me (and maybe others) in making informed responses to any adoption call. - Is this proposing to add "virtual links" to IS-IS (in addition to the flood reflector part)? - Is there a general-purpose non-complex non-tunneling solution possible? Thanks, Chris. [as WG member] > On Jun 6, 2020, at 7:18 AM, Christian Hopps wrote: > > [ all the following is as a WG member ] > > I've been thinking a lot about the proposed flooding reduction drafts > currently vying for adoption by the WG. I've been doing this thinking in the > context of wearing an operator hat (i.e., trying to leverage my prior > experience working for DT on Terastream). In the abstract the Terastream > architecture presented a good way to visualize and compare the benefits of > using one of these solutions w/o getting too complex. A simplified model of > this design can be seen as a horseshoe of horseshoes. the Major horseshoe is > L2 and the minor horseshoes are L1. Each L1 area has 2 L2 routers for > redundancy (I'll consider more though), and all L2 routers are full-mesh > connected to support that redundancy. > > Telco > > > > But let's map this to a more DC centric view (I guess?) where each L1 area > now has 10 L2 routers instead of 2 (i.e., 10%, but that could be change that > later if need be). > > Natural Design > > > > > Now for whatever reason some operators do not want to provision > high-bandwidth transit links between their L2 routers in their L1 areas. This > is critically important b/c otherwise you would simply use the above Natural > Design. I'd like to better understand why that isn't just the answer here. > > Anyway, forging ahead, here's what we get with unchanged IS-IS to support > "use everything for transit" > > All In > > > > So the 800 L2 LSPs, and the impact on flooding dynamics, are what these > drafts are trying to reduce or avoid. > > Area Proxy > > First I'll look at area proxy. This seems a fairly simple idea, basically > it's taking the now L1L2 areas and advertising them externally as a single > LSP so the impact is very similar to if they were L1 only. This maps fairly > closely to the Telco and Natural Design from above. Each L1 router in the > Telco design would have 100 LSPs The L12 routers would have 100 L1 + 16 L2 > LSP. In the Natural Design each L1 router has 100 L1 and each L12 router > would have 100 L1 and 80 L2. With Area Proxy each router has 100 L1 and 100 > "Inner L2 LSPs" and 80 "Outer LSPs" + 8 "Outer L2 LSPs" > > The key thing to note here is that if you double the number of areas you only > add to the Outside LSP and Proxy count just as it would scale in the Natural > Design, so going from 8 to 16 areas here adds 80 more "Outside LSPs" and 8 > more L2 Proxy LSPs for a total of 276 L2 LSPs even though you've added 800 > more routers to your network. > > > > > Flood Reflectors > > I'm less sure I understand this draft so please forgive me if I get this > wrong. After reading this draft a few times I believe it is basically > providing "L2 virtual links" over an L1 area between the area's L2 edge > routers and then using a "flood reflector" to reduce the number of required > "virtual links" by creating a hub-and-spoke topology of them. > > The draft does a bit of hand-waving at "judicious scoping of reachability" > being used in place of tunneling. I could guess at what this might mean, but > I shouldn't have to; the draft should spell it out so it can be judged as a > viable option or not. So, the only choice I see presented is to use a > full-mesh of L1 tunnels between the L12 edge routers. > > Anyway, here's a picture > > > > If, in fact, the draft is talking about adding virtual links to IS-IS I think > it should say that. There's a lot of experience in OSPF with virtual links > and some of the trouble they have caused. There's also important details in > making them work right in OSPF that doesn't seem to be in the current draft, > so it's not clear what actual level of complexity is going to be required to > make this all work, and importantly, if that would then be palatable. > > I also do not like the idea of all of these automatic "fowarding-tunnels". > They would be disjoint from the advertised "flood reflector" tunnel topology > (right?) -- it seems like a management/debugability nightmare to me, and I'm > curious which operators are keen to have a bunch of unadvertised tunnels > added to their IGP network. :) If the draft really can work in general w/o > the tunnels I would appreciate seeing those mechanics described rather than > just hinted at. > > The ideas are interesting for sure, but the draft doesn't seem fully fleshed > out, and I'm worried it'll be overly complex when it is. > >
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
> On Jun 6, 2020, at 5:15 PM, Tony Li wrote: > > > We’ve made some changes in the latest version of the draft. In the current > version, we require that each router in the area be L1L2. However, only one > LSP is advertised externally for each area. Thus, each router will see 100 > L1 LSPs, 100 L2 LSPs and 8 L2 Proxy LSPs. > > >> The key thing to note here is that if you double the number of areas you >> only add to the Outside LSP and Proxy count just as it would scale in the >> Natural Design, so going from 8 to 16 areas here adds 80 more "Outside LSPs" >> and 8 more L2 Proxy LSPs for a total of 276 L2 LSPs even though you've added >> 800 more routers to your network. > > > Doubling the number of areas would give you 16 L2 proxy LSPs, so you end up > going from 208 LSPs to 216. The key point here is that the database now > scales linearly with the number of areas. Ah yes, I missed that the Inside Edge Routers were masquerading using the Proxy System ID to the Outside Edge Routers. This does offer even better scaling. This also does directly address your use case where the Area edge is very large compared to old designs. Interestingly, if we consider Peter's pair of huge "R2" (L1L2 area edge) in Terastream (Telco Design), in effect you are creating a single Proxy LSP that continues to represent that concept topologically (although with a single "huge" router (Proxy LSP) rather than a pair of "R2"s), while allowing the huge multi-chassis physical routers to now be made up of N-many smaller physical routers. On the industry direction, it'll be interesting to see if using N-many physical routers with all the external L2/L3 interconnect required and the extra operational/management costs actually works better than the single large multi-chassis with it's internal fabric interconnect that could be managed as a single router. It would seem to be adding a lot of operational cost on to the operator. Terastream was specifically trying to avoid this extra operational cost, as capex wasn't the problem, the opex was. Not having to forklift upgrade these huge routers was critical to that design though, and it sounds like maybe we're giving up on the vendors delivering on this? Thanks, Chris. [as WG member] P.S. Curiously, I had "fixed" my diagram and text just prior to sending the email as originally I had it as 108 L2 LSPs and then 216 when doubling the network, maybe I had noticed subconsciously. :) ___ Lsr mailing list Lsr@ietf.org https://www.ietf.org/mailman/listinfo/lsr
Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.
Thanks Chris/Tony, I wish we’d have more of this kind of discussions on the list, discussing pro/cons of the solutions proposed! Have a great weekend! Regards, Jeff > On Jun 6, 2020, at 14:15, Tony Li wrote: > > > > Hi Chris, > > Thank you for your thoughtful comments. > > >> A simplified model of this design can be seen as a horseshoe of horseshoes. >> the Major horseshoe is L2 and the minor horseshoes are L1. Each L1 area has >> 2 L2 routers for redundancy (I'll consider more though), and all L2 routers >> are full-mesh connected to support that redundancy. >> >> Telco >> >> >> >> But let's map this to a more DC centric view (I guess?) where each L1 area >> now has 10 L2 routers instead of 2 (i.e., 10%, but that could be change that >> later if need be). > > > The key point here is not a DC centric view, but one of scale. The above > worked just fine back when we were terminating T1’s and E1’s. Now, PE > routers easily have terabit capacity. WIth the 100:1 ratio between the PE and > the L1L2 routers (Area Edge routers), the bandwidth requirements drive us to > petabit, multi-chassis routers. While these have been great fun to build, > operators are not happy with this direction. Forklift upgrades are necessary > with every silicon cycle. The complexity of the router has multiplied, the > blast radius is off the charts, and the premiums charged for these devices > are impressive. As a result, folks are trying to explore a ’scale-out’ > approach. Rather than 2 huge area edge routers, they would much rather be > able to scale out the area edge to be many routers. > > If you pursue this design direction, the first thing you observe is that you > can see is that you cannot afford to build a full-mesh of inter-area circuits > across all of the area edge routers. > > And if your network is a bit more geographically dispersed, you find that it > becomes inefficient to have even a full mesh at the area level. This forces > some areas to become transit. > > Between these two forces, we are compelled to push transit traffic through > the internals of an area. > >> Natural Design >> >> >> >> > > >> Now for whatever reason some operators do not want to provision >> high-bandwidth transit links between their L2 routers in their L1 areas. >> This is critically important b/c otherwise you would simply use the above >> Natural Design.. I'd like to better understand why that isn't just the >> answer here. > > > In this design, you’ve shown ten area edge routers. Yes, you could provision > a full mesh of links between them. The issue remains one of scalability and > uniformity. The number of ports per area edge router has to scale linearly > with the size of area edge and the overall number of links used for this > purpose is O(n^2). Combine this with the fact that any non-uniformity in the > traffic pattern and your full mesh ends up being congested and under-utilized > simultaneously. > > All is not lost, however. Charles Clos to the rescue. By structuring each > area as a Clos (or Benes) network (which my employer seems to insist on > spelling ‘leaf-spine’), you avoid this. I assume I don’t need to go into > details on this. > > >> Area Proxy >> >> First I'll look at area proxy. This seems a fairly simple idea, basically >> it's taking the now L1L2 areas and advertising them externally as a single >> LSP so the impact is very similar to if they were L1 only. This maps fairly >> closely to the Telco and Natural Design from above. Each L1 router in the >> Telco design would have 100 LSPs The L12 routers would have 100 L1 + 16 L2 >> LSP. In the Natural Design each L1 router has 100 L1 and each L12 router >> would have 100 L1 and 80 L2. With Area Proxy each router has 100 L1 and 100 >> "Inner L2 LSPs" and 80 "Outer LSPs" + 8 "Outer L2 LSPs" > > > We’ve made some changes in the latest version of the draft. In the current > version, we require that each router in the area be L1L2. However, only one > LSP is advertised externally for each area. Thus, each router will see 100 > L1 LSPs, 100 L2 LSPs and 8 L2 Proxy LSPs. > > >> The key thing to note here is that if you double the number of areas you >> only add to the Outside LSP and Proxy count just as it would scale in the >> Natural Design, so going from 8 to 16 areas here adds 80 more "Outside LSPs" >> and 8 more L2 Proxy LSPs for a total of 276 L2 LSPs even though you've added >> 800 more routers to your network. > > > Doubling the number of areas would give you 16 L2 proxy LSPs, so you end up > going from 208 LSPs to 216. The key point here is that the database now > scales linearly with the number of areas. > > Demo time: > > In a lab setup, we have an area of five routers. We have three L2 routers.. > The database on one of the pure L2 routers is just 5 entries (three L2, one > proxy LSP, one pseudonode): > > rtrmpls8>show isis data > > IS-IS Instance: Amun VRF:
