Re: BIRD / BGP-ORR experiences?
On 16 Apr 2020, at 22:35, Mark Tinka wrote: > > > >> On 15/Apr/20 19:07, Saku Ytti wrote: >> >> >> Don't run Cisco ORR RR or have IGP next-hops :/ > > Does it break NEXT_HOP=self in Cisco-land? > > Mark. We’re testing ORR at the moment as part of core upgrades (XRv on ESXi), and next-hop self not only works, it’s required for ORR to work properly I’ve not noticed any major issues with it yet but it’s still early days in terms of our deployment
RE: BIRD / BGP-ORR experiences?
On 15/Apr/20 17:59, Deepak Jain wrote: > Thanks for your input. How do you handle next-hops? Tunnels between all eBGP > speakers as if they were fully meshed as their potential next-hops? I should imagine NEXT_HOP=self still works in an ORR world, non :-)? The question resolves around if the external borders are not physically connected to each other, the next-hop won't matter once it hits the next router... without a tunnel or TE or similar, as far as I know, the packet destination is not overwritten, so there is the potential for IGP loops without a mechanism for consistency. Deepak
Re: BIRD / BGP-ORR experiences?
On 16/Apr/20 16:50, Saku Ytti wrote: > That would be in IGP, so that'll work. The other way that some people > do this, is that next-hop is CE, which is in iBGP, but recurses to > loop0. There are some TE reasons why people might do this, and it > would not work with Cisco ORR. Reasonably safe to say that if you are doing TE, you are not always satisfied with what the IGP thinks, and want to override it to some extent :-). Mark.
Re: BIRD / BGP-ORR experiences?
On Thu, 16 Apr 2020 at 15:36, Mark Tinka wrote: > Does it break NEXT_HOP=self in Cisco-land? That would be in IGP, so that'll work. The other way that some people do this, is that next-hop is CE, which is in iBGP, but recurses to loop0. There are some TE reasons why people might do this, and it would not work with Cisco ORR. -- ++ytti
Re: BIRD / BGP-ORR experiences?
On 16/Apr/20 15:17, Chris Jones wrote: > We’re testing ORR at the moment as part of core upgrades (XRv on ESXi), and > next-hop self not only works, it’s required for ORR to work properly Yes, that would be my simple 1+1, as it's all about optimizing for the best IGP exit for far-away nodes. Mark.
Re: BIRD / BGP-ORR experiences?
On 15/Apr/20 19:07, Saku Ytti wrote: > > Don't run Cisco ORR RR or have IGP next-hops :/ Does it break NEXT_HOP=self in Cisco-land? Mark.
Re: BIRD / BGP-ORR experiences?
On 15/Apr/20 17:59, Deepak Jain wrote: > Thanks for your input. How do you handle next-hops? Tunnels between all eBGP > speakers as if they were fully meshed as their potential next-hops? I should imagine NEXT_HOP=self still works in an ORR world, non :-)? Mark.
Re: BIRD / BGP-ORR experiences?
On 15/Apr/20 17:51, Deepak Jain wrote: > > How is this approach working for you? It's working out beautifully, since 2014. We wanted ORR at the time, but it was immature, so this was our only option. Yes, it's an old school approach, but it's simple, so we don't have to enable any trickery. > We were considering BGP ORR on bare metal (maybe a VM if we can get OSPF > into the VM) and putting a primary RR at each eBGP node (physical site) and > at least one link to a backup RR at a different site, perhaps enabling BGP > ORR for that to minimize suboptimal paths. > > Adding a VM or a server node for this function is hardly the technical > challenge it used to be with so many linux-based white box switches and > things running around nowadays. Well, our RR's run on ESXi (Cisco CSR1000v), but I'm sure your favorite hypervisor will work just fine. Your main concern is whether you want to run an established code base (Cisco, Juniper, Nokia, e.t.c.), or if you want to run something more open source. We chose the former considering our iBGP is a critical piece of infrastructure. Mark.
Re: BIRD / BGP-ORR experiences?
On Wed, 15 Apr 2020 at 19:00, Deepak Jain wrote: > Thanks for your input. How do you handle next-hops? Tunnels between all eBGP > speakers as if they were fully meshed as their potential next-hops? Don't run Cisco ORR RR or have IGP next-hops :/ -- ++ytti
RE: BIRD / BGP-ORR experiences?
> Do we even like BGP ORR? I like it, I think ADD-PATH and ORR are mandatory features in modern RR infra. However proper interaction between them may not exist in every implementation. Basically you want a) send all ECMPable paths b) send one backup path This will lead to superior to full-mesh by every reasonable metric: - smaller rib (no routes that you don't need, but all the routes you care about) - redundancy (one iBGP down, is not customer outage) - less state changes, less code stress ORR is not an RFC and there are some open questions. What to reflect, when next-hop is not in IGP? Do we hope that receiver would recurse to the same IGP next-hop? Juniper makes this assumption, which to me is decidedly the common case. Cisco makes no assumption and doesn't reflect if next-hop is not in IGP, but as I understand they will fix to the same assumption as Juniper. - Thanks for your input. How do you handle next-hops? Tunnels between all eBGP speakers as if they were fully meshed as their potential next-hops? Deepak
RE: BIRD / BGP-ORR experiences?
> Nice to hear ORR has come a long way that it's somewhat usable. It is usable, we have taken it even a step forward: - virtualized RR - add-path - ORR - IGP topology to RR via BGP-LS so we don't have to extend ISIS to VMs (there are some issues with SR-IOV) -- That sounds pretty exciting and its good to see traditional router vendors (embedded) leading the way on features. I don't exactly understand how add-path plus ORR works, I figured you'd only use one or the other, but that just shows you I haven't been up on this in a while. Deepak
RE: BIRD / BGP-ORR experiences?
On 15/Apr/20 13:36, Saku Ytti wrote: > > ORR is not an RFC and there are some open questions. What to reflect, > when next-hop is not in IGP? Do we hope that receiver would recurse to > the same IGP next-hop? Juniper makes this assumption, which to me is > decidedly the common case. Cisco makes no assumption and doesn't > reflect if next-hop is not in IGP, but as I understand they will fix > to the same assumption as Juniper. When we wanted this bad, it wasn't ready (2014), so we ended up deploying an RR in every major PoP, since that wasn't too costly (there was a time when a network I knew of used a Juniper M120 as an RR). Nice to hear ORR has come a long way that it's somewhat usable. How is this approach working for you? We were considering BGP ORR on bare metal (maybe a VM if we can get OSPF into the VM) and putting a primary RR at each eBGP node (physical site) and at least one link to a backup RR at a different site, perhaps enabling BGP ORR for that to minimize suboptimal paths. Adding a VM or a server node for this function is hardly the technical challenge it used to be with so many linux-based white box switches and things running around nowadays. Thanks! Deepak
Re: BIRD / BGP-ORR experiences?
On 15/Apr/20 14:53, Tarko Tikan wrote: > > > Well ISIS works with bridge but we like to keep our virtualized NFs > simple so KVM hosts have dedicated 10G port for NFs (that connects > directly to a metro node) and we run SR-IOV. Got you. Mark.
Re: BIRD / BGP-ORR experiences?
hey, I was asking in relation to your IS-IS + SR-IOV issues. Well ISIS works with bridge but we like to keep our virtualized NFs simple so KVM hosts have dedicated 10G port for NFs (that connects directly to a metro node) and we run SR-IOV. -- tarko
Re: BIRD / BGP-ORR experiences?
On 15/Apr/20 14:40, Tarko Tikan wrote: > > No, we had RR function inline on ASBRs. > > To be clear, our RRs are not BIRD but Nokia VSRs. I was asking in relation to your IS-IS + SR-IOV issues. Mark.
Re: BIRD / BGP-ORR experiences?
hey, Were you previously running IS-IS on a UNIX/Linux system running in a VM? No, we had RR function inline on ASBRs. To be clear, our RRs are not BIRD but Nokia VSRs. -- tarko
Re: BIRD / BGP-ORR experiences?
On 15/Apr/20 14:28, Tarko Tikan wrote: > > It is usable, we have taken it even a step forward: > > - virtualized RR > - add-path > - ORR > - IGP topology to RR via BGP-LS so we don't have to extend ISIS to VMs > (there are some issues with SR-IOV) Awesome! Never been a fan of Add-Path or Diverse-Path, but good to know it's working well for you and others. Were you previously running IS-IS on a UNIX/Linux system running in a VM? Mark.
Re: BIRD / BGP-ORR experiences?
hey, Nice to hear ORR has come a long way that it's somewhat usable. It is usable, we have taken it even a step forward: - virtualized RR - add-path - ORR - IGP topology to RR via BGP-LS so we don't have to extend ISIS to VMs (there are some issues with SR-IOV) -- tarko
Re: BIRD / BGP-ORR experiences?
On 15/Apr/20 13:36, Saku Ytti wrote: > > ORR is not an RFC and there are some open questions. What to reflect, > when next-hop is not in IGP? Do we hope that receiver would recurse to > the same IGP next-hop? Juniper makes this assumption, which to me is > decidedly the common case. Cisco makes no assumption and doesn't > reflect if next-hop is not in IGP, but as I understand they will fix > to the same assumption as Juniper. When we wanted this bad, it wasn't ready (2014), so we ended up deploying an RR in every major PoP, since that wasn't too costly (there was a time when a network I knew of used a Juniper M120 as an RR). Nice to hear ORR has come a long way that it's somewhat usable. Mark.
Re: BIRD / BGP-ORR experiences?
On Wed, 15 Apr 2020 at 10:56, Deepak Jain wrote: Hey, > Do we even like BGP ORR? I like it, I think ADD-PATH and ORR are mandatory features in modern RR infra. However proper interaction between them may not exist in every implementation. Basically you want a) send all ECMPable paths b) send one backup path This will lead to superior to full-mesh by every reasonable metric: - smaller rib (no routes that you don't need, but all the routes you care about) - redundancy (one iBGP down, is not customer outage) - less state changes, less code stress ORR is not an RFC and there are some open questions. What to reflect, when next-hop is not in IGP? Do we hope that receiver would recurse to the same IGP next-hop? Juniper makes this assumption, which to me is decidedly the common case. Cisco makes no assumption and doesn't reflect if next-hop is not in IGP, but as I understand they will fix to the same assumption as Juniper. -- ++ytti
Re: BIRD / BGP-ORR experiences?
Deepak Jain wrote on 15/04/2020 08:52: Do we even like BGP ORR? yes, but it needs to be planned carefully. Nick
BIRD / BGP-ORR experiences?
Thinking about setting up BGP-ORR on some BIRD VMs (https://bird.network.cz) for lab purposes, I'm sure its more than sufficient. Does anyone use these in production? Any thoughts, experiences, caveats? Do we even like BGP ORR? Thanks in advance, Deepak
