Re: Best way to have redundancy announcing on separate routers

[Christopher Morrow]

On Thu, Jan 2, 2025 at 3:24 PM Tom Beecher  wrote:
>
> I'm not saying I *would* do it, or you *should* do it, I'm just answering the 
> questions being asked. :)
>

oh! fair enough... "that is a ton of complexity" still applies, or at
least for my view that's more complexity than: "always I announce to
all, with some metrics to make me seem there but just farther away on
link X"

> On Thu, Jan 2, 2025 at 3:21 PM Christopher Morrow  
> wrote:
>>
>> On Thu, Jan 2, 2025 at 2:55 PM Tom Beecher  wrote:
>> >
>> > Jean-
>> >
>> > Thanks. Many BGP implementations have the ability to do conditional 
>> > advertisements, where you announce (or don't) a set of prefixes based on 
>> > the presents (or absence) of other routes. I don't think quagga does 
>> > natively, and not sure if VyOS has added that on.
>> >
>> > Conceptually, you want to be doing "announce these prefixes from this 
>> > router only if I don't see routes from the upstream on the other router". 
>> > The 'safest' way is probably to just monitor default, but it depends on 
>> > your environment.
>> >
>>
>> That sort of thing seems like extra complexity, no?
>> If the 2 internal routers have iBGP and you are fairly sure that you
>> won't lose that path/view you should be able to just announce
>> the same prefixes to both ISP peerings and possibly add some
>> metric-equivalent data to distance one link vs the other, no?
>> (common metric for this is the as-path, add your as N times, where N
>> is <10 and > 2 probably?)
>>
>> how exact do you want your split here to be jfranco ? (is 'mostly
>> everything over PRIMARY with some over SECONDARY' ok?)
>>
>> > On Fri, Dec 27, 2024 at 6:09 PM Jean Franco  wrote:
>> >>
>> >> Hi Tom,
>> >> This is exactly what I was planning.
>> >> I'm announcing a block via ISP1 and another set of blocks via ISP2, and 
>> >> have iBGP running between them.
>> >>
>> >> Thanks a lot!!
>> >>
>> >> Best regards,
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher  wrote:
>> >>>
>> >>> Jean-
>> >>>
>> >>> Yeah, don't worry about people complaining.
>> >>>
>> >>> Is this an accurate description of what you are trying to achieve?
>> >>>
>> >>> - Have 2 different sets of prefixes that you announce. Set A via 
>> >>> router1/ISP1 , Set B via router2/ISP2
>> >>> - If BGP to one of your ISPs goes down, start announcing those prefixes 
>> >>> to the other ISP. ( Example, if ISP2 goes down, start announcing prefix 
>> >>> Set B over ISP1 )
>> >>>
>> >>> On Thu, Dec 26, 2024 at 8:16 AM Jean Franco  wrote:
>> 
>>  Hi guys,
>>  I've been on the list for as long as I cannot even remember.
>>  So just you know, I'm not new at this.
>> 
>>  This is no easy task, that's why I came here looking for help.
>>  I'm sorry if I brought anguish to the experts on the list!
>>  I thought I could bring something that someone may have experienced 
>>  before.
>> 
>>  I haven't solved this yet, but at least I've received some valuable 
>>  suggestions and I Thank you!
>> 
>>  About all the details of the connections, numbers of peerings, PNI's 
>>  and IXP's I have left them out, since I figured this additional 
>>  information could make things worse.
>> 
>>  ISP 1  20KM20KM ISP2
>> 
>>  The ISP connections are all 10G.
>>  I don't believe these routers are DFZ capable.
>>  All the routers are well capable and already receive the full routes.
>>  The connections between these routers are 40G.
>> 
>>  Best regards,
>> 
>> 
>>  On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields  
>>  wrote:
>> >
>> > On 12/25/24 6:18 PM, Randy Bush wrote:
>> > > where does one go for is-is help?  the mtu issie can be painful!!!
>> >
>> > I think here would be good too.  I recently had to do this between a 
>> > Cisco
>> > 3945e and a Juniper, and from my unrevised notes:
>> >
>> > vlan {
>> >   unit 405 {
>> > family iso {
>> > # holy shit this is important.  CISCO and Juniper will not talk 
>> > unless the
>> > MTU is set
>> > mtu 1492;
>> >   }
>> >}
>> > }
>> >
>> > :-)
>> >
>> > --
>> > Bryan Fields
>> >
>> > 727-409-1194 - Voice
>> > http://bryanfields.net

Re: Best way to have redundancy announcing on separate routers

[Tom Beecher]

I'm not saying I *would* do it, or you *should* do it, I'm just answering
the questions being asked. :)

On Thu, Jan 2, 2025 at 3:21 PM Christopher Morrow 
wrote:

> On Thu, Jan 2, 2025 at 2:55 PM Tom Beecher  wrote:
> >
> > Jean-
> >
> > Thanks. Many BGP implementations have the ability to do conditional
> advertisements, where you announce (or don't) a set of prefixes based on
> the presents (or absence) of other routes. I don't think quagga does
> natively, and not sure if VyOS has added that on.
> >
> > Conceptually, you want to be doing "announce these prefixes from this
> router only if I don't see routes from the upstream on the other router".
> The 'safest' way is probably to just monitor default, but it depends on
> your environment.
> >
>
> That sort of thing seems like extra complexity, no?
> If the 2 internal routers have iBGP and you are fairly sure that you
> won't lose that path/view you should be able to just announce
> the same prefixes to both ISP peerings and possibly add some
> metric-equivalent data to distance one link vs the other, no?
> (common metric for this is the as-path, add your as N times, where N
> is <10 and > 2 probably?)
>
> how exact do you want your split here to be jfranco ? (is 'mostly
> everything over PRIMARY with some over SECONDARY' ok?)
>
> > On Fri, Dec 27, 2024 at 6:09 PM Jean Franco 
> wrote:
> >>
> >> Hi Tom,
> >> This is exactly what I was planning.
> >> I'm announcing a block via ISP1 and another set of blocks via ISP2, and
> have iBGP running between them.
> >>
> >> Thanks a lot!!
> >>
> >> Best regards,
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher  wrote:
> >>>
> >>> Jean-
> >>>
> >>> Yeah, don't worry about people complaining.
> >>>
> >>> Is this an accurate description of what you are trying to achieve?
> >>>
> >>> - Have 2 different sets of prefixes that you announce. Set A via
> router1/ISP1 , Set B via router2/ISP2
> >>> - If BGP to one of your ISPs goes down, start announcing those
> prefixes to the other ISP. ( Example, if ISP2 goes down, start announcing
> prefix Set B over ISP1 )
> >>>
> >>> On Thu, Dec 26, 2024 at 8:16 AM Jean Franco 
> wrote:
> 
>  Hi guys,
>  I've been on the list for as long as I cannot even remember.
>  So just you know, I'm not new at this.
> 
>  This is no easy task, that's why I came here looking for help.
>  I'm sorry if I brought anguish to the experts on the list!
>  I thought I could bring something that someone may have experienced
> before.
> 
>  I haven't solved this yet, but at least I've received some valuable
> suggestions and I Thank you!
> 
>  About all the details of the connections, numbers of peerings, PNI's
> and IXP's I have left them out, since I figured this additional information
> could make things worse.
> 
>  ISP 1  20KM20KM ISP2
> 
>  The ISP connections are all 10G.
>  I don't believe these routers are DFZ capable.
>  All the routers are well capable and already receive the full routes.
>  The connections between these routers are 40G.
> 
>  Best regards,
> 
> 
>  On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields 
> wrote:
> >
> > On 12/25/24 6:18 PM, Randy Bush wrote:
> > > where does one go for is-is help?  the mtu issie can be painful!!!
> >
> > I think here would be good too.  I recently had to do this between a
> Cisco
> > 3945e and a Juniper, and from my unrevised notes:
> >
> > vlan {
> >   unit 405 {
> > family iso {
> > # holy shit this is important.  CISCO and Juniper will not talk
> unless the
> > MTU is set
> > mtu 1492;
> >   }
> >}
> > }
> >
> > :-)
> >
> > --
> > Bryan Fields
> >
> > 727-409-1194 - Voice
> > http://bryanfields.net
>

Re: Best way to have redundancy announcing on separate routers

[Christopher Morrow]

On Thu, Jan 2, 2025 at 2:55 PM Tom Beecher  wrote:
>
> Jean-
>
> Thanks. Many BGP implementations have the ability to do conditional 
> advertisements, where you announce (or don't) a set of prefixes based on the 
> presents (or absence) of other routes. I don't think quagga does natively, 
> and not sure if VyOS has added that on.
>
> Conceptually, you want to be doing "announce these prefixes from this router 
> only if I don't see routes from the upstream on the other router". The 
> 'safest' way is probably to just monitor default, but it depends on your 
> environment.
>

That sort of thing seems like extra complexity, no?
If the 2 internal routers have iBGP and you are fairly sure that you
won't lose that path/view you should be able to just announce
the same prefixes to both ISP peerings and possibly add some
metric-equivalent data to distance one link vs the other, no?
(common metric for this is the as-path, add your as N times, where N
is <10 and > 2 probably?)

how exact do you want your split here to be jfranco ? (is 'mostly
everything over PRIMARY with some over SECONDARY' ok?)

> On Fri, Dec 27, 2024 at 6:09 PM Jean Franco  wrote:
>>
>> Hi Tom,
>> This is exactly what I was planning.
>> I'm announcing a block via ISP1 and another set of blocks via ISP2, and have 
>> iBGP running between them.
>>
>> Thanks a lot!!
>>
>> Best regards,
>>
>>
>>
>>
>>
>>
>>
>> On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher  wrote:
>>>
>>> Jean-
>>>
>>> Yeah, don't worry about people complaining.
>>>
>>> Is this an accurate description of what you are trying to achieve?
>>>
>>> - Have 2 different sets of prefixes that you announce. Set A via 
>>> router1/ISP1 , Set B via router2/ISP2
>>> - If BGP to one of your ISPs goes down, start announcing those prefixes to 
>>> the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B 
>>> over ISP1 )
>>>
>>> On Thu, Dec 26, 2024 at 8:16 AM Jean Franco  wrote:

 Hi guys,
 I've been on the list for as long as I cannot even remember.
 So just you know, I'm not new at this.

 This is no easy task, that's why I came here looking for help.
 I'm sorry if I brought anguish to the experts on the list!
 I thought I could bring something that someone may have experienced before.

 I haven't solved this yet, but at least I've received some valuable 
 suggestions and I Thank you!

 About all the details of the connections, numbers of peerings, PNI's and 
 IXP's I have left them out, since I figured this additional information 
 could make things worse.

 ISP 1  20KM20KM ISP2

 The ISP connections are all 10G.
 I don't believe these routers are DFZ capable.
 All the routers are well capable and already receive the full routes.
 The connections between these routers are 40G.

 Best regards,


 On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields  
 wrote:
>
> On 12/25/24 6:18 PM, Randy Bush wrote:
> > where does one go for is-is help?  the mtu issie can be painful!!!
>
> I think here would be good too.  I recently had to do this between a Cisco
> 3945e and a Juniper, and from my unrevised notes:
>
> vlan {
>   unit 405 {
> family iso {
> # holy shit this is important.  CISCO and Juniper will not talk 
> unless the
> MTU is set
> mtu 1492;
>   }
>}
> }
>
> :-)
>
> --
> Bryan Fields
>
> 727-409-1194 - Voice
> http://bryanfields.net

Re: Best way to have redundancy announcing on separate routers

[Tom Beecher]

Jean-

Thanks. Many BGP implementations have the ability to do conditional
advertisements, where you announce (or don't) a set of prefixes based on
the presents (or absence) of other routes. I don't think quagga does
natively, and not sure if VyOS has added that on.

Conceptually, you want to be doing "announce these prefixes from this
router only if I don't see routes from the upstream on the other router".
The 'safest' way is probably to just monitor default, but it depends on
your environment.




On Fri, Dec 27, 2024 at 6:09 PM Jean Franco  wrote:

> Hi Tom,
> This is exactly what I was planning.
> I'm announcing a block via ISP1 and another set of blocks via ISP2, and
> have iBGP running between them.
>
> Thanks a lot!!
>
> Best regards,
>
>
>
>
>
>
>
> On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher  wrote:
>
>> Jean-
>>
>> Yeah, don't worry about people complaining.
>>
>> Is this an accurate description of what you are trying to achieve?
>>
>> - Have 2 different sets of prefixes that you announce. Set A via
>> router1/ISP1 , Set B via router2/ISP2
>> - If BGP to one of your ISPs goes down, start announcing those prefixes
>> to the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set
>> B over ISP1 )
>>
>> On Thu, Dec 26, 2024 at 8:16 AM Jean Franco  wrote:
>>
>>> Hi guys,
>>> I've been on the list for as long as I cannot even remember.
>>> So just you know, I'm not new at this.
>>>
>>> This is no easy task, that's why I came here looking for help.
>>> I'm sorry if I brought anguish to the experts on the list!
>>> I thought I could bring something that someone may have
>>> experienced before.
>>>
>>> I haven't solved this yet, but at least I've received some
>>> valuable suggestions and I Thank you!
>>>
>>> About all the details of the connections, numbers of peerings, PNI's and
>>> IXP's I have left them out, since I figured this additional information
>>> could make things worse.
>>>
>>> ISP 1  20KM20KM ISP2
>>>
>>> The ISP connections are all 10G.
>>> I don't believe these routers are DFZ capable.
>>> All the routers are well capable and already receive the full routes.
>>> The connections between these routers are 40G.
>>>
>>> Best regards,
>>>
>>>
>>> On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields 
>>> wrote:
>>>
 On 12/25/24 6:18 PM, Randy Bush wrote:
 > where does one go for is-is help?  the mtu issie can be painful!!!

 I think here would be good too.  I recently had to do this between a
 Cisco
 3945e and a Juniper, and from my unrevised notes:

 vlan {
   unit 405 {
 family iso {
 # holy shit this is important.  CISCO and Juniper will not talk
 unless the
 MTU is set
 mtu 1492;
   }
}
 }

 :-)

 --
 Bryan Fields

 727-409-1194 - Voice
 http://bryanfields.net

>>>

Re: Best way to have redundancy announcing on separate routers

[Sam Roche]

We have a similiar setup with each of our BGP routers connected to different 
ISPs. Each ISP sends us full tables.

The simplest method we came up with for outbound connection redundancy was to 
have router 1 use a default route to router 2, and router 2 use a default route 
to router 1. We don't redistribute those static defaults or our learned BGP 
routes to the other routers.

During normal times, that default route is never used, as there are more 
specific routes learned via BGP.

If router 1 loses its connection to ISP 1 for any reason, it simply pushes 
traffic over to router 2 as it's routing table only has the default and any 
learned OSPF routes from our PE boxes. Outbound traffic from router 1 then goes 
out through my BGP router 2, then ISP2 based on its learned BGP routes.

Redundancy for inbound traffic just works via BGP magic 🙂.

10+ years with this setup and no major issues that we know of. We do have more 
config & peers on the routers, but I've omitted that for simplicity.

ISP 1(announcing full tables) <--->     <-->ISP2 (announcing full tables)

Happy New Year everyone!!


Sam Roche

Manager, Network Operations

[Logo  Description automatically generated]

196 Taylor Road, Bracebridge, ON P1L 1J9

Support: supp...@lakelandnetworks.com<mailto:supp...@lakelandnetworks.com>  
705-640-0556 TF: 1-844-444-4249

Direct: sro...@lakelandnetworks.com<mailto:sro...@lakelandnetworks.com>  
705-640-0086 | https://www.lakelandnetworks.com/faqs/

Lakeland Networks<http://www.lakelandnetworks.com/>




From: NANOG  on behalf of 
Jean Franco 
Sent: Friday, December 27, 2024 6:09 PM
To: Tom Beecher 
Cc: Bryan Fields ; nanog@nanog.org 
Subject: Re: Best way to have redundancy announcing on separate routers

Hi Tom,
This is exactly what I was planning.
I'm announcing a block via ISP1 and another set of blocks via ISP2, and have 
iBGP running between them.

Thanks a lot!!

Best regards,







On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher 
mailto:beec...@beecher.cc>> wrote:
Jean-

Yeah, don't worry about people complaining.

Is this an accurate description of what you are trying to achieve?

- Have 2 different sets of prefixes that you announce. Set A via router1/ISP1 , 
Set B via router2/ISP2
- If BGP to one of your ISPs goes down, start announcing those prefixes to the 
other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B over 
ISP1 )

On Thu, Dec 26, 2024 at 8:16 AM Jean Franco 
mailto:jfra...@maila.inf.br>> wrote:
Hi guys,
I've been on the list for as long as I cannot even remember.
So just you know, I'm not new at this.

This is no easy task, that's why I came here looking for help.
I'm sorry if I brought anguish to the experts on the list!
I thought I could bring something that someone may have experienced before.

I haven't solved this yet, but at least I've received some valuable suggestions 
and I Thank you!

About all the details of the connections, numbers of peerings, PNI's and IXP's 
I have left them out, since I figured this additional information could make 
things worse.

ISP 1  20KM20KM ISP2

The ISP connections are all 10G.
I don't believe these routers are DFZ capable.
All the routers are well capable and already receive the full routes.
The connections between these routers are 40G.

Best regards,


On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields 
mailto:br...@bryanfields.net>> wrote:
On 12/25/24 6:18 PM, Randy Bush wrote:
> where does one go for is-is help?  the mtu issie can be painful!!!

I think here would be good too.  I recently had to do this between a Cisco
3945e and a Juniper, and from my unrevised notes:

vlan {
  unit 405 {
family iso {
# holy shit this is important.  CISCO and Juniper will not talk unless the
MTU is set
mtu 1492;
  }
   }
}

:-)

--
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: Best way to have redundancy announcing on separate routers

[Jean Franco]

Hi Tom,
This is exactly what I was planning.
I'm announcing a block via ISP1 and another set of blocks via ISP2, and
have iBGP running between them.

Thanks a lot!!

Best regards,







On Fri, Dec 27, 2024 at 1:00 PM Tom Beecher  wrote:

> Jean-
>
> Yeah, don't worry about people complaining.
>
> Is this an accurate description of what you are trying to achieve?
>
> - Have 2 different sets of prefixes that you announce. Set A via
> router1/ISP1 , Set B via router2/ISP2
> - If BGP to one of your ISPs goes down, start announcing those prefixes to
> the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B
> over ISP1 )
>
> On Thu, Dec 26, 2024 at 8:16 AM Jean Franco  wrote:
>
>> Hi guys,
>> I've been on the list for as long as I cannot even remember.
>> So just you know, I'm not new at this.
>>
>> This is no easy task, that's why I came here looking for help.
>> I'm sorry if I brought anguish to the experts on the list!
>> I thought I could bring something that someone may have
>> experienced before.
>>
>> I haven't solved this yet, but at least I've received some
>> valuable suggestions and I Thank you!
>>
>> About all the details of the connections, numbers of peerings, PNI's and
>> IXP's I have left them out, since I figured this additional information
>> could make things worse.
>>
>> ISP 1  20KM20KM ISP2
>>
>> The ISP connections are all 10G.
>> I don't believe these routers are DFZ capable.
>> All the routers are well capable and already receive the full routes.
>> The connections between these routers are 40G.
>>
>> Best regards,
>>
>>
>> On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields 
>> wrote:
>>
>>> On 12/25/24 6:18 PM, Randy Bush wrote:
>>> > where does one go for is-is help?  the mtu issie can be painful!!!
>>>
>>> I think here would be good too.  I recently had to do this between a
>>> Cisco
>>> 3945e and a Juniper, and from my unrevised notes:
>>>
>>> vlan {
>>>   unit 405 {
>>> family iso {
>>> # holy shit this is important.  CISCO and Juniper will not talk
>>> unless the
>>> MTU is set
>>> mtu 1492;
>>>   }
>>>}
>>> }
>>>
>>> :-)
>>>
>>> --
>>> Bryan Fields
>>>
>>> 727-409-1194 - Voice
>>> http://bryanfields.net
>>>
>>

Re: Best way to have redundancy announcing on separate routers

[Tom Beecher]

Jean-

Yeah, don't worry about people complaining.

Is this an accurate description of what you are trying to achieve?

- Have 2 different sets of prefixes that you announce. Set A via
router1/ISP1 , Set B via router2/ISP2
- If BGP to one of your ISPs goes down, start announcing those prefixes to
the other ISP. ( Example, if ISP2 goes down, start announcing prefix Set B
over ISP1 )

On Thu, Dec 26, 2024 at 8:16 AM Jean Franco  wrote:

> Hi guys,
> I've been on the list for as long as I cannot even remember.
> So just you know, I'm not new at this.
>
> This is no easy task, that's why I came here looking for help.
> I'm sorry if I brought anguish to the experts on the list!
> I thought I could bring something that someone may have experienced before.
>
> I haven't solved this yet, but at least I've received some
> valuable suggestions and I Thank you!
>
> About all the details of the connections, numbers of peerings, PNI's and
> IXP's I have left them out, since I figured this additional information
> could make things worse.
>
> ISP 1  20KM20KM ISP2
>
> The ISP connections are all 10G.
> I don't believe these routers are DFZ capable.
> All the routers are well capable and already receive the full routes.
> The connections between these routers are 40G.
>
> Best regards,
>
>
> On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields 
> wrote:
>
>> On 12/25/24 6:18 PM, Randy Bush wrote:
>> > where does one go for is-is help?  the mtu issie can be painful!!!
>>
>> I think here would be good too.  I recently had to do this between a Cisco
>> 3945e and a Juniper, and from my unrevised notes:
>>
>> vlan {
>>   unit 405 {
>> family iso {
>> # holy shit this is important.  CISCO and Juniper will not talk
>> unless the
>> MTU is set
>> mtu 1492;
>>   }
>>}
>> }
>>
>> :-)
>>
>> --
>> Bryan Fields
>>
>> 727-409-1194 - Voice
>> http://bryanfields.net
>>
>

Re: Best way to have redundancy announcing on separate routers

[Saku Ytti]

On Thu, 26 Dec 2024 at 17:51, Bryan Fields  wrote:

> I believe it was that one side was vlan tagged and the other wasn't or
> something to the effect of how the ISO MTU was calculated in the classic-IOS
> version vs. Junos.  As this was for less than a carrier grade setup (tactical
> deployment during an hurricane) I didn't have time to get deep into it.  A
> 3945e isn't exactly current anyway.  I think the other juniper involved was a
> 4500, so kind of in the same age as the cisco.

My money is that doing IPv4 ping with DF-bit on, you'd see loss from
one side with lower MTU. Likely the Junos side had a lower MTU, as you
are supposed to increase the IFD MTU by 4 if you add a VLAN encap,
which you do not need to do in 3945. However, personally if you're
going to have ifl's, I would set the ifd to maximum, and set ifls at
1500, so that when one ifl one day wants larger mtu, you don't have to
break the world.





-- 
  ++ytti

Re: Best way to have redundancy announcing on separate routers

[Bryan Fields]

On 12/26/24 8:24 AM, Saku Ytti wrote:
> On Thu, 26 Dec 2024 at 05:55, Bryan Fields  wrote:
> 
>> vlan {
>>   unit 405 {
>> family iso {
>> # holy shit this is important.  CISCO and Juniper will not talk unless 
>> the
>> MTU is set
>> mtu 1492;
>>   }
>>}
>> }
> This is almost certainly the wrong solution to a real problem. Real
> problem being, you have MTU mismatch, and the wrong solution is you
> solve it for one ifa. You almost certainly want the MTU to be fixed in
> ifl or maybe even ifd.

I believe it was that one side was vlan tagged and the other wasn't or
something to the effect of how the ISO MTU was calculated in the classic-IOS
version vs. Junos.  As this was for less than a carrier grade setup (tactical
deployment during an hurricane) I didn't have time to get deep into it.  A
3945e isn't exactly current anyway.  I think the other juniper involved was a
4500, so kind of in the same age as the cisco.

-- 
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: Best way to have redundancy announcing on separate routers

[Saku Ytti]

On Thu, 26 Dec 2024 at 05:55, Bryan Fields  wrote:

> vlan {
>   unit 405 {
> family iso {
> # holy shit this is important.  CISCO and Juniper will not talk unless the
> MTU is set
> mtu 1492;
>   }
>}
> }

This is almost certainly the wrong solution to a real problem. Real
problem being, you have MTU mismatch, and the wrong solution is you
solve it for one ifa. You almost certainly want the MTU to be fixed in
ifl or maybe even ifd.

-- 
  ++ytti

Re: Best way to have redundancy announcing on separate routers

[Jean Franco]

Hi guys,
I've been on the list for as long as I cannot even remember.
So just you know, I'm not new at this.

This is no easy task, that's why I came here looking for help.
I'm sorry if I brought anguish to the experts on the list!
I thought I could bring something that someone may have experienced before.

I haven't solved this yet, but at least I've received some
valuable suggestions and I Thank you!

About all the details of the connections, numbers of peerings, PNI's and
IXP's I have left them out, since I figured this additional information
could make things worse.

ISP 1  20KM20KM ISP2

The ISP connections are all 10G.
I don't believe these routers are DFZ capable.
All the routers are well capable and already receive the full routes.
The connections between these routers are 40G.

Best regards,


On Thu, Dec 26, 2024 at 12:53 AM Bryan Fields  wrote:

> On 12/25/24 6:18 PM, Randy Bush wrote:
> > where does one go for is-is help?  the mtu issie can be painful!!!
>
> I think here would be good too.  I recently had to do this between a Cisco
> 3945e and a Juniper, and from my unrevised notes:
>
> vlan {
>   unit 405 {
> family iso {
> # holy shit this is important.  CISCO and Juniper will not talk unless
> the
> MTU is set
> mtu 1492;
>   }
>}
> }
>
> :-)
>
> --
> Bryan Fields
>
> 727-409-1194 - Voice
> http://bryanfields.net
>

Re: Best way to have redundancy announcing on separate routers

[Bryan Fields]

On 12/25/24 6:18 PM, Randy Bush wrote:
> where does one go for is-is help?  the mtu issie can be painful!!!

I think here would be good too.  I recently had to do this between a Cisco
3945e and a Juniper, and from my unrevised notes:

vlan {
  unit 405 {
family iso {
# holy shit this is important.  CISCO and Juniper will not talk unless the
MTU is set
mtu 1492;
  }
   }
}

:-)

-- 
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: Best way to have redundancy announcing on separate routers

[Randy Bush]

>>> IMHO, this is exactly the thing NANOG is here for, helping others run
>>> BGP.
>> where does one go for is-is help?  the mtu issie can be painful!!!
> I don’t think a mailing list would be of help… perhaps a local tavern
> instead!

actually, the mailing list, particularly saku among others if i remember
aright, helped a lot.

randy

Re: Best way to have redundancy announcing on separate routers

[Tim Burke]

> On Dec 25, 2024, at 17:18, Randy Bush  wrote:
> 
>> IMHO, this is exactly the thing NANOG is here for, helping others run
>> BGP.
> 
> where does one go for is-is help?  the mtu issie can be painful!!!


I don’t think a mailing list would be of help… perhaps a local tavern instead!

Re: Best way to have redundancy announcing on separate routers

[Randy Bush]

> IMHO, this is exactly the thing NANOG is here for, helping others run
> BGP.

where does one go for is-is help?  the mtu issie can be painful!!!

randy

Re: Best way to have redundancy announcing on separate routers

[Tim Burke]

> On Dec 25, 2024, at 12:09, Bryan Fields  wrote:
> 
> If you want, post your IP block(s), ASN and your upstreams' ASNs, a number of
> people will be able to see if you're prepending correctly.  Also feel free to
> contact your upstreams' BGP support, you're paying them, and they may explain
> what they support too if it's not written down publicly.
> 
> IMHO, this is exactly the thing NANOG is here for, helping others run BGP.

Wholeheartedly agree — I would *much* rather someone send a message to NANOG 
asking for help, instead of haphazardly doing BGP to the detriment of their 
customers and the internet as a whole. Helping someone may be a small 
“inconvenience” but sure as hell beats another AS7007 incident.

> -- 
> Bryan Fields
> 
> 727-409-1194 - Voice
> http://bryanfields.net

Re: Best way to have redundancy announcing on separate routers

[Bryan Fields]

On 12/24/24 5:19 AM, Jean Franco wrote:
> Thanks Bill for the well explanation!
> I'll probably will have to go into the communities then, some of tests I've
> done got me nowhere!
> I'm using VyOS (quagga) and prepending didn't help.

If you want, post your IP block(s), ASN and your upstreams' ASNs, a number of
people will be able to see if you're prepending correctly.  Also feel free to
contact your upstreams' BGP support, you're paying them, and they may explain
what they support too if it's not written down publicly.

IMHO, this is exactly the thing NANOG is here for, helping others run BGP.
-- 
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: Best way to have redundancy announcing on separate routers

[Jack Bates]

On 12/23/2024 5:33 PM, Jean Franco wrote:

I'm trying to achieve total redundancy on a multihomed environment:

ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2
Where X is my Network.




The hardest part can be handling a failure of either of the routers and 
having X still be able to talk to the other in smaller networks. While 
VRRP, MC-LAG, and MPLS do exist, platform, vendor, and your requirements 
all make for a lot of fun. It's easy to accidentally make routers do 
things the vendor hadn't intended (What do you mean subscriber services 
aren't designed to work with mc-ae? Is that why dhcp sync only works 
with vrrp and mpls and not mc-ae with unnumbered interfaces?)


I'll try not to cover what others have said, but there are a few things 
to consider on dealing with your ISPs. They may run RPF filtering, so 
even if you don't want them to route traffic for a network to you, if 
you might send traffic from that network out, they'll need a route, so 
always send the aggregates to everyone you send outbound traffic to. If 
you have trouble getting a network added to a peer, you may have to not 
send any outbound their way.


Many ISPs run local prefs to prefer directly connected networks over 
more costly paths. This will override AS prepends. Some may let you 
change it with a community. Some will not. If you must force traffic, 
use a more specific route. Even if others filter it out, it should still 
get enough distance to force traffic the way you want. If your 
redundancy is slightly oversold and you need rough load balancing, more 
specific routes are the way to handle that, but try and minimize their 
use. We do have routing table bloat.



Jack

Re: Best way to have redundancy announcing on separate routers

[Randy Bush]

> Sorry, this is not a general help list for basic networking skills.

wow!  that was helpful, polite, and considerate.  good on ya

the list ihas been abput helping others, including the less senior, for
some decades.  the nanog list has been the one of the most significant
public help to networkers since dirt was invented.

randy

Re: Best way to have redundancy announcing on separate routers

[Mike Hammett]

How much capacity and how much network do you have between router 1 and router 
2? 


Are the routers between DFZ capable? 
Do the links have the ability to carry the full load? 
How many routers between router 1 and router 2? 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Jean Franco"  
To: "North American Network Operators' Group"  
Sent: Monday, December 23, 2024 5:33:06 PM 
Subject: Best way to have redundancy announcing on separate routers 


Hi Folks, 


I'm trying to achieve total redundancy on a multihomed environment: 


ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2 
Where X is my Network. 


In the example below, he announces separate blocks to each ISP. 



https://www.networkstraining.com/cisco-bgp-configuration-tutorial/ 


I would like to do a failover model, where if one ISP goes down the other would 
take over. 
Please share your thoughts on this. 


Best regards, 

Re: Best way to have redundancy announcing on separate routers

[Willy Manga]

Hi Jean,

On 24/12/2024 03:33, Jean Franco wrote:

Hi Folks,

I'm trying to achieve total redundancy on a multihomed environment:

ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2
Where X is my Network.
[...]
I would like to do a failover model, where if one ISP goes down the other
would take over.
Please share your thoughts on this.


I would suggest you read the materials here https://learn.nsrc.org/bgp 
"Multi-Homing" section. I believe you will find more extensive 
documentation about what you are trying to achieve.




--
Willy Manga

Re: Best way to have redundancy announcing on separate routers

[Pedro Prado]

Jean,The internet is a network like any other (I’m lying, but fundamentally it is).If you advertise your own subnet via two paths then the network can use those two paths to reach your subnet. In the internet that is not so usual as you’d need to pay to own that subnet and an AS. Usually you’ll use a subnet from your ISP, and with one exit only, just use default routing (so you don’t need to learn 900k+ routes that all go via the same next hop).With two ISPs, operating in failover, you can also use just the default from both. You will then use some policy to select which default to use. If you want to customize it (say YouTube goes out of ISP A, while the rest uses B), then you will need to learn more routes than the default.Other than routing, you need to understand how those links will be used. Users browsing need to use addresses from the ISP so if they are egressing A, they need to be using A’s addresses. This is usually done with NAT and the device doing it has to be aware of the egress. Often, it’s the router itself doing NAT based on the exit interface. But there are multiple possibilities…HTH,/PedroOn 24 Dec 2024, at 10:21, Jean Franco  wrote:Thanks Bill for the well explanation!I'll probably will have to go into the communities then, some of tests I've done got me nowhere!I'm using VyOS (quagga) and prepending didn't help.Best regards,On Tue, Dec 24, 2024 at 12:42 AM William Herrin  wrote:On Mon, Dec 23, 2024 at 4:53 PM Christopher Hawker  wrote:
> You can establish an iBGP session between the two routers that exchange
> either default & own routes, or they can send their own routes with fulls and
> use local pref to preference the directly-connected transit session before routes
> learnt from the iBGP session, depending on how you want engineer your traffic.

That's the easy part. If you want the ISPs to be equal with the
shortest path getting the traffic then you're done. Congratulations.

> I would like to do a failover model, where if one ISP goes down the other would take over.

If you want to weigh one ISP to be "primary" and the other to be
"backup," you've a long hard road ahead of you. Localprefs can make
you prefer one ISP over the other for _outbound_ traffic but the
levers for controlling _inbound_ traffic are more complicated.

You can get part of the way there by "prepending" your AS number
several times on the backup path. That makes the AS path longer from
the backup ISP which tends to cause BGP selection to pick the shorter
path via the primary ISP.  That's basically BGP's default: shorter AS
path wins.

Except for all the jackals out there who use a local mechanism to pick
the best path without regard to the AS path length. For those, you'll
have to learn about "communities." Communities are basically tags: you
tag a route and if your ISP understands the tag it does something
different than normal with that route. Your ISPs publish a list of
communities they understand along with what they will do differently
if you tag a route with that community. Typically you'll want to find
the community that tells your ISP to set their own localpref
differently than the default. You may even need to find the
communities that tell your ISP's ISPs to set their localprefs
differently than their defaults. It gets complicated fast.

Regards,
Bill Herrin




-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: Best way to have redundancy announcing on separate routers

[Jean Franco]

Thanks Bill for the well explanation!
I'll probably will have to go into the communities then, some of tests I've
done got me nowhere!
I'm using VyOS (quagga) and prepending didn't help.

Best regards,


On Tue, Dec 24, 2024 at 12:42 AM William Herrin  wrote:

> On Mon, Dec 23, 2024 at 4:53 PM Christopher Hawker 
> wrote:
> > You can establish an iBGP session between the two routers that exchange
> > either default & own routes, or they can send their own routes with
> fulls and
> > use local pref to preference the directly-connected transit session
> before routes
> > learnt from the iBGP session, depending on how you want engineer your
> traffic.
>
> That's the easy part. If you want the ISPs to be equal with the
> shortest path getting the traffic then you're done. Congratulations.
>
> > I would like to do a failover model, where if one ISP goes down the
> other would take over.
>
> If you want to weigh one ISP to be "primary" and the other to be
> "backup," you've a long hard road ahead of you. Localprefs can make
> you prefer one ISP over the other for _outbound_ traffic but the
> levers for controlling _inbound_ traffic are more complicated.
>
> You can get part of the way there by "prepending" your AS number
> several times on the backup path. That makes the AS path longer from
> the backup ISP which tends to cause BGP selection to pick the shorter
> path via the primary ISP.  That's basically BGP's default: shorter AS
> path wins.
>
> Except for all the jackals out there who use a local mechanism to pick
> the best path without regard to the AS path length. For those, you'll
> have to learn about "communities." Communities are basically tags: you
> tag a route and if your ISP understands the tag it does something
> different than normal with that route. Your ISPs publish a list of
> communities they understand along with what they will do differently
> if you tag a route with that community. Typically you'll want to find
> the community that tells your ISP to set their own localpref
> differently than the default. You may even need to find the
> communities that tell your ISP's ISPs to set their localprefs
> differently than their defaults. It gets complicated fast.
>
> Regards,
> Bill Herrin
>
>
>
>
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
>

Re: Best way to have redundancy announcing on separate routers

[William Herrin]

On Mon, Dec 23, 2024 at 4:53 PM Christopher Hawker  wrote:
> You can establish an iBGP session between the two routers that exchange
> either default & own routes, or they can send their own routes with fulls and
> use local pref to preference the directly-connected transit session before 
> routes
> learnt from the iBGP session, depending on how you want engineer your traffic.

That's the easy part. If you want the ISPs to be equal with the
shortest path getting the traffic then you're done. Congratulations.

> I would like to do a failover model, where if one ISP goes down the other 
> would take over.

If you want to weigh one ISP to be "primary" and the other to be
"backup," you've a long hard road ahead of you. Localprefs can make
you prefer one ISP over the other for _outbound_ traffic but the
levers for controlling _inbound_ traffic are more complicated.

You can get part of the way there by "prepending" your AS number
several times on the backup path. That makes the AS path longer from
the backup ISP which tends to cause BGP selection to pick the shorter
path via the primary ISP.  That's basically BGP's default: shorter AS
path wins.

Except for all the jackals out there who use a local mechanism to pick
the best path without regard to the AS path length. For those, you'll
have to learn about "communities." Communities are basically tags: you
tag a route and if your ISP understands the tag it does something
different than normal with that route. Your ISPs publish a list of
communities they understand along with what they will do differently
if you tag a route with that community. Typically you'll want to find
the community that tells your ISP to set their own localpref
differently than the default. You may even need to find the
communities that tell your ISP's ISPs to set their localprefs
differently than their defaults. It gets complicated fast.

Regards,
Bill Herrin




-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: Best way to have redundancy announcing on separate routers

[Jean Franco]

Hi Chris,

Thank you for taking your time and point me in the right direction!
I'm getting full routes, so it should be easy for me to achieve your
concept.

Best regards,

On Mon, Dec 23, 2024 at 9:53 PM Christopher Hawker 
wrote:

> Hi Jean,
>
> You can establish an iBGP session between the two routers that exchange
> either default & own routes, or they can send their own routes with fulls
> and use local pref to preference the directly-connected transit session
> before routes learnt from the iBGP session, depending on how you want
> engineer your traffic. If you are not receiving full tables and only
> getting a default from each transit provider you would need to weight the
> defaults so it uses the preferred default. If you're planning to add (for
> example) peering or PNIs to either router in the future, you will want full
> tables for greater traffic control.
>
> Regards,
> Christopher Hawker
> --
> *From:* NANOG  on behalf of
> Jean Franco 
> *Sent:* Tuesday, December 24, 2024 10:33 AM
> *To:* North American Network Operators' Group 
> *Subject:* Best way to have redundancy announcing on separate routers
>
> Hi Folks,
>
> I'm trying to achieve total redundancy on a multihomed environment:
>
> ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2
> Where X is my Network.
>
> In the example below, he announces separate blocks to each ISP.
>
> https://www.networkstraining.com/cisco-bgp-configuration-tutorial/
>
> I would like to do a failover model, where if one ISP goes down the other
> would take over.
> Please share your thoughts on this.
>
> Best regards,
>

Re: Best way to have redundancy announcing on separate routers

[Dave Edelman via NANOG]

Agreed, especially since I have seen more than a few professional installations where they got it wrong. Regards,David Edelman who is responsible for the concept of this message. Unfortunately, autocorrect is responsible for the contentOn Dec 23, 2024, at 19:45, Christopher Hawker  wrote:






Tony,




"The NANOG Mailing List is a community-moderated forum, open to all. Established in 1994 to provide the
open exchange of technical information, it provides the opportunity for lively
discussions of specific implementation challenges that require
cooperation among network service providers." - 
NANOG Usage Guidelines




I disagree with your remark that "this is not an appropriate channel for this query". I would believe that Jean's query falls under the "open exchange of technical information" category.




Regards,

Christopher Hawker


From: NANOG  on behalf of Tony Wicks 
Sent: Tuesday, December 24, 2024 11:33 AM
To: 'Jean Franco' 
Cc: 'North American Network Operators' Group' 
Subject: RE: Best way to have redundancy announcing on separate routers
 




Sorry, this is not a general help list for basic networking skills. There are many options for appropriate training available, but this is not an appropriate channel for this query.
 

From: NANOG 
On Behalf Of Jean Franco
Sent: Tuesday, 24 December 2024 12:33 pm
To: North American Network Operators' Group 
Subject: Best way to have redundancy announcing on separate routers

 

Hi Folks,

 


I'm trying to achieve total redundancy on a multihomed environment:


 


ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2


Where X is my Network.


 


In the example below, he announces separate blocks to each ISP.



 


https://www.networkstraining.com/cisco-bgp-configuration-tutorial/


 


I would like to do a failover model, where if one ISP goes down the other would take over.


Please share your thoughts on this.


 


Best regards,

Re: Best way to have redundancy announcing on separate routers

[Christopher Hawker]

Hi Jean,

You can establish an iBGP session between the two routers that exchange either 
default & own routes, or they can send their own routes with fulls and use 
local pref to preference the directly-connected transit session before routes 
learnt from the iBGP session, depending on how you want engineer your traffic. 
If you are not receiving full tables and only getting a default from each 
transit provider you would need to weight the defaults so it uses the preferred 
default. If you're planning to add (for example) peering or PNIs to either 
router in the future, you will want full tables for greater traffic control.

Regards,
Christopher Hawker

From: NANOG  on behalf of Jean 
Franco 
Sent: Tuesday, December 24, 2024 10:33 AM
To: North American Network Operators' Group 
Subject: Best way to have redundancy announcing on separate routers

Hi Folks,

I'm trying to achieve total redundancy on a multihomed environment:

ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2
Where X is my Network.

In the example below, he announces separate blocks to each ISP.

https://www.networkstraining.com/cisco-bgp-configuration-tutorial/

I would like to do a failover model, where if one ISP goes down the other would 
take over.
Please share your thoughts on this.

Best regards,

Re: Best way to have redundancy announcing on separate routers

[Christopher Hawker]

Tony,

"The NANOG Mailing List is a community-moderated forum, open to all. 
Established in 1994 to provide the open exchange of technical information, it 
provides the opportunity for lively discussions of specific implementation 
challenges that require cooperation among network service providers." - NANOG 
Usage Guidelines<https://nanog.org/legal/usage-guidelines/>

I disagree with your remark that "this is not an appropriate channel for this 
query". I would believe that Jean's query falls under the "open exchange of 
technical information" category.

Regards,
Christopher Hawker

From: NANOG  on behalf of Tony 
Wicks 
Sent: Tuesday, December 24, 2024 11:33 AM
To: 'Jean Franco' 
Cc: 'North American Network Operators' Group' 
Subject: RE: Best way to have redundancy announcing on separate routers


Sorry, this is not a general help list for basic networking skills. There are 
many options for appropriate training available, but this is not an appropriate 
channel for this query.



From: NANOG  On Behalf Of Jean Franco
Sent: Tuesday, 24 December 2024 12:33 pm
To: North American Network Operators' Group 
Subject: Best way to have redundancy announcing on separate routers



Hi Folks,



I'm trying to achieve total redundancy on a multihomed environment:



ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2

Where X is my Network.



In the example below, he announces separate blocks to each ISP.



https://www.networkstraining.com/cisco-bgp-configuration-tutorial/



I would like to do a failover model, where if one ISP goes down the other would 
take over.

Please share your thoughts on this.



Best regards,

RE: Best way to have redundancy announcing on separate routers

[Tony Wicks]

Sorry, this is not a general help list for basic networking skills. There are 
many options for appropriate training available, but this is not an appropriate 
channel for this query.

 

From: NANOG  On Behalf Of Jean Franco
Sent: Tuesday, 24 December 2024 12:33 pm
To: North American Network Operators' Group 
Subject: Best way to have redundancy announcing on separate routers

 

Hi Folks,

 

I'm trying to achieve total redundancy on a multihomed environment:

 

ISP 1 <=> Router 1 <= X => Router 2 <=> ISP 2

Where X is my Network.

 

In the example below, he announces separate blocks to each ISP.

 

https://www.networkstraining.com/cisco-bgp-configuration-tutorial/

 

I would like to do a failover model, where if one ISP goes down the other would 
take over.

Please share your thoughts on this.

 

Best regards,