Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/11/23 07:28, Blake Dunlap wrote:

I'm confused here, are you intentionally running larger MTU interfaces 
than the packet filter can handle with default config, and not wanting 
to change the tunable to fix the config for buffer size for the packet 
filter, or am I misreading?


So I've gone ahead and tested "net.bpf.bufsize" and 
"net.bpf.maxbufsize", and none of them are helping any.


I'll send one to the FreeBSD mailing list and ask them if they can 
explain this. I saw something a little similar for DHCP on pfSense, 
although it has quite a few differences in that case:


https://forum.netgate.com/topic/130219/dhcpv6-client-broken-in-latest-snapshot/45

I'll advise when I know more.

Mark.

Re: Best Linux (or BSD) hosted BGP?

[Brandon Zhi]

I use bird2 with Debian11 sometimes, I'm curious, what is the usual
hardware for using Linux as a router? In addition, the Linux ip rule seems
to have a problem with the matching of the ipv4 source address. . .
*Brandon Zhi*
HUIZE LTD
www.huize.asia  | www.ixp.su | Twitter

This e-mail and any attachments or any reproduction of this e-mail in
whatever manner are confidential and for the use of the addressee(s) only.
HUIZE LTD can’t take any liability and guarantee of the text of the email
message and virus.


On Thu, 11 May 2023 at 13:29, Blake Dunlap  wrote:

> I'm confused here, are you intentionally running larger MTU interfaces
> than the packet filter can handle with default config, and not wanting to
> change the tunable to fix the config for buffer size for the packet filter,
> or am I misreading?
>
> On Wed, May 10, 2023 at 11:51 PM Mark Tinka  wrote:
>
>>
>>
>> On 5/10/23 15:55, Tom Beecher wrote:
>>
>> >  That could just as easily happen today. Every OS release has all
>> > kinds of changes to defaults, and frequently don't get caught until
>> > they break something. Even if today's FreeBSD defaults worked for this
>> > scenario, the next release could change to a value that doesn't.
>>
>> We implement a lot of user-defined changes to FreeBSD defaults via
>> "/etc/sysctl.conf", as an example, whose unexpected change would not
>> necessarily break anything as they would reduce scaled performance. We
>> can live with that, because we can afford a reduction in performance
>> until the fault is found, not an outright outage.
>>
>> The problem with doing this with something like a routing protocol - and
>> in this specific case with FRR on FreeBSD for IS-IS - is that it would
>> not be a reduction in performance if an unexpected change were to find
>> its way into future revisions of FreeBSD... it would, in all likelihood,
>> be a complete outage. That is a steeper price to pay, for us anyway.
>>
>> It's just about weighing the risks for one's particular operating
>> environment, and for us, that risk is too high for a routing protocol.
>>
>> Mark.
>>
>

Re: Best Linux (or BSD) hosted BGP?

[Blake Dunlap]

I'm confused here, are you intentionally running larger MTU interfaces than
the packet filter can handle with default config, and not wanting to change
the tunable to fix the config for buffer size for the packet filter, or am
I misreading?

On Wed, May 10, 2023 at 11:51 PM Mark Tinka  wrote:

>
>
> On 5/10/23 15:55, Tom Beecher wrote:
>
> >  That could just as easily happen today. Every OS release has all
> > kinds of changes to defaults, and frequently don't get caught until
> > they break something. Even if today's FreeBSD defaults worked for this
> > scenario, the next release could change to a value that doesn't.
>
> We implement a lot of user-defined changes to FreeBSD defaults via
> "/etc/sysctl.conf", as an example, whose unexpected change would not
> necessarily break anything as they would reduce scaled performance. We
> can live with that, because we can afford a reduction in performance
> until the fault is found, not an outright outage.
>
> The problem with doing this with something like a routing protocol - and
> in this specific case with FRR on FreeBSD for IS-IS - is that it would
> not be a reduction in performance if an unexpected change were to find
> its way into future revisions of FreeBSD... it would, in all likelihood,
> be a complete outage. That is a steeper price to pay, for us anyway.
>
> It's just about weighing the risks for one's particular operating
> environment, and for us, that risk is too high for a routing protocol.
>
> Mark.
>

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/10/23 15:55, Tom Beecher wrote:

 That could just as easily happen today. Every OS release has all 
kinds of changes to defaults, and frequently don't get caught until 
they break something. Even if today's FreeBSD defaults worked for this 
scenario, the next release could change to a value that doesn't.


We implement a lot of user-defined changes to FreeBSD defaults via 
"/etc/sysctl.conf", as an example, whose unexpected change would not 
necessarily break anything as they would reduce scaled performance. We 
can live with that, because we can afford a reduction in performance 
until the fault is found, not an outright outage.


The problem with doing this with something like a routing protocol - and 
in this specific case with FRR on FreeBSD for IS-IS - is that it would 
not be a reduction in performance if an unexpected change were to find 
its way into future revisions of FreeBSD... it would, in all likelihood, 
be a complete outage. That is a steeper price to pay, for us anyway.


It's just about weighing the risks for one's particular operating 
environment, and for us, that risk is too high for a routing protocol.


Mark.

Re: Best Linux (or BSD) hosted BGP?

[Tom Beecher]

>
> or if future FreeBSD updates decide to "go their own
> way"... yes.
>

 That could just as easily happen today. Every OS release has all kinds of
changes to defaults, and frequently don't get caught until they break
something. Even if today's FreeBSD defaults worked for this scenario, the
next release could change to a value that doesn't.

On Wed, May 10, 2023 at 12:46 AM Mark Tinka  wrote:

>
>
> On 5/10/23 03:40, Tom Beecher wrote:
>
> >
> > Adjusting a single tunable is 'onerous'? Ok.
>
> In the context of long term administration of the environment, years
> after everybody has forgotten about the hack, or worse, folk leave and
> others take over; or if future FreeBSD updates decide to "go their own
> way"... yes.
>
> Mark.
>
>

Re: Best Linux (or BSD) hosted BGP?

[Tom Beecher]

>
> No, but it's brittle. A workaround, not a solution. Likely to break
> during future maintenance. "Unpredictable" as Mark put it.
>
> Nothing a routing daemon does should involve the kernel BPF. The next
> sysadmin won't be expecting it.


Not sure I agree.

Implemented defaults may not be appropriate in all environments and
situations, so mechanisms are provided for admins to tune and adjust for
what they need. We do this all the time on everything. I'm not ignorant of
the fact that this can lend to 'oops' scenarios where someone didn't
document the adjustment from default, or it was forgotten, lost, etc. But
that's not a technical issue, that's a human one.

There's always a tradeoff to be made. Sysadmins could adjust tunables, but
then they have to manage that delta. Or developers can add code to work
around the situation, but they have to manage that. Someone has to do work
somewhere.

On Tue, May 9, 2023 at 11:55 PM William Herrin  wrote:

> On Tue, May 9, 2023 at 6:40 PM Tom Beecher  wrote:
> >> The implication being that while it might work, it would make
> administration of the system onerous and unpredictable, considering we are
> dealing with a ton of FreeBSD installations, and not just a single server.
> >
> > Adjusting a single tunable is 'onerous'?
>
> No, but it's brittle. A workaround, not a solution. Likely to break
> during future maintenance. "Unpredictable" as Mark put it.
>
> Nothing a routing daemon does should involve the kernel BPF. The next
> sysadmin won't be expecting it.
>
> Regards,
> Bill Herrin
>
>
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
>

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/10/23 03:40, Tom Beecher wrote:



Adjusting a single tunable is 'onerous'? Ok.


In the context of long term administration of the environment, years 
after everybody has forgotten about the hack, or worse, folk leave and 
others take over; or if future FreeBSD updates decide to "go their own 
way"... yes.


Mark.

Re: Best Linux (or BSD) hosted BGP?

[Jay R. Ashworth]

- Original Message -
> From: "William Herrin" 

> On Tue, May 9, 2023 at 6:40 PM Tom Beecher  wrote:
>>> The implication being that while it might work, it would make 
>>> administration of
>>> the system onerous and unpredictable, considering we are dealing with a ton 
>>> of
>>> FreeBSD installations, and not just a single server.
>>
>> Adjusting a single tunable is 'onerous'?
> 
> No, but it's brittle. A workaround, not a solution. Likely to break
> during future maintenance. "Unpredictable" as Mark put it.
> 
> Nothing a routing daemon does should involve the kernel BPF. The next
> sysadmin won't be expecting it.

That's such an important thought that it has a name.

The Principle of Least Astonishment.

"When doing things, try to pick the way among many that will least confuse 
the people who have to pick up the pieces when you get hit by a bus."

Cheers,
-- jra
-- 
Jay R. Ashworth  Baylink   j...@baylink.com
Designer The Things I Think   RFC 2100
Ashworth & Associates   http://www.bcp38.info  2000 Land Rover DII
St Petersburg FL USA  BCP38: Ask For It By Name!   +1 727 647 1274

Re: Best Linux (or BSD) hosted BGP?

[William Herrin]

On Tue, May 9, 2023 at 6:40 PM Tom Beecher  wrote:
>> The implication being that while it might work, it would make administration 
>> of the system onerous and unpredictable, considering we are dealing with a 
>> ton of FreeBSD installations, and not just a single server.
>
> Adjusting a single tunable is 'onerous'?

No, but it's brittle. A workaround, not a solution. Likely to break
during future maintenance. "Unpredictable" as Mark put it.

Nothing a routing daemon does should involve the kernel BPF. The next
sysadmin won't be expecting it.

Regards,
Bill Herrin


-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: Best Linux (or BSD) hosted BGP?

[Tom Beecher]

>
> The implication being that while it might work, it would make
> administration of the system onerous and unpredictable, considering we are
> dealing with a ton of FreeBSD installations, and not just a single server.
>

Adjusting a single tunable is 'onerous'? Ok.

On Tue, May 9, 2023 at 9:00 AM Mark Tinka  wrote:

>
>
> On 5/9/23 14:32, Tom Beecher wrote:
>
>
> Except you didn't exactly "call out limitations". You simply said :
>
> IS-IS in Quagga and FRR are not yet ready for business, is what I would
>> caution.
>>
>
> The reality is that's not true.
>
>
> And just a few weeks prior, I had given an update about this very issue,
> as per attached.
>
> I figured if anyone needed more details (as I'd provided weeks prior),
> they'd reach out, like Bryan did.
>
>
>
> - You have a specific environment ( FRR on FreeBSD ) that has an issue
> with IS-IS.
> - You identified the issue in Apr 2020 as related to the size of the PSNPs
> being larger than the default BPF packet buffer size :
> https://seclists.org/nanog/2020/Apr/238
> - Although a solution was provided (
> https://seclists.org/nanog/2020/Apr/240 ) , you made an affirmative
> choice you didn't want to. ( https://seclists.org/nanog/2020/Apr/241 )
>
> You didn't say "I don't want to adjust net.bpf.bufsize because it would
> have negative impacts in my environment, so I need an option in FRR to
> adjust the PSNP size." You said "I don't want to."
>
>
> Ummh, not sure if you need help reading, but my exact words were:
>
> "Probably could - but I'd prefer solutions that don't mess with the
> base
> system, which ensures long term usability of FRR across future
> upgrades."
>
> The implication being that while it might work, it would make
> administration of the system onerous and unpredictable, considering we are
> dealing with a ton of FreeBSD installations, and not just a single server.
>
> You, somehow, read my response is me being petulant; which is entirely
> your right, of course. But I also won't let you make a false inference of
> what my response actually was.
>
>
> You are of course perfectly free to not make that change, and perfectly
> free to gripe that FRR development has not done what you asked. But it's
> pretty disingenuous to say that the software isn't "ready" strictly because
> of issues in your use case. That doesn't really help anyone.
>
>
> I made a statement, I was asked to explain, I did. There was historical
> context, even though I can't expect you to file all member's postings to
> memory.
>
> But if you want to let yourself get into your feelings, I can't help you
> there.
>
> Mark.
>

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/9/23 14:32, Tom Beecher wrote:



Except you didn't exactly "call out limitations". You simply said :

IS-IS in Quagga and FRR are not yet ready for business, is what I
would
caution.


The reality is that's not true.


And just a few weeks prior, I had given an update about this very issue, 
as per attached.


I figured if anyone needed more details (as I'd provided weeks prior), 
they'd reach out, like Bryan did.





- You have a specific environment ( FRR on FreeBSD ) that has an issue 
with IS-IS.
- You identified the issue in Apr 2020 as related to the size of the 
PSNPs being larger than the default BPF packet buffer size : 
https://seclists.org/nanog/2020/Apr/238
- Although a solution was provided ( 
https://seclists.org/nanog/2020/Apr/240 ) , you made an affirmative 
choice you didn't want to. ( https://seclists.org/nanog/2020/Apr/241 )


You didn't say "I don't want to adjust net.bpf.bufsize because it 
would have negative impacts in my environment, so I need an option in 
FRR to adjust the PSNP size." You said "I don't want to."


Ummh, not sure if you need help reading, but my exact words were:

    "Probably could - but I'd prefer solutions that don't mess with the 
base
    system, which ensures long term usability of FRR across future 
upgrades."


The implication being that while it might work, it would make 
administration of the system onerous and unpredictable, considering we 
are dealing with a ton of FreeBSD installations, and not just a single 
server.


You, somehow, read my response is me being petulant; which is entirely 
your right, of course. But I also won't let you make a false inference 
of what my response actually was.



You are of course perfectly free to not make that change, and 
perfectly free to gripe that FRR development has not done what you 
asked. But it's pretty disingenuous to say that the software isn't 
"ready" strictly because of issues in your use case. That doesn't 
really help anyone.


I made a statement, I was asked to explain, I did. There was historical 
context, even though I can't expect you to file all member's postings to 
memory.


But if you want to let yourself get into your feelings, I can't help you 
there.


Mark.--- Begin Message ---
Despite a new release of FRR since my last testing in 2021, IS-IS 
appears to be more broken, with no hopes of it getting any attention.


Will drop back to Quagga.

If anyone is successfully using IS-IS on FRR, I'd be keen to hear.

Mark.

On 3/31/23 09:16, Mark Tinka wrote:

Hi all.

I posted the below in 2020 about IS-IS in FRR (7.3, at the time):

https://seclists.org/nanog/2020/Apr/226

I gave up and moved back to Quagga with OSPF.

I decided to try again today, with FRR 8.5, on FreeBSD-13.1-RELEASE-p6.

Things seem to have taken a major step back since then. I can't even 
get an adjacency going with a Cisco IOS XE router:


2023/03/31 07:12:01 ISIS: [WDVAD-TY6HY] ISIS-LFA: failed to unregister 
RLFA with LDP
2023/03/31 07:12:01 ISIS: [WDVAD-TY6HY] ISIS-LFA: failed to unregister 
RLFA with LDP
2023/03/31 07:12:01 ISIS: [WDVAD-TY6HY] ISIS-LFA: failed to unregister 
RLFA with LDP

2023/03/31 07:12:01 ISIS: [YDRGH-0DJ94] Opened BPF device /dev/bpf0
2023/03/31 07:12:01 ISIS: [QJWNQ-3FFKM] BPF buffer len = 4096
2023/03/31 07:12:01 ISIS: [V3AHD-XE2Z5] failed to set BPF buffer len 
(4096 to 9000)
2023/03/31 07:12:01 ISIS: [W02G1-H56T1] IS-IS bpf: could not transmit 
packet on em0: Input/output error
2023/03/31 07:12:01 ISIS: [G7ZA5-AT06A][EC 67108865] ISIS-Adj (1): 
Send L2 IIH on em0 failed
2023/03/31 07:12:04 ISIS: [W02G1-H56T1] IS-IS bpf: could not transmit 
packet on em0: Input/output error
2023/03/31 07:12:04 ISIS: [G7ZA5-AT06A][EC 67108865] ISIS-Adj (1): 
Send L2 IIH on em0 failed
2023/03/31 07:12:07 ISIS: [W02G1-H56T1] IS-IS bpf: could not transmit 
packet on em0: Input/output error
2023/03/31 07:12:07 ISIS: [G7ZA5-AT06A][EC 67108865] ISIS-Adj (1): 
Send L2 IIH on em0 failed


Thoughts?

Mark.
--- End Message ---

Re: Best Linux (or BSD) hosted BGP?

[Tom Beecher]

>
> I think we all appreciate how open source projects work. Calling out
> their limitations is as old as mailing lists. I don't code. I test a
> lot, and continue to test IS-IS in FRR on FreeBSD every year or so. I'll
> keep testing and giving feedback at least once or twice a year. If it's
> still broken, I'll call it out.
>

Except you didn't exactly "call out limitations". You simply said :

IS-IS in Quagga and FRR are not yet ready for business, is what I would
> caution.
>

The reality is that's not true.

- You have a specific environment ( FRR on FreeBSD ) that has an issue with
IS-IS.
- You identified the issue in Apr 2020 as related to the size of the PSNPs
being larger than the default BPF packet buffer size :
https://seclists.org/nanog/2020/Apr/238
- Although a solution was provided ( https://seclists.org/nanog/2020/Apr/240
) , you made an affirmative choice you didn't want to. (
https://seclists.org/nanog/2020/Apr/241 )

You didn't say "I don't want to adjust net.bpf.bufsize because it would
have negative impacts in my environment, so I need an option in FRR to
adjust the PSNP size." You said "I don't want to."

You are of course perfectly free to not make that change, and perfectly
free to gripe that FRR development has not done what you asked. But it's
pretty disingenuous to say that the software isn't "ready" strictly because
of issues in your use case. That doesn't really help anyone.

On Tue, May 9, 2023 at 12:16 AM Mark Tinka  wrote:

>
>
> On 5/9/23 00:03, Jeff Tantsura wrote:
>
> > Saying that IS-IS in FRR is broken is incorrect, that it is in many ways
> weird - no offense to folks who coded it :)  (especially if you have worked
> with commercial code bases), that it doesn’t scale/naive, missing features
> - for sure.
> > FRR runs today some of the biggest DCs in the world and is reasonably
> stable within the feature set used (RFC7938), there’s interest in further
> IS-IS development - you could see some minor bug fixes/features coming from
> a particular company, if you are interested - join them and work together.
>
> When one can't get a base adjacency going, use of further features is
> rather pointless.
>
> But again, it seems those running it on Linux have better luck than
> those on FreeBSD, so that is something to remember. We are a FreeBSD
> house. Our alternative is to use OSPF in Quagga and redistribute it into
> IS-IS. That works well. Would we like to use IS-IS natively? Sure. But
> the alternative has been working well for nearly 10 years.
>
>
> > FRR is an Open Source project, whining about missing features is not
> helpful, coding (or at least testing) and contributing is.
>
> I think we all appreciate how open source projects work. Calling out
> their limitations is as old as mailing lists. I don't code. I test a
> lot, and continue to test IS-IS in FRR on FreeBSD every year or so. I'll
> keep testing and giving feedback at least once or twice a year. If it's
> still broken, I'll call it out.
>
> Mark.
>

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/9/23 00:03, Jeff Tantsura wrote:


Saying that IS-IS in FRR is broken is incorrect, that it is in many ways weird 
- no offense to folks who coded it :)  (especially if you have worked with 
commercial code bases), that it doesn’t scale/naive, missing features - for 
sure.
FRR runs today some of the biggest DCs in the world and is reasonably stable 
within the feature set used (RFC7938), there’s interest in further IS-IS 
development - you could see some minor bug fixes/features coming from a 
particular company, if you are interested - join them and work together.


When one can't get a base adjacency going, use of further features is 
rather pointless.


But again, it seems those running it on Linux have better luck than 
those on FreeBSD, so that is something to remember. We are a FreeBSD 
house. Our alternative is to use OSPF in Quagga and redistribute it into 
IS-IS. That works well. Would we like to use IS-IS natively? Sure. But 
the alternative has been working well for nearly 10 years.




FRR is an Open Source project, whining about missing features is not helpful, 
coding (or at least testing) and contributing is.


I think we all appreciate how open source projects work. Calling out 
their limitations is as old as mailing lists. I don't code. I test a 
lot, and continue to test IS-IS in FRR on FreeBSD every year or so. I'll 
keep testing and giving feedback at least once or twice a year. If it's 
still broken, I'll call it out.


Mark.

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/8/23 22:52, William Herrin wrote:


Hi Mark,

I'm not sure I'd call bugs that don't appear when running FRR on Linux
(only FreeBSD), "not yet ready for business." Or did I misunderstand
your bug report?

Don't get me wrong: if you tell me it's not right until it works
without Linux-centric assumptions, I'll agree with you. But not being
ready for your custom environment is not quite the same thing as not
being broadly ready for others.


A fair point.

We are not a Linux environment.

Mark.

Re: Best Linux (or BSD) hosted BGP?

[Jeff Tantsura]

Saying that IS-IS in FRR is broken is incorrect, that it is in many ways weird 
- no offense to folks who coded it :)  (especially if you have worked with 
commercial code bases), that it doesn’t scale/naive, missing features - for 
sure.
FRR runs today some of the biggest DCs in the world and is reasonably stable 
within the feature set used (RFC7938), there’s interest in further IS-IS 
development - you could see some minor bug fixes/features coming from a 
particular company, if you are interested - join them and work together.

FRR is an Open Source project, whining about missing features is not helpful, 
coding (or at least testing) and contributing is.

Cheers,
Jeff

> On May 8, 2023, at 9:51 AM, Mark Tinka  wrote:
> 
> 
> 
> On 5/8/23 18:45, Mark Tinka wrote:
> 
>> Broken when talking to Cisco IOS XE. Catalogued here:
>> 
>> https://lists.frrouting.org/pipermail/frog/2023-March/001265.html
>> 
>> I have no doubt FRR can talk IS-IS to other instances of FRR, but that is 
>> not a realistic scenario in a large scale network with multiple vendors.
> 
> And do add, IS-IS in Quagga has been broken from the get. There is no work 
> happening there to develop it. All hopes for a working IS-IS in 
> non-commercial vendor code is happening in FRR.
> 
> I have not considered IS-IS in VyOS, as the my use-case is for Anycast 
> running on FreeBSD.
> 
> Mark.

Re: Best Linux (or BSD) hosted BGP?

[Jeff Tantsura]

All fixed (thanks Donald)

CVE-2022-40302 and CVE-2022-40318: https://github.com/FRRouting/frr/pull/12043
CVE-2022-43681: https://github.com/FRRouting/frr/pull/12247

Cheers,
Jeff

> On May 3, 2023, at 2:52 AM, Hank Nussbacher  wrote:
> 
> On 02/05/2023 17:56, Warren Kumari wrote:
> 
> For those that like FRR:
> https://thehackernews.com/2023/05/researchers-uncover-new-bgp-flaws-in.html
> 
> Regards,
> Hank
> 
>> +lots.
>> 
>> I've used a number of Linux routing thingies (BIRD, Quagga, VyOS/Ubiquiti, 
>> OpenBGPd, ExBGP), and FRR is (for me at least) by far the friendliest. It's 
>> trivial to spin this up on a cloud VM and start announcing a prefix.
>> 
>> For doing something like Anycast though (where you are mostly just 
>> announcing a route on demand), ExaBGP is great.
>> 
>> W
>> 
>> 
>> On Mon, May 01, 2023 at 2:03 PM, Jean Franco  wrote:
>> 
>>https://frrouting.org/ 
>> 
> 

Re: Best Linux (or BSD) hosted BGP?

[William Herrin]

On Wed, May 3, 2023 at 9:04 PM Mark Tinka  wrote:
> IS-IS in Quagga and FRR are not yet ready for business, is what I would
> caution.

Hi Mark,

I'm not sure I'd call bugs that don't appear when running FRR on Linux
(only FreeBSD), "not yet ready for business." Or did I misunderstand
your bug report?

Don't get me wrong: if you tell me it's not right until it works
without Linux-centric assumptions, I'll agree with you. But not being
ready for your custom environment is not quite the same thing as not
being broadly ready for others.


The thing that bugs me about FRR versus Quagga is that for reasons I
don't follow, the BGP table takes twice as much ram. That's why
there's still some Quagga in my environment.

Regards,
Bill Herrin


-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: Best Linux (or BSD) hosted BGP?

[Bryan Holloway]

On 5/8/23 18:45, Mark Tinka wrote:



On 5/8/23 15:44, Bryan Holloway wrote:


You said, "IS-IS in Quagga and FRR are not yet ready for business, ..."

Not ready for business in what way? Performance? Cross-vendor 
compatibility? Features?


Or did I misunderstand your statement?


Broken when talking to Cisco IOS XE. Catalogued here:

https://lists.frrouting.org/pipermail/frog/2023-March/001265.html

I have no doubt FRR can talk IS-IS to other instances of FRR, but that 
is not a realistic scenario in a large scale network with multiple vendors.


Mark.



Interesting, and thank you.

FWIW, we're running it against regular IOS, IOS-XR, and Arista with no 
issues (so far ...)

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/8/23 18:45, Mark Tinka wrote:


Broken when talking to Cisco IOS XE. Catalogued here:

https://lists.frrouting.org/pipermail/frog/2023-March/001265.html

I have no doubt FRR can talk IS-IS to other instances of FRR, but that 
is not a realistic scenario in a large scale network with multiple 
vendors.


And do add, IS-IS in Quagga has been broken from the get. There is no 
work happening there to develop it. All hopes for a working IS-IS in 
non-commercial vendor code is happening in FRR.


I have not considered IS-IS in VyOS, as the my use-case is for Anycast 
running on FreeBSD.


Mark.

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/8/23 15:44, Bryan Holloway wrote:


You said, "IS-IS in Quagga and FRR are not yet ready for business, ..."

Not ready for business in what way? Performance? Cross-vendor 
compatibility? Features?


Or did I misunderstand your statement?


Broken when talking to Cisco IOS XE. Catalogued here:

https://lists.frrouting.org/pipermail/frog/2023-March/001265.html

I have no doubt FRR can talk IS-IS to other instances of FRR, but that 
is not a realistic scenario in a large scale network with multiple vendors.


Mark.

Re: Best Linux (or BSD) hosted BGP?

[Bryan Holloway]

On 5/8/23 07:03, Mark Tinka wrote:



On 5/8/23 00:22, Bryan Holloway wrote:


Curious to hear more specifics about your IS-IS assertion.

We've been running it on FRR for some time without incident, but I'll 
concede that we don't do very much with it other than saying, "hey -- 
we're here; oh, and you're there."


Talking to other vendors, or other FRR installations?

Mark.


You said, "IS-IS in Quagga and FRR are not yet ready for business, ..."

Not ready for business in what way? Performance? Cross-vendor 
compatibility? Features?


Or did I misunderstand your statement?

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/8/23 00:22, Bryan Holloway wrote:


Curious to hear more specifics about your IS-IS assertion.

We've been running it on FRR for some time without incident, but I'll 
concede that we don't do very much with it other than saying, "hey -- 
we're here; oh, and you're there."


Talking to other vendors, or other FRR installations?

Mark.

Re: Best Linux (or BSD) hosted BGP?

[Bryan Holloway]

Curious to hear more specifics about your IS-IS assertion.

We've been running it on FRR for some time without incident, but I'll 
concede that we don't do very much with it other than saying, "hey -- 
we're here; oh, and you're there."



On 5/4/23 06:04, Mark Tinka wrote:



On 5/4/23 00:51, Matt Corallo wrote:

Lots of replies saying which of BIRD/exabgp/frr/quagga/openbgpd folks 
prefer, but they're all pretty good. Honestly for such a project 
they're all just as great, it comes down mostly to what you're used to 
config-wise. Used to big metal router configuration? You might find 
BIRD foreign. Used to more functional code stuff? BIRD is pretty 
great. Others I have less experience with.


IS-IS in Quagga and FRR are not yet ready for business, is what I would 
caution.


I don't know if the other options support it or not.

Mark.

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/4/23 00:51, Matt Corallo wrote:

Lots of replies saying which of BIRD/exabgp/frr/quagga/openbgpd folks 
prefer, but they're all pretty good. Honestly for such a project 
they're all just as great, it comes down mostly to what you're used to 
config-wise. Used to big metal router configuration? You might find 
BIRD foreign. Used to more functional code stuff? BIRD is pretty 
great. Others I have less experience with.


IS-IS in Quagga and FRR are not yet ready for business, is what I would 
caution.


I don't know if the other options support it or not.

Mark.

Re: Best Linux (or BSD) hosted BGP?

[Tom Beecher]

No argument there at all. Just felt like there was enough FUD in that link
it was worth calling out that specific.

On Wed, May 3, 2023 at 2:39 PM Glenn Kelley 
wrote:

> Tom - you are correct
>
> Of course - who keeps things like BGP Route Servers and FRR up to date -
>
> cough cough
>
>
> Glenn S. Kelley,
> I am a Connectivity.Engineer
> Text and Voice Direct:  740-206-9624
>
>
> a Division of CreatingNet.Works
> IMPORTANT: The contents of this email and any attachments are
> confidential. They are intended for the named recipient(s) only. If
> you have received this email by mistake, please notify Glenn Kelley,
> the sender, immediately and do not disclose the contents to anyone or
> make copies thereof.
>
> On Wed, May 3, 2023 at 10:34 AM Tom Beecher  wrote:
> >>
> >> For those that like FRR:
> >>
> https://thehackernews.com/2023/05/researchers-uncover-new-bgp-flaws-in.html
> >
> >
> > All 3 of those CVEs look like they were fixed and backported into 8.2
> through 8.4 at least 6 months ago.
> >
> > On Wed, May 3, 2023 at 5:54 AM Hank Nussbacher 
> wrote:
> >>
> >> On 02/05/2023 17:56, Warren Kumari wrote:
> >>
> >> For those that like FRR:
> >>
> https://thehackernews.com/2023/05/researchers-uncover-new-bgp-flaws-in.html
> >>
> >> Regards,
> >> Hank
> >>
> >> > +lots.
> >> >
> >> > I've used a number of Linux routing thingies (BIRD, Quagga,
> >> > VyOS/Ubiquiti, OpenBGPd, ExBGP), and FRR is (for me at least) by far
> >> > the friendliest. It's trivial to spin this up on a cloud VM and start
> >> > announcing a prefix.
> >> >
> >> > For doing something like Anycast though (where you are mostly just
> >> > announcing a route on demand), ExaBGP is great.
> >> >
> >> > W
> >> >
> >> >
> >> > On Mon, May 01, 2023 at 2:03 PM, Jean Franco 
> wrote:
> >> >
> >> > https://frrouting.org/ 
> >> >
> >>
>

Re: Best Linux (or BSD) hosted BGP?

[Matt Corallo]

Lots of replies saying which of BIRD/exabgp/frr/quagga/openbgpd folks prefer, but they're all pretty 
good. Honestly for such a project they're all just as great, it comes down mostly to what you're 
used to config-wise. Used to big metal router configuration? You might find BIRD foreign. Used to 
more functional code stuff? BIRD is pretty great. Others I have less experience with.


As for "something better than cobbling it together", I'd recommend just do that. Install your usual 
Linux/BSD distro, install one of the BGP daemons, and run a flow logger (eg pmacctd, which can split 
out flowspec which you can consume with whatever you're comfortable with). Setting up everything 
short of the actual flowspec inspection should be a half-hour endeavor, maybe three hours if you 
have to fight with the VM provider to get BGP filters working right :).


Matt

On 5/1/23 9:01 AM, Bryan Fields wrote:
I know best subjective, but I'm looking at a project to announce some IP space that's between uses 
now and see what's there.  I'm planing to run a flow logger and ntop on the VM and see what is 
coming in if anything.  I'm looking at the options for BGP out there, and there's quite a few (other 
than running a VM with a router doing BGP), but most data I've seen is focused on scale and 
filtering use, or RPKI.  My use case is a bit different, and I can't find any best practices for 
this use case from what I've found.


That said, is there a better solution other than linux/ntop/ipt-netflow?

Re: Best Linux (or BSD) hosted BGP?

[Jakob Heitz (jheitz) via NANOG]

I just checked the Cisco IOS-XR code. It's not vulnerable to any of the 3 flaws 
listed in the below linked hackernews article.

Kind Regards,
Jakob


Date: Wed, 3 May 2023 12:52:46 +0300
From: Hank Nussbacher 

On 02/05/2023 17:56, Warren Kumari wrote:

For those that like FRR:
https://thehackernews.com/2023/05/researchers-uncover-new-bgp-flaws-in.html

Regards,
Hank

Re: Best Linux (or BSD) hosted BGP?

[Glenn Kelley]

Tom - you are correct

Of course - who keeps things like BGP Route Servers and FRR up to date -

cough cough


Glenn S. Kelley,
I am a Connectivity.Engineer
Text and Voice Direct:  740-206-9624


a Division of CreatingNet.Works
IMPORTANT: The contents of this email and any attachments are
confidential. They are intended for the named recipient(s) only. If
you have received this email by mistake, please notify Glenn Kelley,
the sender, immediately and do not disclose the contents to anyone or
make copies thereof.

On Wed, May 3, 2023 at 10:34 AM Tom Beecher  wrote:
>>
>> For those that like FRR:
>> https://thehackernews.com/2023/05/researchers-uncover-new-bgp-flaws-in.html
>
>
> All 3 of those CVEs look like they were fixed and backported into 8.2 through 
> 8.4 at least 6 months ago.
>
> On Wed, May 3, 2023 at 5:54 AM Hank Nussbacher  wrote:
>>
>> On 02/05/2023 17:56, Warren Kumari wrote:
>>
>> For those that like FRR:
>> https://thehackernews.com/2023/05/researchers-uncover-new-bgp-flaws-in.html
>>
>> Regards,
>> Hank
>>
>> > +lots.
>> >
>> > I've used a number of Linux routing thingies (BIRD, Quagga,
>> > VyOS/Ubiquiti, OpenBGPd, ExBGP), and FRR is (for me at least) by far
>> > the friendliest. It's trivial to spin this up on a cloud VM and start
>> > announcing a prefix.
>> >
>> > For doing something like Anycast though (where you are mostly just
>> > announcing a route on demand), ExaBGP is great.
>> >
>> > W
>> >
>> >
>> > On Mon, May 01, 2023 at 2:03 PM, Jean Franco  wrote:
>> >
>> > https://frrouting.org/ 
>> >
>>

Re: Best Linux (or BSD) hosted BGP?

[Tom Beecher]

>
> For those that like FRR:
> https://thehackernews.com/2023/05/researchers-uncover-new-bgp-flaws-in.html


All 3 of those CVEs look like they were fixed and backported into 8.2
through 8.4 at least 6 months ago.

On Wed, May 3, 2023 at 5:54 AM Hank Nussbacher  wrote:

> On 02/05/2023 17:56, Warren Kumari wrote:
>
> For those that like FRR:
> https://thehackernews.com/2023/05/researchers-uncover-new-bgp-flaws-in.html
>
> Regards,
> Hank
>
> > +lots.
> >
> > I've used a number of Linux routing thingies (BIRD, Quagga,
> > VyOS/Ubiquiti, OpenBGPd, ExBGP), and FRR is (for me at least) by far
> > the friendliest. It's trivial to spin this up on a cloud VM and start
> > announcing a prefix.
> >
> > For doing something like Anycast though (where you are mostly just
> > announcing a route on demand), ExaBGP is great.
> >
> > W
> >
> >
> > On Mon, May 01, 2023 at 2:03 PM, Jean Franco 
> wrote:
> >
> > https://frrouting.org/ 
> >
>
>

Re: Best Linux (or BSD) hosted BGP?

[Hank Nussbacher]

On 02/05/2023 17:56, Warren Kumari wrote:

For those that like FRR:
https://thehackernews.com/2023/05/researchers-uncover-new-bgp-flaws-in.html

Regards,
Hank


+lots.

I've used a number of Linux routing thingies (BIRD, Quagga, 
VyOS/Ubiquiti, OpenBGPd, ExBGP), and FRR is (for me at least) by far 
the friendliest. It's trivial to spin this up on a cloud VM and start 
announcing a prefix.


For doing something like Anycast though (where you are mostly just 
announcing a route on demand), ExaBGP is great.


W


On Mon, May 01, 2023 at 2:03 PM, Jean Franco  wrote:

https://frrouting.org/ 

Re: Best Linux (or BSD) hosted BGP?

[Warren Kumari]

+lots.

I've used a number of Linux routing thingies (BIRD, Quagga, VyOS/Ubiquiti,
OpenBGPd, ExBGP), and FRR is (for me at least) by far the friendliest. It's
trivial to spin this up on a cloud VM and start announcing a prefix.

For doing something like Anycast though (where you are mostly just
announcing a route on demand), ExaBGP is great.

W


On Mon, May 01, 2023 at 2:03 PM, Jean Franco  wrote:

> https://frrouting.org/
>
> On Mon, May 1, 2023 at 2:28 PM Josh Luthman 
> wrote:
>
> Doesn't VyOS simply use Quagga?
>
> On Mon, May 1, 2023 at 12:09 PM Jean Franco  wrote:
>
> Hi,
>
> VyOS
>
> Best regards,
>
> On Mon, May 1, 2023 at 1:03 PM Bryan Fields  wrote:
>
> I know best subjective, but I'm looking at a project to announce some IP
> space
> that's between uses now and see what's there.  I'm planing to run a flow
> logger and ntop on the VM and see what is coming in if anything.  I'm
> looking
> at the options for BGP out there, and there's quite a few (other than
> running
> a VM with a router doing BGP), but most data I've seen is focused on scale
> and
> filtering use, or RPKI.  My use case is a bit different, and I can't find
> any
> best practices for this use case from what I've found.
>
> That said, is there a better solution other than linux/ntop/ipt-netflow?
> --
> Bryan Fields
>
> 727-409-1194 - Voice
> http://bryanfields.net
>
>

Re: Best Linux (or BSD) hosted BGP?

[Andy Davidson]

Hi, Bryan

You wrote:
> I know best subjective, but I'm looking at a project to announce some IP space
> that's between uses now and see what's there.  I'm planing to run a flow
> logger and ntop on the VM and see what is coming in if anything.  I'm looking
> at the options for BGP out there, and there's quite a few […]

Others have pointed several great open source BGP daemons which can interface
with Linux routing.  If you just want to ignore BGP for forwarding, and point a 
default
at one place, you might like to consider ExaBGP.  This is very lightweight - it 
can do
the BGP announcing of your prefix(es) without syncing routing changes to the 
Linux
routing table.  There are some simple AS112 instances, for example, delivered 
with
this model.  https://github.com/Exa-Networks/exabgp

Andy

Re: Best Linux (or BSD) hosted BGP?

[Uesley Correa]

Hi!

I like VyOS or FRR (both in Debian Linux).

Regards,
Uesley Corrêa - Analista de Telecomunicaciones
CEO Telecom Consultoria, Entrenamiento y Servicios
CEO Telecom Fiber Solutions


On Mon, May 1, 2023 at 3:58 PM Mark Tinka  wrote:

>
>
> On 5/1/23 20:04, Tomas Jonsson wrote:
>
> > VyOS uses FRR, but they used to run quagga.
> >
> > And most bsd(?)/linux package managers has frr in their repository so
> > maybe that could be something to look at?
>
> pfSense running FRR is a pretty solid BGP router.
>
> Mark.
>

Re: Best Linux (or BSD) hosted BGP?

[Charlie]

Agreed with Mark, used pfSense running FRR a few months ago without issues.

- Charlie

From: NANOG  on behalf of Mark Tinka 

Sent: 01 May 2023 08:55 PM
To: nanog@nanog.org 
Subject: Re: Best Linux (or BSD) hosted BGP?



On 5/1/23 20:04, Tomas Jonsson wrote:

> VyOS uses FRR, but they used to run quagga.
>
> And most bsd(?)/linux package managers has frr in their repository so
> maybe that could be something to look at?

pfSense running FRR is a pretty solid BGP router.

Mark.

Re: Best Linux (or BSD) hosted BGP?

[Nickolas Stevermer via NANOG]

Hi Bryan,

Openbsd project has openbgpd built right in. Simple and security-minded
implementation.

Presentations: https://www.openbgpd.org/papers.html
Testimonials: https://www.openbgpd.org/users.html

Regards,

Nick

-- 
Confidentiality Notice: This E-mail message, including any attachments, is 
for the sole use of the intended recipient(s) and may contain confidential 
and privileged information. Any unauthorized review, use, disclosure or 
distribution is prohibited. If you are not the intended recipient, please 
contact the sender by reply E-mail and destroy all copies of the original 
message.

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/1/23 20:04, Tomas Jonsson wrote:


VyOS uses FRR, but they used to run quagga.

And most bsd(?)/linux package managers has frr in their repository so 
maybe that could be something to look at?


pfSense running FRR is a pretty solid BGP router.

Mark.

Re: Best Linux (or BSD) hosted BGP?

[Josh Luthman]

I think FRR is a fork of Quagga.

On Mon, May 1, 2023 at 2:04 PM Tomas Jonsson  wrote:

> VyOS uses FRR, but they used to run quagga.
>
> And most bsd(?)/linux package managers has frr in their repository so
> maybe that could be something to look at?
>
>
> On 23/05/01 13:27, Josh Luthman wrote:
> >Doesn't VyOS simply use Quagga?
> >
> >On Mon, May 1, 2023 at 12:09 PM Jean Franco  wrote:
> >
> >> Hi,
> >>
> >> VyOS
> >>
> >> Best regards,
> >>
> >> On Mon, May 1, 2023 at 1:03 PM Bryan Fields 
> wrote:
> >>
> >>> I know best subjective, but I'm looking at a project to announce some
> IP
> >>> space
> >>> that's between uses now and see what's there.  I'm planing to run a
> flow
> >>> logger and ntop on the VM and see what is coming in if anything.  I'm
> >>> looking
> >>> at the options for BGP out there, and there's quite a few (other than
> >>> running
> >>> a VM with a router doing BGP), but most data I've seen is focused on
> >>> scale and
> >>> filtering use, or RPKI.  My use case is a bit different, and I can't
> find
> >>> any
> >>> best practices for this use case from what I've found.
> >>>
> >>> That said, is there a better solution other than
> linux/ntop/ipt-netflow?
> >>> --
> >>> Bryan Fields
> >>>
> >>> 727-409-1194 - Voice
> >>> http://bryanfields.net
> >>>
> >>
>

Re: Best Linux (or BSD) hosted BGP?

[Tomas Jonsson]

VyOS uses FRR, but they used to run quagga.

And most bsd(?)/linux package managers has frr in their repository so maybe 
that could be something to look at?


On 23/05/01 13:27, Josh Luthman wrote:

Doesn't VyOS simply use Quagga?

On Mon, May 1, 2023 at 12:09 PM Jean Franco  wrote:


Hi,

VyOS

Best regards,

On Mon, May 1, 2023 at 1:03 PM Bryan Fields  wrote:


I know best subjective, but I'm looking at a project to announce some IP
space
that's between uses now and see what's there.  I'm planing to run a flow
logger and ntop on the VM and see what is coming in if anything.  I'm
looking
at the options for BGP out there, and there's quite a few (other than
running
a VM with a router doing BGP), but most data I've seen is focused on
scale and
filtering use, or RPKI.  My use case is a bit different, and I can't find
any
best practices for this use case from what I've found.

That said, is there a better solution other than linux/ntop/ipt-netflow?
--
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: Best Linux (or BSD) hosted BGP?

[Jean Franco]

https://frrouting.org/

On Mon, May 1, 2023 at 2:28 PM Josh Luthman 
wrote:

> Doesn't VyOS simply use Quagga?
>
> On Mon, May 1, 2023 at 12:09 PM Jean Franco  wrote:
>
>> Hi,
>>
>> VyOS
>>
>> Best regards,
>>
>> On Mon, May 1, 2023 at 1:03 PM Bryan Fields 
>> wrote:
>>
>>> I know best subjective, but I'm looking at a project to announce some IP
>>> space
>>> that's between uses now and see what's there.  I'm planing to run a flow
>>> logger and ntop on the VM and see what is coming in if anything.  I'm
>>> looking
>>> at the options for BGP out there, and there's quite a few (other than
>>> running
>>> a VM with a router doing BGP), but most data I've seen is focused on
>>> scale and
>>> filtering use, or RPKI.  My use case is a bit different, and I can't
>>> find any
>>> best practices for this use case from what I've found.
>>>
>>> That said, is there a better solution other than linux/ntop/ipt-netflow?
>>> --
>>> Bryan Fields
>>>
>>> 727-409-1194 - Voice
>>> http://bryanfields.net
>>>
>>

Re: Best Linux (or BSD) hosted BGP?

[William Herrin]

On Mon, May 1, 2023 at 9:01 AM Bryan Fields  wrote:
> I know best subjective, but I'm looking at a project to announce some IP space
> that's between uses now and see what's there.  I'm planing to run a flow
> logger and ntop on the VM and see what is coming in if anything.  I'm looking
> at the options for BGP out there, and there's quite a few (other than running
> a VM with a router doing BGP), but most data I've seen is focused on scale and
> filtering use, or RPKI.

FRR on Linux with one of the VPS providers that support BGP like Vultr.

Also consider asking CAIDA for access to Telescope -- that's where
they collect packets on unused IP addresses.

Regards,
Bill Herrin


-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: Best Linux (or BSD) hosted BGP?

[Josh Luthman]

Doesn't VyOS simply use Quagga?

On Mon, May 1, 2023 at 12:09 PM Jean Franco  wrote:

> Hi,
>
> VyOS
>
> Best regards,
>
> On Mon, May 1, 2023 at 1:03 PM Bryan Fields  wrote:
>
>> I know best subjective, but I'm looking at a project to announce some IP
>> space
>> that's between uses now and see what's there.  I'm planing to run a flow
>> logger and ntop on the VM and see what is coming in if anything.  I'm
>> looking
>> at the options for BGP out there, and there's quite a few (other than
>> running
>> a VM with a router doing BGP), but most data I've seen is focused on
>> scale and
>> filtering use, or RPKI.  My use case is a bit different, and I can't find
>> any
>> best practices for this use case from what I've found.
>>
>> That said, is there a better solution other than linux/ntop/ipt-netflow?
>> --
>> Bryan Fields
>>
>> 727-409-1194 - Voice
>> http://bryanfields.net
>>
>

Re: Best Linux (or BSD) hosted BGP?

[Michael Spears via NANOG]

I run BIRD on Ubuntu and it works well. Feel free to reach out off list Bryan if you want some examples of a basic configThank you,Michael SpearsOn May 1, 2023 12:01 PM, Bryan Fields  wrote:I know best subjective, but I'm looking at a project to announce some IP space 
that's between uses now and see what's there.  I'm planing to run a flow 
logger and ntop on the VM and see what is coming in if anything.  I'm looking 
at the options for BGP out there, and there's quite a few (other than running 
a VM with a router doing BGP), but most data I've seen is focused on scale and 
filtering use, or RPKI.  My use case is a bit different, and I can't find any 
best practices for this use case from what I've found.

That said, is there a better solution other than linux/ntop/ipt-netflow?
-- 
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: Best Linux (or BSD) hosted BGP?

[Jean Franco]

Hi,

VyOS

Best regards,

On Mon, May 1, 2023 at 1:03 PM Bryan Fields  wrote:

> I know best subjective, but I'm looking at a project to announce some IP
> space
> that's between uses now and see what's there.  I'm planing to run a flow
> logger and ntop on the VM and see what is coming in if anything.  I'm
> looking
> at the options for BGP out there, and there's quite a few (other than
> running
> a VM with a router doing BGP), but most data I've seen is focused on scale
> and
> filtering use, or RPKI.  My use case is a bit different, and I can't find
> any
> best practices for this use case from what I've found.
>
> That said, is there a better solution other than linux/ntop/ipt-netflow?
> --
> Bryan Fields
>
> 727-409-1194 - Voice
> http://bryanfields.net
>