RE: Uganda Communications Commission shutdown order

[adamv0025]

> From: Sean Donelan
> Sent: Tuesday, January 19, 2021 1:58 AM
> 
> In 2016, U.N. Human Rights Council, resolution A/HRC/RES/32/13: "condemns
> unequivocally measures to intentionally prevent or disrupt access to or
> dissemination of information online in violation of international human
rights
> law and calls on all States to refrain from and cease such measures".
> 
> https://thehill.com/policy/technology/286236-un-rights-council-condemns-
> internet-blocking
> 
> 
> Netblocks.org tracks internet shutdowns in near-realtime.  Access Now and
> Internet Society have been reporting on internet shutdowns for many years.
> 
> Top10VPN.com annual report give a reputable summary of intentional
> government internet shutdowns around the world.
> 
> https://www.top10vpn.com/cost-of-internet-shutdowns/
> The Global Cost of Internet Shutdowns in 2020
> 
> 93 major shutdowns took place in 21 countries in 2020
> 
> 27,165 hours: total duration of major disruptions around the world, up 49%
> from the previous year.
> Internet blackouts: 10,693 hours
> Internet throttling: 10,920 hours
> Social media shutdowns: 5,552 hours
>
Hopefully starlink and other similar projects will help bring these numbers
down a bit.
But I think starlink has been already outlawed in some countries? 


adam

Would you get Cisco 8201 or Juniper PTX10001-35MR and why?

[adamv0025]

Hi folks,

I'd like to get other operator's views on the Cisco 8201 versus Juniper
PTX10001-35MR choice, 

What made you (or what would make you) go for one or the other? 

 

Speeds and feeds are identical between these two, Cisco seems much better
with regards to power consumption, but Juniper's licensing scheme seems more
compelling, especially in my case since I don't need even half the Tbps day
one or 400G for that matter (just plenty of 100g ports for now).   

 

 

adam   

RE: Re Parler

[adamv0025]

Shut , so the boilerplate termination of agreement for any reason with 30 day 
notice was indeed part of the contract?
-so they forgot to bump the default up I guess, but anyways wouldn't help them 
either way.

Was there anything else in the contract that would allow amazon to terminate 
the contract in less than 30days (termination for cause maybe)?
 
(b) Termination for Cause.
(i) material breach remains uncured for a period of 30 days from receipt of 
notice  -30 days again,...
(ii) By Us. We may also terminate this Agreement immediately upon notice to you 
(A) for cause if we have the right to suspend under Section 6  <- 6. 
Temporary Suspension. -I guess that's what you was referring to right?
(B) if our relationship with a third-party partner who provides 
software or other technology we use to provide the Service Offerings expires, 
terminates or requires us to change the way we provide the software or other 
technology as part of the Services  -this would affect everybody on the 
platform hard to justify singling out one customer  
(C) in order to comply with the law or requests of governmental 
entities. -also not the case right?


Section 6:
6. Temporary Suspension.
6.1 Generally. We may suspend your or any End User’s right to access or use any 
portion or all of the Service Offerings immediately upon notice to you if we 
determine:

(a) your or an End User’s use of the Service Offerings 
(i) poses a security risk to the Service Offerings or any third party, 
(ii) could adversely impact our systems, the Service Offerings or the 
systems or Content of any other AWS customer, 
(iii) could subject us, our affiliates, or any third party to 
liability, or 
(iv) could be fraudulent;

(b) you are, or any End User is, in breach of this Agreement;

(c) you are in breach of your payment obligations under Section 5; or

(d) you have ceased to operate in the ordinary course, made an assignment for 
the benefit of creditors or similar disposition of your assets, or become the 
subject of any bankruptcy, reorganization, liquidation, dissolution or similar 
proceeding.

So if AWS acted according to section 6 then I guess only point (a) options are 
remotely plausible, but I guess any of the points there would be hard to proof. 
 
In any case it's a pretty powerful tool to have in a contract this section 6. 
(especially with subsection (a) which seem to provide a lot of options for 
interpretation and manoeuvring space)


adam

-Original Message-
From: Mel Beckman  
Sent: Thursday, January 14, 2021 5:02 PM
To: adamv0...@netconsultings.com
Cc: Keith Medcalf ; nanog@nanog.org
Subject: Re: Re Parler

I, however, do know that this is the contract that was in force. Because I read 
the lawsuit, and the contract, which I’ve verified is identical to the one 
posted online, is included as an exhibit (although the courts managed to get 
the pages out of order).

And yes, Amazon had a duty to provide 30 days notice in advance of termination. 
Amazon says they are calling this a “suspension”, but that’s weaselwording, 
because they told Parler that they had secured Parler’s data so that Parler 
could “move to another provider.” You would only do that in a termination.

Parler also has an excellent antitrust case, as the idea that three companies 
would simultaneously pull the plug on their services for a single common 
customer is going to be hard to explain to a judge. 

Right now I think Amazon’s safest escape from this mess is to restore Parlor’s 
services, and pay them damages. Otherwise, why would anyone do business with 
Amazon if they can pull the rug out with zero advance notice (Parler learned of 
Amazon’s termination from the news, since Amazon gave the media a scoop before 
notifying its customers).

However you look at this, Amazon’s actions have huge implications for anyone 
using them for operational networking.

 -mel 

> On Jan 14, 2021, at 7:48 AM, adamv0...@netconsultings.com wrote:
> 
> 
>> 
>> Medcalf
>> Sent: Thursday, January 14, 2021 1:06 PM
>> 
>> 
>>> On Thursday, 14 January, 2021 04:53, adamv0...@netconsultings.com wrote:
>>> 
>>> https://aws.amazon.com/agreement/
>>> 7.2 Termination.
>>> (a) Termination for Convenience. You may terminate this Agreement 
>>> for any reason by providing us notice and closing your account for 
>>> all Services for which we provide an account closing mechanism. We 
>>> may terminate this Agreement for any reason by providing you at 
>>> least 30 days’ advance notice.
>> 
>> How do you know that this is the contract that was in effect?  
> No you're right I don't and neither do you so arguing about whether the 
> Communication Decency Act section 230 was violated is useless without knowing 
> contents of the contract. 
> 
> 
> 
>>> With regards to business continuity,
>> 
>>> My experience is that the above is a standard clause in all 
>>> contracts (and 30 days is pretty standard as well),
>> 
>> I 

RE: Re Parler

[adamv0025]

> Medcalf
> Sent: Thursday, January 14, 2021 1:06 PM
> 
> 
> On Thursday, 14 January, 2021 04:53, adamv0...@netconsultings.com wrote:
> 
> >https://aws.amazon.com/agreement/
> >7.2 Termination.
> >(a) Termination for Convenience. You may terminate this Agreement for
> >any reason by providing us notice and closing your account for all
> >Services for which we provide an account closing mechanism. We may
> >terminate this Agreement for any reason by providing you at least 30
> >days’ advance notice.
> 
> How do you know that this is the contract that was in effect?  
No you're right I don't and neither do you so arguing about whether the 
Communication Decency Act section 230 was violated is useless without knowing 
contents of the contract. 



> >With regards to business continuity,
> 
> >My experience is that the above is a standard clause in all contracts
> >(and 30 days is pretty standard as well),
> 
> I have never ever seen that in any contract to which I am a party.  
Well let's just say our experience on the matter differs.


> 
> >If say Cisco tells you one day that "in 30days we stop taking your
> >support calls cause we don't feel like working with you anymore", and
> >you'd be like omg the license on the 32x100G core cards will expire in
> >2 months and I can't renew cause these guys won't talk to me anymore.
> 
> So, that is Cisco's problem, not yours.  I would take the position that if 
> Cisco
> no longer wants to take money then that is their choice and it has absolutely
> zero effect on the validity of the license (in fact, the license is now free).
Well taking this position is of no real help if the router (or any product) 
stops working after a period of time (i.e. when licenses expire).
The only valid question then is whether one can migrate off of the product onto 
something else in time. 

> Of
> course, it depends on what the contract says, if it says anything at all that 
> is
> relevant.
> 
And that was the question I was trying to raise, to see whether/how folks 
usually capture this eventuality in their contracts with vendors. 



> >Also an interesting business continuity case is when a vendor goes
> >under (yes highly unlikely in case of AWS/Juniper/Cisco/etc..), but do
> >you have contractual terms governing this case?
> 
> >What if you're licenses are about to expire say in 2 months and the
> >vendor goes under? Even if the product still works can you actually
> >legally use it? Do you own it then? Etc..
> 
> Yes.
Hmm, that doesn’t feel right, so if it just so happens that while I'm renting a 
car the rental company goes under I now own the car?
I mean if a license gives me right to use a product for a period of time and 
when the license expires, wouldn't my right to use the product expire as well? 
(i.e. regardless of the fact that I didn't get a chance to renew the license, 
cause well the entity with which I could do so doesn't exist anymore).

adam  

RE: Re Parler

[adamv0025]

https://aws.amazon.com/agreement/
7.2 Termination.
(a) Termination for Convenience. You may terminate this Agreement for any 
reason by providing us notice and closing your account for all Services for 
which we provide an account closing mechanism. We may terminate this Agreement 
for any reason by providing you at least 30 days’ advance notice.

How/where does the above violate the Communication Decency Act section 230?
See AWS can claim they terminated the contract because it was sunny outside and 
they just felt like it, in other words using section 7.2 (a) of their customer 
agreement.


With regards to business continuity, 

My experience is that the above is a standard clause in all contracts (and 30 
days is pretty standard as well),
I know we negotiated longer advance notices with some of our vendors, but I'm 
actually not sure what it says on our contracts with say big router vendors? 
Do you folks?
If say Cisco tells you one day that "in 30days we stop taking your support 
calls cause we don't feel like working with you anymore", and you'd be like omg 
the license on the 32x100G core cards will expire in 2 months and I can't renew 
cause these guys won't talk to me anymore.

Also an interesting business continuity case is when a vendor goes under (yes 
highly unlikely in case of AWS/Juniper/Cisco/etc..), but do you have 
contractual terms governing this case? 
What if you're licenses are about to expire say in 2 months and the vendor goes 
under? Even if the product still works can you actually legally use it? Do you 
own it then? Etc..
   

adam

-Original Message-
From: NANOG  On Behalf Of 
Keith Medcalf
Sent: Thursday, January 14, 2021 10:08 AM
To: nanog@nanog.org
Subject: RE: Re Parler


I thought y'all yankee doodles had this thing called the Communication Decency 
Act section 230 that prevented a "service provider" from being responsible for 
the content of third-party's -- whether or not they were acting as a publisher; 
and, also the principle of law that an agreement to violate the law (as in a 
Contract which ignored that provision that the "service provider" was not 
liable for the content provided by third-parties) was nul ab initio?

Therefore it would appear to me that AWS has not a leg to stand on, that the 
terms of the contract which violate section 230 constitute a prior agreement to 
violate the law and therefore are a nullity, and that Parler is entitled to 
specific performance of the contract and/or damages, including aggravated or 
punitive damages, from Amazon.

The only exception would be if the "content" were Criminal and that would 
require a court finding that the content was Criminal but, such facts not in 
evidence, Amazon has violated the law and should be held liable.  You cannot 
convict someone of murder and have them executed simply because they have a 
hand which may hold a gun which may then be used to commit murder in order to 
prevent the murder.  

First there must be establishment of the fact of the murder, not the mere 
establishment of a hypothetical fantasy of fact.

But then again it is likely that the lawyers representing Parler are of low 
ability and unable to make the case required.

--
Be decisive.  Make a decision, right or wrong.  The road of life is paved with 
flat squirrels who could not make a decision.

>-Original Message-
>From: NANOG  On Behalf Of 
>Jeff P
>Sent: Wednesday, 13 January, 2021 10:43
>To: nanog@nanog.org
>Subject: Re Parler
>
>ICYMI: Amazon's response to Parler Antitrust relief:
>
>https://cdn.pacermonitor.com/pdfserver/LHNWTAI/137249864/Parler_LLC_v_A
>ma zon_Web_Services_Inc__wawdce-21-00031__0010.0.pdf
>
>JeffP
>je...@jeffp.us 
>
>

RE: The Real AI Threat?

[adamv0025]

> Automated resource discovery + automated resource allocation = recipe for 
> disaster

That is literally how OpenStack works. 

 

For now, don’t worry about AI taking away your freedom on its own, rather worry 
about how people using it might…

 

 

adam

 

From: NANOG  On Behalf Of 
Miles Fidelman
Sent: Thursday, December 10, 2020 2:44 PM
To: 'NANOG' 
Subject: Re: The Real AI Threat?

 

adamv0...@netconsultings.com   wrote:

> Put them together, and the nightmare scenario is:
> - machine learning algorithm detects need for more resources
All good so far
 
> - machine learning algorithm makes use of vulnerability analysis library 
> to find other systems with resources to spare, and starts attaching
> those resources

Right so a company would built, trained and fine-tuned an AI, or would have 
bought such a product and implemented it as part of its NMS/DDoS mitigation 
suite, to do the above? 

What is the probability of anyone thinking that to be a good idea?

To me that does sound like an AI based virus rather than a tool one would want 
to develop or buy from a third party and then integrate into the day to day 
operations.

 

You can’t take for instance alpha-0 or GPT-3 and make it do the above. You’d 
have to train it to do so over millions of examples and trials. 

Oh and also these won’t “wake up” one day and “think” to themselves oh I’m fed 
up with Atari games I’m going to learn myself some chess and then do some 
reading on wiki about the chess rules. 


Jeez... some guys seem to take a joke literally - while ignoring a real and 
present danger - which was the point.

Meanwhile, yes, I think that a poorly ENGINEERED DDoS mitigation suite might 
well have failure modes that just keep eating up resources until systems start 
crashing all over the place.  Heck, spinning off processes until all available 
resources have been exhausted has been a failure mode of systems for years.  
Automated resource discovery + automated resource allocation = recipe for 
disaster.  (No need for AIs eating the world.)

Miles







-- 
In theory, there is no difference between theory and practice.
In practice, there is.   Yogi Berra
 
Theory is when you know everything but nothing works. 
Practice is when everything works but no one knows why. 
In our lab, theory and practice are combined: 
nothing works and no one knows why.  ... unknown

RE: The Real AI Threat?

[adamv0025]

> Put them together, and the nightmare scenario is:
> - machine learning algorithm detects need for more resources
All good so far
 
> - machine learning algorithm makes use of vulnerability analysis library 
> to find other systems with resources to spare, and starts attaching
> those resources

Right so a company would built, trained and fine-tuned an AI, or would have 
bought such a product and implemented it as part of its NMS/DDoS mitigation 
suite, to do the above? 

What is the probability of anyone thinking that to be a good idea?

To me that does sound like an AI based virus rather than a tool one would want 
to develop or buy from a third party and then integrate into the day to day 
operations.

 

You can’t take for instance alpha-0 or GPT-3 and make it do the above. You’d 
have to train it to do so over millions of examples and trials. 

Oh and also these won’t “wake up” one day and “think” to themselves oh I’m fed 
up with Atari games I’m going to learn myself some chess and then do some 
reading on wiki about the chess rules. 

 



adam

 

 

From: NANOG  On Behalf Of 
Miles Fidelman
Sent: Wednesday, December 9, 2020 7:07 PM
To: NANOG 
Subject: The Real AI Threat?

 

Hi Folks,

It occurs to me that network & systems admins are the the folks who really have 
to worry about AI threats.
 
After watching yet another AI takes over the world show - you know, the 
same general theme, AI wipes out humans to preserve its existence - it 
occurred to me:
 
Perhaps the real AI threat is "self-healing systems" gone wild. Consider:
 
- automated system management
- automated load management
- automated resource management - spin up more instances of  
as necessary
- automated threat detection & response
- automated vulnerability analysis & response
 
Put them together, and the nightmare scenario is:
- machine learning algorithm detects need for more resources
- machine learning algorithm makes use of vulnerability analysis library 
to find other systems with resources to spare, and starts attaching 
those resources
- unbounded demand for more resources
 
Kind of what spambots have done to the global email system.
 
"For Homo Sapiens, the telephone bell had tolled."
(Dial F for Frankenstein, Arthur C. Clarke)
 
I think I need to start putting whisky in my morning coffee.  And maybe not 
thinking 
about NOT replacing third shift with AI tools.
 
Miles Fidelman
-- 
In theory, there is no difference between theory and practice.
In practice, there is.   Yogi Berra
 
Theory is when you know everything but nothing works. 
Practice is when everything works but no one knows why. 
In our lab, theory and practice are combined: 
nothing works and no one knows why.  ... unknown

RE: Telia Not Withdrawing v6 Routes

[adamv0025]

> Saku Ytti
> Sent: Tuesday, November 17, 2020 6:55 AM
> 
> On Tue, 17 Nov 2020 at 03:40, Sabri Berisha  wrote:
> 
> Hey Sabri,
> 
> > Also, in the case that I described it wasn't a Junos device. Makes me
> > wonder how bugs like that get introduced. One would expect that after
> > 20+ years of writing BGP code, handling a withdrawl would be easy-peasy.
> 
> I don't think this is related to skill, that there was some hard programming
> problem that DE couldn't solve. These are honest mistakes.
> I've not experienced in my tenure the frequency of these bugs change at all,
> NOS are as common now as they were in the 90s.
> 
> I put most of the blame on the market, we've modelled commercial router
> market so that poor quality NOS is good for business and good quality NOS is
> bad for business, I don't think this is in anyone's formal business plan or 
> that
> companies even realise they are not even trying to make good NOS. I think it's
> emergent behaviour due to the market and people follow that market demand
> unknowingly.
> If we suddenly had one commercial NOS which is 100% bug free, many of their
> customers would stop buying support, would rely on spare HW and Internet
> forums for configuration help. Lot of us only need contracts to deal with 
> novel
> bugs all of us find on a regular basis, so good NOS would immediately reduce
> revenue. For some reason Windows, macOS or Linux almost never have novel
> bugs that the end user finds and when those are found, it's big news. While we
> don't go a month without hitting a novel bug in one of our NOS, and no one
> cares about it, it's business as usual.
> 
> I also put a lot of blame on C, it was a terrific language when compiling had 
> to
> be fast. Basically macro assembler. Now the utility of being 'close to HW' is
> gone, as the CPU does so much C compiler has no control over, it's not really
> even executing the same code as-written anymore. MSFT estimated >70% of
> their bugs are related to memory safety. We could accomplish significant
> improvements in software quality if we'd ditch C and allow the computer to do
> more formal correctness checks at compile time and design languages which
> lend towards this.
> 
> 
> We constantly misattribute problems (like in this post) to config or HW, while
> most common reasons for outages are pilot error and SW defect, and very little
> engineering time is spent on those. And often the time spent improving the two
> first increases the risk of the two latter, reducing mean availability over 
> time.
> 
I agree with everything but the last statement. 
>From my experience, most of the SPs spend a considerable time testing for SW 
>defects on features (and combinations of features) that will be used and at 
>scale intended, that's how you identify most of the bugs. What you're left 
>with afterwards are special packets of death or some slow memory leaks 
>(basically the more exotic stuff).
 
adam
 

RE: Telia Not Withdrawing v6 Routes

[adamv0025]

> On Behalf Of Mark Tinka
> Sent: Tuesday, November 17, 2020 4:32 PM
> 
> On 11/17/20 08:54, Saku Ytti wrote:
> 
> > I put most of the blame on the market, we've modelled commercial
> > router market so that poor quality NOS is good for business and good
> > quality NOS is bad for business, I don't think this is in anyone's
> > formal business plan or that companies even realise they are not even
> > trying to make good NOS. I think it's emergent behaviour due to the
> > market and people follow that market demand unknowingly.
> > If we suddenly had one commercial NOS which is 100% bug free, many of
> > their customers would stop buying support, would rely on spare HW and
> > Internet forums for configuration help.
> 
> Not to mention that many of us would not need to be around to babysit all this
> dodgy software.
> 
> Definitely bad for business :-).
> 
Being obsoleted already by "self-driving networks", there's no limit to what 
one can automate...
But then one needs someone to babysit all the automation systems.

adam

RE: cheap MPLS router recommendations

[adamv0025]

> Eric Kuhnke 
> Sent: Monday, October 26, 2020 6:52 PM
>
> If we're talking about whitebox router and ipifusion, what we're really 
> talking
> about is vyatta/vyOS and the linux foundation DANOS stuff on an ordinary x86-
> 64 server that has a weird shape.
> 
> https://www.ipinfusion.com/commercial-version-of-danos-product-page/
> 
> https://www.danosproject.org/
> 
> In which case it really comes down to how comfortable you are with the
> feature sets of the individual daemons contained within Vyatta/VyOS derived
> products (FRR, etc), and then your trust level in the hardware. Typically
> something such as a Taiwanese industrial/embedded platform manufacturer
> such as Lanner:
> 
> http://www.lannerinc.com/products/network-appliances/x86-rackmount-
> network-appliances
> 
Yes I agree and I have been looking at Lanner (NCA-2510, NCA-2512)  -and I'll 
be thankful if I could be pointed towards other companies "producing funny 
looking servers" for comparison. 
I've been also looking at the whole, a bit confusing, ecosystem of 
DANOS/Vytta/VyOS and their capabilities through my PE requirement optics (i.e. 
MPLS L3VPN, RSVP-TE/SR-TE & fast reroute, NETCONF-YANG) -which these NOS-es 
seem to have very little to no support for.


> I guess the point I'm trying to make above is that **if** you're confident in
> both the SW and HW, you can disaggregate your choice of software
> (vyatta/vyos/DANOS etc) from your own choice of hardware to best fit your
> needs, rather than purchasing it together as a package.
> 
I agree, however I need this whole SW to HW mapping and all associated pain of 
interworking and operational support to be absorbed and nicely packaged by a 
vendor.
On that note, can Lanner fix me up with some DANOS/Vytta/VyOS on their NCA-2510 
or NCA-2512 please? 

Or in more general terms, are there folks out there that specialize in 
providing support for NOS X, Y, Z  with HW X, Y, Z -in any combination?
-so that I could just pick NOS X with HW Y from their portfolio based on my 
project requirements and get similar support I'd get from a "standard" vendor?

adam

RE: Ingress filtering on transits, peers, and IX ports

[adamv0025]

> Randy Bush
> Sent: Tuesday, October 20, 2020 6:19 AM
> 
> term blocked-ports {
> from {
>   protocol [ tcp udp ];
>   first-fragment;
>   destination-port
>   [ 0 sunrpc 135 netbios-ns netbios-dgm netbios-ssn 111 445 syslog
> 11211];
>   }
> then {
>   sample;
>   discard;
>   }
> }
> 
Actually what's the latest in the net neutrality talks? Shouldn't these be
just rate-limited rather than blocked? -transit traffic.
(assuming ICMP is the only thing that can talk to infrastructure ranges &
BGP to selected IPs with rest being dropped)

adam

RE: cheap MPLS router recommendations

[adamv0025]

Just to clarify what cheap means, ideally  -$2000 to $4000 new 

-new is preferred as buying used kit on second hand market one is at the mercy 
of the price fluctuations and availability.

 

And the likes of the M2400 looks good 4x10G plus some 1G, unfortunately there 
are no details on the webpage (and the datasheet can’t be downloaded… ) 

 

Are there more folks out there bundling open NOS and white-box HW along with 
the support for the whole thing?

 

 

adam

 

From: NANOG  On Behalf Of 
Colton Conor
Sent: Monday, October 19, 2020 4:51 PM
To: t...@pelican.org
Cc: NANOG 
Subject: Re: cheap MPLS router recommendations

 

I haven't tried one myself, but Dasan Zhone has the M2400 and M3000. Basically, 
a whitebox with IP Infusion code on it. New, I think the price point is sub 
$2000 to $4000 new. That's a ton of ports for that price point. Anyone tried 
these yet?  https://dzsi.com/product-category/mobile-xhaul/ 

 

 

On Mon, Oct 19, 2020 at 3:38 AM t...@pelican.org   
mailto:t...@pelican.org> > wrote:

On Saturday, 17 October, 2020 00:41, "Tony Wicks" mailto:t...@wicks.co.nz> > said:

> Well, there is always the MX104 (if you want redundancy) or MX80 if you
> don’t. That will give you 80gig wire speed just don’t load it up with
> more than one full table.

Bear in mind that the MX80 is now in the EoL process, you have <4 years of 
support left.  Depending on your expected life-time / depreciation rules, 
buying one new right now might be unwise.

Do *not* throw a full table at it (or any of the PowerPC Junipers) unless you 
have a lot of patience for reconvergence, and black-holes while you wait.

MX104 is a nice box for getting dual-RE in something relatively compact and 
cheap, and has environmental hardening if that matters to you, but is still not 
best pleased with full tables.

OP could do with clarifying "cheap" :)

Regards,
Tim.

RE: cheap MPLS router recommendations

[adamv0025]

For this particular gig even the MX204 would be overkill in terms of price as 
well as performance. 

Ideally something like 204 but with only those 8 10/1G ports (i.e. without the 
4x100G ports)

 

adam

From: Tony Wicks  
Sent: Friday, October 16, 2020 10:36 PM
To: adamv0...@netconsultings.com
Cc: nanog@nanog.org
Subject: RE: cheap MPLS router recommendations 

 

Juniper MX204, easy

 

From: NANOG mailto:nanog-bounces+tony=wicks.co...@nanog.org> > On Behalf Of 
adamv0...@netconsultings.com  
Sent: Saturday, 17 October 2020 10:31 am
To: 'Jakub Horn (jakuhorn)' mailto:jakuh...@cisco.com> >; 
nanog@nanog.org  
Subject: RE: cheap MPLS router recommendations 

 

Yeah the XR thing would be great but NCS540 would be too expensive and too much 
throughput meaning draws too much power,

 

adam 

RE: cheap MPLS router recommendations

[adamv0025]

Yeah the XR thing would be great but NCS540 would be too expensive and too much 
throughput meaning draws too much power,

 

adam 

 

From: Jakub Horn (jakuhorn)  
Sent: Friday, October 16, 2020 10:08 PM
To: adamv0...@netconsultings.com; nanog@nanog.org
Subject: Re: cheap MPLS router recommendations 

 

If cisco, I wouldn’t consider 920… I think NCS540 is better option.

XR based, feature rich  and here are different models supporting different BW, 
different port density…

I think price/performance ratio is superb!

 

For better evaluation you should be more specific about scale, precise about 
required features…..

 

Disclaimer: cisco employee…..

 

Cheers

-j

 

From: NANOG mailto:nanog-bounces+jakuhorn=cisco@nanog.org> > on behalf of 
"adamv0...@netconsultings.com  " 
mailto:adamv0...@netconsultings.com> >
Date: Friday, 16 October 2020 at 22:59
To: "nanog@nanog.org  " mailto:nanog@nanog.org> >
Subject: cheap MPLS router recommendations 

 

Hi folks,

 

I’m looking for recommendations on a cheap MPLS router (L3VPNs RSVP-TE and BFD).

Around 60G throughput would do , heck even 30G. 

Few 1/10G ports.

But netconf yang is almost a must.

 

You know something like asr920 or juniper equivalent, but something that that 
is not EoS or EoL

Oh and something not from China (you know, bad PR)… 

 

Any pointers much appreciated.

 

adam

 

 

cheap MPLS router recommendations

[adamv0025]

Hi folks,

 

I'm looking for recommendations on a cheap MPLS router (L3VPNs RSVP-TE and
BFD).

Around 60G throughput would do , heck even 30G. 

Few 1/10G ports.

But netconf yang is almost a must.

 

You know something like asr920 or juniper equivalent, but something that
that is not EoS or EoL

Oh and something not from China (you know, bad PR). 

 

Any pointers much appreciated.

 

adam

 

 

RE: Ingress filtering on transits, peers, and IX ports

[adamv0025]

> Chris Adams
> Sent: Thursday, October 15, 2020 3:59 PM
> 
> Once upon a time, adamv0...@netconsultings.com
>  said:
> > Actually ideally there would be a feature/knob to automatically sync BGP
> (and static routes) with packet filters.
> 
> Junos has prefix-lists that can be referenced in both BGP policy and
firewall
> statements.
>
What I mean was a firewall filter that would get updated with accordance to
prefixes received/advertised via BGP, even better if this was a default
behaviour.


adam 

RE: Ingress filtering on transits, peers, and IX ports

[adamv0025]

> From: Saku Ytti 
> Sent: Thursday, October 15, 2020 3:30 PM
> 
> On Thu, 15 Oct 2020 at 17:22, Tim Durack  wrote:
> 
> 
> > We deploy urpf strict on all customer end-host and broadband circuits. In
> this scenario urpf = ingress acl I don't have to think about.
> 
> But you have to think about what prefixes a customer has. If BGP you need
> to generate prefix-list, if static you need to generate a static route. As you
> already have to know and manage this information, what is the incremental
> cost to also emit an ACL?
> 
Actually ideally there would be a feature/knob to automatically sync BGP (and 
static routes) with packet filters.

adam 

RE: Ingress filtering on transits, peers, and IX ports

[adamv0025]

> From: Saku Ytti 
> Sent: Thursday, October 15, 2020 11:12 AM
> 
> Hey,
> 
Hey Saku,

> > All stub autonomous systems should have a simple egress ACL allowing
> only PI of their customers and their own PAs -it’s a simple ACL at each 
> AS-Exit
> points (towards transits/peers), that’s it.
> >
> > -not sure why this isn’t the first sentence in every BCP and “security
> > bulletin”…
> 
> I will venture a guess.
> 
>   1) it's very specific scenario to be stubby and have downstream PI
Yes it is and in most cases it's about "I have these few /XYs from which I 
address my customers (eyeballs) so nothing outside of these should ever leave 
my AS" -as simple as that.

>   2) it won't address customers spoofing each other arbitrarily and
> customer1 spoofing as customer2 on the internet, giving large chunk of the
> utility of spoofing even with protection in place
> 
Yes its not 100% effective as you pointed out,
However if every stub AS implemented this egress ACL on the few of their 
transit/peering links, there would be a lot less inter-AS garbage floating 
around.

> How do you maintain that ACL? 
Well you're gonna update the ACL every time you acquire a new PA space (every 
now and then), or when another customer wants you route their PI (rare, simply 
cause you're a stub AS with mostly eyeballs).
It's one ACL you need to update on every box with AS-exit links -usually there 
are not that many in a stub-AS. 
  
> Why doesn't that same mechanism allow
> ingress ACL on the customer port? Your proposal looks low utility for work
> needed.
> 
Yes one should absolutely do that, but...
But considering to become a good netizen what is more work? 
a) Testing and the enabling uRPF on every customer facing box or setting up 
precise ACLs on every customer facing port, and then maintaining all that?
b) Gathering  all your PAs (potentially PIs) (hint: show bgp nei x.x.x.x 
advertised routes) crafting an ACL and apply it on several peering/transit 
links?
One of them is couple of weeks work and one is an afternoon job.

adam

RE: Ingress filtering on transits, peers, and IX ports

[adamv0025]

Simple, 

All stub autonomous systems should have a simple egress ACL allowing only PI of 
their customers and their own PAs -it’s a simple ACL at each AS-Exit points 
(towards transits/peers), that’s it.

-not sure why this isn’t the first sentence in every BCP and “security 
bulletin”…

 

 

adam 

 

From: NANOG  On Behalf Of 
Baldur Norddahl
Sent: Thursday, October 15, 2020 8:38 AM
To: nanog@nanog.org
Subject: Re: Ingress filtering on transits, peers, and IX ports

 

All DNS resolvers discovered on our network belong to customers. Our own 
resolvers, running unbound, were not discovered. 

 

While filtering same AS on ingress could help those customers (but only one was 
a open relay), filtering bogons is something the customer can also do. Or the 
software can be fixed. Do we really expect the ISP to implement firewalls 
instead of customers upgrading software?

 

I also note that apparently our own ISPs (transits) do not filter bogons 
either. 

 

The above is a principal question. I am going to filter bogons, it just is not 
very high on my long list of stuff to do. 

 

Regards 

 

Baldur 

 

 

ons. 14. okt. 2020 20.53 skrev Casey Deccio mailto:ca...@deccio.net> >:

Hi Bryan,

> On Oct 14, 2020, at 12:43 PM, Bryan Holloway   > wrote:
> 
> I too would like to know more about their methodology

We've written up our methodology and results in a paper that will be available 
in a few weeks.  Happy to post it here if folks are interested.  Obviously, no 
networks are individually identified; it's all aggregate.

Also, we're working on a self-test tool, but it's not quite ready yet.  Sorry.

> and actual tangibles ideally in the form of PCAPs.

What do you mean by "tangibles in the form of PCAPs"?

Casey

RE: Juniper configuration recommendations/BCP

[adamv0025]

Here's a fun one.
By default Junos accepts extended communities on any BGP session (not just
on MP-BGP sessions like it's the default case on cisco -unless explicitly
enabled).
Since most operators are not aware of this default Junos behaviour, one can
be importing routes to interesting places if one were so inclined.  

-so yeah bleach unwanted communities on ingress (bleach those that would
interfere with the ones used by the AS internally -so called
"untaggable"/"untouchable" ).  

adam

> -Original Message-
> From: NANOG  bounces+adamv0025=netconsultings@nanog.org> On Behalf Of
> Chriztoffer Hansen
> Sent: Thursday, October 8, 2020 11:05 AM
> To: nanog@nanog.org
> Subject: Juniper configuration recommendations/BCP
> Importance: Low
> 
> 
> On 08/10/2020 11:37, Forrest Christian (List Account) wrote:
> > Is there anything I should worry about which is Juniper-specific?
> 
> JUNOS default ARP timeout: 20 min.
> 
> If you connect to IXP's. Recommended ARP timeout: 4 hours.

RE: BGP Community - AS0 is de-facto "no-export-to" marker - Any ASN reserved to "export-only-to"?'

[adamv0025]

Reg the BCP38 vs RFC I guess is point in case (standard or best practice, the 
adoption is still low)

 

Reg the community tagging design, 

Well it’s daily job of architects/designers to come up with new designs, 
standards and frameworks that can then be adopted by engineering/ops or 
automation system workflows or the business as a whole.  

Now would it be useful to have a reference design for various L2VPN options, or 
RR infrastructures, hub-spoke RT allocations, community tagging designs, or BGP 
input sanity checking, if nothing else like food for thought, sure…

 

adam

 

From: Jeff Tantsura  
Sent: Wednesday, September 9, 2020 6:01 PM
To: adamv0...@netconsultings.com
Cc: nanog@nanog.org
Subject: Re: BGP Community - AS0 is de-facto "no-export-to" marker - Any ASN 
reserved to "export-only-to"?'

 

BCP38 is an RFC, 2827.

It is a grand advise if you can:

-find someone who is actually well versed

-afford that someone.

 

Personally - when in early 2000s I had to write complete community tagging 
design for a multi country network, I wish I had  a “how to” 

Regards,

Jeff





On Sep 9, 2020, at 15:35, adamv0...@netconsultings.com 
<mailto:adamv0...@netconsultings.com>  wrote:



My advice to “someone who is setting up a new ISP and has a very little clue as 
where to start” would be just don’t and instead hire someone who’s well versed 
in this topic.

But I see what you mean, RFC7938 was a good food for thought. But at the same 
time I’m sceptical, for instance would it help if BCP38 was an RFC? 

Would be nice for instance if the community could put together a checklist of 
things to consider for ISPs (could be in no particular order) (and actually 
there are such lists albeit concentrated around security)   

 

adam

 

From: Jeff Tantsura mailto:jefftant.i...@gmail.com> > 
Sent: Wednesday, September 9, 2020 9:52 AM
To: adamv0...@netconsultings.com <mailto:adamv0...@netconsultings.com> 
Cc: nanog@nanog.org <mailto:nanog@nanog.org> 
Subject: Re: BGP Community - AS0 is de-facto "no-export-to" marker - Any ASN 
reserved to "export-only-to"?'

 

I don’t think, anyone has proposed to use ‘’reserved ASNs” as a BCP, example of 
“ab”use of ASN0 is a de-facto artifact (unfortunate one).

My goal would be to provide a viable source of information to someone who is 
setting up a new ISP and has a very little clue as where to start. Do’s and 
don’t’s wrt inter-domain communities use. 

 

I really enjoyed the difference RFC7938 (Use of BGP for Routing in Large-Scale 
Data Centers) made, literally 100s of companies have used it to educate 
themselves/ implemented their DC networking.

 

Cheers,

Jeff






On Sep 9, 2020, at 10:04, adam via NANOG mailto:nanog@nanog.org> > wrote:



I don’t agree with the use of reserved ASNs, let alone making it BCP, cause it 
defeats the whole purpose of the community structure.

Community is basically sending a message to an AS. If I want your specific AS 
to interpret the message I set it in format YOUR_ASN:, your AS 
in the first part of the community means that your rules of how to interpret 
the community value apply.

Turning AS#0 or any other reserved AS# into a “broadcast-AS#” in terms of 
communities (or any other attribute for that matter) just doesn’t sit right 
with me (what’s next? multicast-ASNs that we can subscribe to?).

All the examples in Robert’s draft or wide community RFC, all of them use an 
example AS# the community is addressed to (not some special reserved AS#).

 

Also should something like this become standard it needs to be properly 
standardized and implemented as a well-known community by most vendors (like 
RFCs defining the wide communities or addition to standard communities like 
no_export/no_advertise/…). This would also eliminate the adoption friction from 
operators rightly claiming “my AS my rules”.   

 

adam

 

 

From: NANOG mailto:nanog-bounces+adamv0025=netconsultings@nanog.org> > On Behalf Of 
Douglas Fischer via NANOG
Sent: Tuesday, September 8, 2020 4:56 PM
To: NANOG mailto:nanog@nanog.org> >
Subject: BGP Community - AS0 is de-facto "no-export-to" marker - Any ASN 
reserved to "export-only-to"?'

 

Most of us have already used some BGP community policy to no-export some routes 
to some where.

On the majority of IXPs, and most of the Transit Providers, the very common 
community tell to route-servers and routers "Please do no-export these routes 
to that ASN" is:

 -> 0:

 

So we could say that this is a de-facto standard.

 

 

But the Policy equivalent to "Please, export these routes only to that ASN" is 
very varied on all the IXPs or Transit Providers.

 

 

With that said, now comes some questions:

1 - Beyond being a de-facto standard, there is any RFC, Public Policy, or 
something like that, that would define 0: as "no-export-to" standard?

 

2 - What about reser

RE: TCP and UDP Port 0 - Should an ISP or ITP Block it?

[adamv0025]

> William Herrin
> Sent: Tuesday, August 25, 2020 4:20 PM
> 
> On Tue, Aug 25, 2020 at 4:15 AM Douglas Fischer
>  wrote:
> > a) Should an ISP block that Kind of traffic?
> 
> Hi Douglas,
> 
> Generally speaking the answer is NO, You should not presume that your
> understanding of your customers' data traffic is sufficiently complete or
> correct to make blocking decisions for them.
> 
Agree, but there are less invasive options as well like rate limiting or comb 
rate-limiting (i.e. rate-limiter per address range).


> > b) Should a Transit Provider block that Kind of traffic?
> 
> Preemptively? Never. If I found my business transit provider was doing this,
> I'd treat it as a breach of contract.
> 
Agree, but again one can still do proactive rate limit based on historical data 
(to address the hit and run type of attacks -that exploit the reactive 
application of filters).  

adam

RE: Has virtualization become obsolete in 5G?

[adamv0025]

> From: Mark Tinka 
> Sent: Tuesday, August 11, 2020 7:45 PM
> 
> On 11/Aug/20 17:55, adamv0...@netconsultings.com wrote:
> 
> > Can you elaborate?
> > Apart from licensing scheme what stops one from redirecting traffic to
> > one vTMS instance per say each transit link or per destination /24
> > (i.e. horizontal scaling)? (vTMS is not stateful or is it?)
> 
> In an effort to control costs, we considered a vTMS from Arbor.
> 
> Even Arbor didn't recommend it, which was completely unsurprising.
> 
> Arbor can flog you a TMS that can sweep 10Gbps, 20Gbps, 40Gbps or
> 100Gbps worth of traffic. I don't see how you can run that kind of traffic in 
> a
> VM.
> 
Fair enough, but you actually haven't answered my question about why you think 
that VNFs such as vTMS can not be implemented in a horizontal scaling model? 
In my opinion any NF virtual or physical can be horizontally scaled. 

> 
> 
> > Can you please point out any efforts where operators are trying to
> standardize the orchestration piece?
> 
> NETCONF, YANG, LSO.
> 
Right, and of these 3 you mentioned, what is it that you'd say operators are 
waiting for to get standardized, in order for them to start implementing 
network services orchestration?

> 
> > I think industry is not falling over on this just progressing at steady rate
> while producing artefacts in the process that you may or may not want to use
> (I actually find them very useful and not impeding).
> 
> What's 10 years between friends :-)...
> 
> 
> > Personally, I don't need a standard on how I should orchestrate network
> services. There are very interesting and useful ideas, or better put
> "frameworks", that anyone can follow (and most are), but standardizing
> these, ...no point in my opinion.
> 
> Now that's something we can agree on... and once folk realize that getting
> your solution going is the end-goal - rather than bickering over whether
> NETCONF or YANG or SSH or whatever should be the BCOP - is when we shall
> finally see some real progress.
> 
> Personally, I don't really care of you choose to keep CLI or employ thousands
> of software heads to automate said CLI. As long as you are happy and not
> wasting time taking every meeting from every vendor about "automation".
> 
Agreed, all I'm trying to understand is what makes you claim things like: 
progress is slow, or there's a lack of standardization, or operators need to 
wait till things get standardized in order to start doing network service 
orchestration... 
I'm asking cause I just don't see that. My personal experience is quite 
different to what you're claiming. 

Yes the landscape is quite diverse ranging from fire and forget CLI scrapers 
(Puppet, Chef, Ansible, SaltStack) through open network service orchestration 
frameworks all the way to a range of commercial products for network service 
orchestration, but the point is options are there and one can start today, no 
need to wait for anything to get standardized or things to settle.  
 

adam
 

RE: Has virtualization become obsolete in 5G?

[adamv0025]

> From: Mark Tinka 
> Sent: Tuesday, August 11, 2020 2:19 PM
> 
> On 10/Aug/20 15:15, adamv0...@netconsultings.com wrote:
> 
> > Mark,
> > 1) first you have your edge - lots of small instances that are meant
> > to be horizontally scaled (not vertically- i.e. not 40's/100's of Gbps
> > pushed via single Intel CPU)
> > - that's your NFVI.
> > - could be compute host in a DC "cloud", or in a customer office (acting as
> CPE), or at the rooftop of the office building i.e. (fog/edge computing) -e.g.
> hosting self-driving intersection apps via 5G -to your point regarding latency
> in metro), or in the same rack as core routers (acting as vRR), or actually
> inside a router as a routing engine card (hosting some containerized app).
> 
> Nothing new. This happens today already.
> 
Yes apart from the fog/edge computing all aforementioned is business as usual.

> The main issue, as discussed earlier, was licensing options for CPE-type
> deployments. This is what killed our plan for the same.
> 
Yes vendors need to abandon the old physical unit types of licensing schemes 
for the horizontal scaling to make sense.   

> 
> > 2) Any of the compute hosts mentioned above can host one or more of any
> type of the network function you can think of ranging from EPG, SBC,  PBX, all
> the way to PE-Router, LB, FW/WAF or IDS.
> 
> Yes, but the use-case determines the scale limitations. And there are many
> services that you cannot scale by offloading to several little boxes and 
> expect
> the same predictability, e.g., a vArborTMS.
> 
Can you elaborate? 
Apart from licensing scheme what stops one from redirecting traffic to one vTMS 
instance per say each transit link or per destination /24 (i.e. horizontal 
scaling)? (vTMS is not stateful or is it?)


> > 4) Now how you make changes to control-planes of these NFs (i.e. virtual
> CPU-based NFs and physical NPU-based NFs) programmatically, that's the
> realm of SDN.
> > - If you want to do it right you do it in an abstracted declarative
> > way (not exposing the complexity to a control program/user - but rather
> localizing it to a given abstraction layer) Performing tasks like:
> > - Defining service topology/access control a.k.a. micro segmentation (e.g. A
> and B can both talk to C, but not to each other).
> > - Traffic engineering a.k.a. service chaining, a.k.a. network slicing
> > (e.g. traffic type x should pas through NF A, B and C, but traffic
> > type Y should pass only through A and C)
> 
> This is the bit that I see working well on a per deployment basis, if 
> operators
> aren't too concerned about standardizing the solution.
> 
> Where the industry has kept falling over is wanting to standardize the entire
> orchestration piece, which is very noble, but ultimately, fraught with many a
> complication.
> 
Can you please point out any efforts where operators are trying to standardize 
the orchestration piece? 
I think industry is not falling over on this just progressing at steady rate 
while producing artefacts in the process that you may or may not want to use (I 
actually find them very useful and not impeding).   


> I'm sure we'd all like to see a standard on how we orchestrate the network
> and services, but I'm not sure that is practical. After all, operators are
> autonomous systems.
> 
Personally, I don't need a standard on how I should orchestrate network 
services. There are very interesting and useful ideas, or better put 
"frameworks", that anyone can follow (and most are), but standardizing these, 
...no point in my opinion.


> > 5) And for completeness, in the virtual world you have the task of VNF
> > lifecycle management (cause the VNFs and virtual networks connecting
> > them can be instantiated on demand)
> 
> So I first read all about this in 2015, through a document Cisco published
> called "Cisco vMS 1.0 Introduction and Overview Design Guide".
> 
> Safe to say not much as changed in the objective, since then :-).
> 
Really? Never mind then ;)

adam

RE: Has virtualization become obsolete in 5G?

[adamv0025]

> From: Mark Tinka 
> Cc: adamv0...@netconsultings.com; North American Network Operators'
> 
> 
> On 6/Aug/20 15:43, Shane Ronan wrote:
> 
> > Yes they are for 5G core.
> 
> Right, but for legacy operators, or new entrants?
> 
> If you know where we can find some info about deployment and
> experiences, that would be very interesting to read.
> 
> We've all been struggling to make Intel CPU's shift 10's, 40's and 100's of 
> Gbps
> of revenue traffic as a routing platform, so would like to know how the
> operators are getting on with this.
> 
Mark, 
1) first you have your edge - lots of small instances that are meant to be 
horizontally scaled (not vertically- i.e. not 40's/100's of Gbps pushed via 
single Intel CPU) 
- that's your NFVI.  
- could be compute host in a DC "cloud", or in a customer office (acting as 
CPE), or at the rooftop of the office building i.e. (fog/edge computing) -e.g. 
hosting self-driving intersection apps via 5G -to your point regarding latency 
in metro), or in the same rack as core routers (acting as vRR), or actually 
inside a router as a routing engine card (hosting some containerized app).

2) Any of the compute hosts mentioned above can host one or more of any type of 
the network function you can think of ranging from EPG, SBC,  PBX, all the way 
to PE-Router, LB, FW/WAF or IDS. 
  
3) While inside a compute host it's CPU based forwarding, but as soon as you 
leave compute host's NICs there's world of solely NPU based forwarding (that's 
where you do 40's, 100's, or even 400's Gbps). 
 
4) Now how you make changes to control-planes of these NFs (i.e. virtual 
CPU-based NFs and physical NPU-based NFs) programmatically, that's the realm of 
SDN.
- If you want to do it right you do it in an abstracted declarative way (not 
exposing the complexity to a control program/user - but rather localizing it to 
a given abstraction layer)
Performing tasks like:
- Defining service topology/access control a.k.a. micro segmentation (e.g. A 
and B can both talk to C, but not to each other).
- Traffic engineering a.k.a. service chaining, a.k.a. network slicing (e.g. 
traffic type x should pas through NF A, B and C, but traffic type Y should pass 
only through A and C)
 
5) And for completeness, in the virtual world you have the task of VNF 
lifecycle management (cause the VNFs and virtual networks connecting them can 
be instantiated on demand)
  
adam
 

RE: Has virtualization become obsolete in 5G?

[adamv0025]

And for other stuff as well.

 

adam

 

From: Shane Ronan  
Sent: Thursday, August 6, 2020 2:43 PM
To: Mark Tinka 
Cc: adamv0...@netconsultings.com; North American Network Operators' Group 

Subject: Re: Has virtualization become obsolete in 5G?

 

Yes they are for 5G core.

 

On Wed, Aug 5, 2020, 11:28 AM Mark Tinka mailto:mark.ti...@seacom.com> > wrote:



On 5/Aug/20 17:07, Shane Ronan wrote:

> I think you'd be surprised how much of the 5G Core is containerized
> for both the data and control planes in the next generations providers
> are currently deploying.

It's what I expect for new entrants that don't want to deal with
traditional vendors.

I'd be curious to see if legacy operators are shifting traffic away from
iron to servers, and at what rate.

Mark.

RE: Has virtualization become obsolete in 5G?

[adamv0025]

I was actually talking about routing on the host and virtual control-plane and 
virtualized data-plane.

Currently we either have a VM combining both or a separate VM for each. 
Alternatively we can have a container for the control-plane.

I was wondering if the idea behind containerization is to do virtual data-plane 
as a container as well.

 

In terms of containerization on vendor HW or opening up data-plane, seems like 
XR7 from Cisco is leading the way:

- System runs in containers on RE and Line-cards, allows one to run 3rd party 
containers, 

- Allows one to run 3rd party routing protocols to program RIB

- Allows one to program FIB via Open Forwarding Abstraction (OFA) APIs

- And XR itself can run on selected 3rd party HW. 

 

That pretty much covers all the avenues we as operators are interested in, of 
course it’s not all just roses and unicorns and there will be further 
development and streamlining necessary. 

 

adam

 

From: NANOG  On Behalf Of 
Mark Tinka
Sent: Wednesday, August 5, 2020 1:05 PM
To: nanog@nanog.org
Subject: Re: Has virtualization become obsolete in 5G?

 

 

On 4/Aug/20 17:38, adamv0...@netconsultings.com 
  wrote:

Wondering whether the industry will consider containerised data-plane in 
addition to control-plane (like cRDP). 

Having just control-plane and then hacking to kernel for doing the data-plane 
bit is …well not as straight forward as having a dedicated data-plane VM or 
potentially container. 


Well, there has been some discussion in the past 2 years about whether vendors 
can open up some of their data planes and allow those with enough energy and 
clue (that means not me, hehe) to have their take on what they can do with the 
chips in some kind of form factor, even without their OS.

Outside of that, merchant silicon is the next step, before we try to hack it on 
general-purpose CPU's, as we've been doing for some time.

Mark. 

RE: Has virtualization become obsolete in 5G?

[adamv0025]

> Mark Tinka
> Sent: Tuesday, August 4, 2020 3:54 PM
> 
> On 4/Aug/20 16:46, Djamel Sadok wrote:
> >
> >
> > How about hardware slicing support? such as switch, server and router
> > slicing? is this supported/desirable?
> 
> So you mean dump the VLAN model and give each service its own switch?
> 
> Or do you mean use one server but give each service its own VM? Or worse,
> give each service its own metal server?
> 
> Wouldn't that take us back into the digital stone age :-)?
> 
Yes that's exactly it. 
Instead of a VDOM (or whatever is your FW vendor slicing mechanism) give each 
customer a FW "instance" (VM/Containerized -if there's such a thing already) 
and instantiate it on demand and with resources customer requested and enforce 
utility billing. 
Rinse and repeat for any other NF customer might need on your telco cloud 
(fancy name for a data-canter full of compute)
As simple as that -problem is that all vendors haven't quite gotten up to speed 
with licensing models, we need an overall Gbps throughput pool licenses not per 
VM/Container Gbps pool.   

adam

RE: Has virtualization become obsolete in 5G?

[adamv0025]

Router/switch slicing is supported but not really used much

 

adam

 

From: NANOG  On Behalf Of 
Djamel Sadok
Sent: Tuesday, August 4, 2020 3:47 PM
To: Etienne-Victor Depasquale 
Cc: NANOG 
Subject: Re: Has virtualization become obsolete in 5G?

 

 

 

How about hardware slicing support? such as switch, server and router slicing? 
is this supported/desirable?

 

Djamel

 

 

On Tue, Aug 4, 2020 at 11:37 AM Etienne-Victor Depasquale mailto:ed...@ieee.org> > wrote:

I think that it's validation of QoS that really matters now.

 

If I were to base on this recent video from Keysight 

  (warning: requires registration), 

then it seems that there's a lot of emphasis on making grounded claims about 
the QoS that the operator sells.

 

Cheers,

 

Etienne

 

On Mon, Aug 3, 2020 at 12:52 PM Mark Tinka mailto:mark.ti...@seacom.com> > wrote:

 

On 3/Aug/20 08:40, Etienne-Victor Depasquale wrote:

Is the following extract from this Heavy Reading white paper 

 , useful? 

 

" For transport network slicing, 

operators strongly prefer soft slicing with virtual private networks (VPNs), 

regardless of the VPN flavor.

Ranking at the top of the list was Layer 3 VPNs (selected by 66% of 
respondents), 

but Layer 2 VPNs, Ethernet VPNs (EVPNs), and segment routing 

also ranked highly at 47%, 46%, and 46%, respectively. 

The point is underscored by the low preferences among all of the hard slicing 
technologies— 

those that physically partition resources among slices. 

Hard slicing options formed the bottom tier among preferences."


Well, it's what I've been saying - we have tried & tested systems and solutions 
that are already native to IP/MPLS networks. Why try to reinvent network 
virtualization when there are plenty of existing solutions in the wild for next 
to cheap? VLAN's. l2vpn's. l3vpn's. EVPN. DWDM. And all the rest?

The whole fuss, for example, about the GRX vs. IPX all came down to 2Mbps 
private or public IP-based GTP tunnels vs. 100Mbps l3vpn's.

Mobile operators know how to make everyday protocols seem overly complicated.

If we go by their nomenclature, the simple operators on this list have been 
slicing infrastructure for yonks :-).

Mark.




 

-- 

Ing. Etienne-Victor Depasquale
Assistant Lecturer
Department of Communications & Computer Engineering
Faculty of Information & Communication Technology
University of Malta

Web. https://www.um.edu.mt/profile/etiennedepasquale

RE: Has virtualization become obsolete in 5G?

[adamv0025]

Not sure what you mean NFV is NFV, 

>From NFV perspective cRDP is no different than vMX -it’s just a virtualized 
>router function nothing special…

 

Also with regards to NFV markets, it’s just CPE or telco-cloud (routing on 
host, FWs, LBs and other domain specific network devices like SBCs), and then 
RRs, no one sane would be replacing high throughput aggregation points like PEs 
or core nodes with NFV ,unless one wants to get into some serious horizontal 
scaling ;).

 

adam 

 

From: NANOG  On Behalf Of 
Mark Tinka
Sent: Saturday, August 1, 2020 9:51 PM
To: nanog@nanog.org
Subject: Re: Has virtualization become obsolete in 5G?

 

 

On 1/Aug/20 18:23, Robert Raszuk wrote:

Virtualization is not becoming obsolete ... quite reverse in fact in all types 
of deployments I can see around. 

 

The point is that VM provides hardware virtualization while kubernetes with 
containers virtualize OS apps and services are running on in isolation. 

 

Clearly to virtualize operating systems as long as your level of virtualization 
mainly in terms of security and resource consumption isolation & reservation is 
satisfactory is a much better and lighter option. 


I see cloud-native as NFV++. It requires some adjustment to how classic NFV has 
been deployed, and that comes down to whether operators (especially those who 
err on the side of network operations rather than services) see value in 
upgrading their stack to cloud-native.

If you're a Netflix or an Uber, sure, a cloud-native architecture is probably 
the only way you can scale. But if you are simple network operators who focus 
more on pushing packets than over-the-top services, particularly if you already 
have some NFV, making the move to cloud-native/NFV++ is a whole consideration.

Mark.

RE: Has virtualization become obsolete in 5G?

[adamv0025]

Wondering whether the industry will consider containerised data-plane in 
addition to control-plane (like cRDP). 

Having just control-plane and then hacking to kernel for doing the data-plane 
bit is …well not as straight forward as having a dedicated data-plane VM or 
potentially container. 

 

adam

 

From: NANOG  On Behalf Of 
Etienne-Victor Depasquale
Sent: Saturday, August 1, 2020 7:09 PM
To: Robert Raszuk 
Cc: NANOG 
Subject: Re: Has virtualization become obsolete in 5G?

 

Clearly to virtualize operating systems as long as your level of virtualization 
mainly in terms of security and resource consumption isolation & reservation is 
satisfactory is a much better and lighter option.  

 

That pretty much sums up Intel's view.

 

To quote an Intel executive I was corresponding with:

 

"The purpose of the paper was to showcase how Communication Service Providers 
can move to a more nimble and future proof microservices based network 
architecture with cloud native functions, via container deployment 
methodologies versus virtual machines.  The paper cites many benefits of moving 
to a microservices architecture beyond whether it is done in a VM environment 
or cloud native. We believe the 5G networks of the future will benefit greatly 
by implementing such an approach to deploying new services."

 

The paper referred to is this one 

 .

 

Cheers,

 

Etienne

 

On Sat, Aug 1, 2020 at 6:23 PM Robert Raszuk mailto:rob...@raszuk.net> > wrote:

I reason that Intel's implication is that virtualization is becoming obsolete.

Would anyone care to let me know his thoughts on this prediction?

 

Virtualization is not becoming obsolete ... quite reverse in fact in all types 
of deployments I can see around. 

 

The point is that VM provides hardware virtualization while kubernetes with 
containers virtualize OS apps and services are running on in isolation. 

 

Clearly to virtualize operating systems as long as your level of virtualization 
mainly in terms of security and resource consumption isolation & reservation is 
satisfactory is a much better and lighter option. 

 

Thx,

R.

 




 

-- 

Ing. Etienne-Victor Depasquale
Assistant Lecturer
Department of Communications & Computer Engineering
Faculty of Information & Communication Technology
University of Malta

Web. https://www.um.edu.mt/profile/etiennedepasquale

RE: BGP route hijack by AS10990

[adamv0025]

> Darrell Budic
> Sent: Sunday, August 2, 2020 6:23 PM
> 
> On Jul 30, 2020, at 5:37 PM, Baldur Norddahl 
> wrote:
> >
> > Telia implements RPKI filtering so the question is did it work? Were any
> affected prefixes RPKI signed? Would any prefixes have avoided being
> hijacked if RPKI signing had been in place?
> >
> > Regards
> >
> > Baldur - who had to turn off RPKI filtering at the request of JTAC to stop 
> > our
> mx204s from crashing :-(
> >
> 
> Oh uh, I’m getting close to getting RPKI going on my mx204s, or was until you
> posted that. What’s the story there, and perhaps which junos version?

Same here, would be interested in affected Junos versions or any details you 
can share please,

adam

RE: cloud backup

[adamv0025]

> Randy Bush
> Sent: Sunday, July 26, 2020 9:10 PM
> 
> i backup using arq on macos catalina.  on two macs, i need maybe 3-4tb
max.
> google seems to be $100/mo for 20tb (big jump from $100/yr for 2tb).
> backblaze b2 looks more like $20/mo for 4tb ($0.005/gb/mo).
> anyone else done a similar analysis?
> 
Just out of curiosity, do you folks encrypt the data prior to upload to the
cloud (especially when uploading to folks like aws/google/ms/(facebook))? 
Was wondering if these folks provide encryption of stored data - quick
google search suggests that google does not. 


adam

RE: questions asked during network engineer interview

[adamv0025]

> Mark Tinka
> Sent: Thursday, July 23, 2020 5:04 AM
> 
> On 23/Jul/20 01:04, Brandon Martin wrote:
> 
> >
> > Of course, there's also plenty of folks out there without them or any
> > certs at all that are just as useful in practice.  Getting those
> > particular certifications does, however, seem to be a useful path to
> > learning things that are actually of use in the "real world".  I look
> > at such certificates similar to how I'd look at a 2- or 4-year degree
> > in a related IT field and just a somewhat different, and perhaps more
> > approachable for the self-coached or differently-learning, path.
> 
> We live in a time where I am concerned about the engineers we
> are creating, where point & click seems to trump basic understanding + CLI
> knowledge. My concern is when it all goes to hell at 3AM, do the next
> generation of network engineers have the base fundamentals to understand
> why iBGP isn't coming up, even though you can "ping" and IGP adjacencies
> are up and stable?
> 
Hopefully well end up in a world where all checks one can do to figure out why 
iBGP session is down along with suggested corrective actions will be coded in 
some network self-healing workflow. 
But to answer your question, probably no, cause current industry is 
systematically converting network engineers into coders. 

adam
   

RE: questions asked during network engineer interview

[adamv0025]

> Jeff Bacon
> Sent: Wednesday, July 22, 2020 1:55 PM
> 
>  > Who said anything about boxing your tooling in to SDN tech? You  >
> described Software Defined Networking as a rabbit hole and snake oil.
>  > It isn't. It's a class of tools in the networking toolbox and an  > 
> increasingly
> useful one.
> 
> As it is, we have marketing people sticking "SDN" onto every bloody thing
> that comes along in the hopes that it'll better catch the attention of someone
> without enough of a clue that they'll cough up (and leave someone else to
> figure out what to do with it). I freely admit that's just what they do and
> expecting them to do anything different is wishful thinking, but I don't see
> any of us doing anything about it except creating long email chains wherein
> we just keep trying to munge apples and oranges together. :)
> 
I suggest we educate users so they can make better decisions on what tool to 
use for what job and not get confused in all the marketing jive around SDN.
Blueprint: MEF-SDN/NFV Certification Exam:
https://wiki.mef.net/pages/viewpage.action?pageId=75990347
- has a pretty good list of self-study material, don't worry you might have 
read a lot of those books already.

adam

RE: questions asked during network engineer interview

[adamv0025]

> William Herrin
> Sent: Tuesday, July 21, 2020 8:21 PM
> 
> On Mon, Jul 20, 2020 at 9:57 PM Mark Tinka 
> wrote:
> > Suffice it to say, to this day, we still don't know what SDN means to
> > us, hehe.
> 
> Hi Mark,
> 
> The Software Defined Network concept started as, "Let's use commodity
> hardware running commodity operating systems to form the control plane
> for our network devices." The concept has expanded somewhat to: "Lets use
> commodity hardware running commodity operating systems AS our network
> devices." For example, if you build a high-rate firewall with DPDK on Linux,
> that's now considered SDN since its commodity hardware, commodity OS
> and custom packet handling (DPDK) that skips the OS.
> 
The higher level takeaway would be: 
"Modularity based on abstraction is the way things get done”
− Barbara Liskov

Abstractions -> Interfaces -> Modularity
This applies at the individual device level as well as you described above, 
however I'd say that application of these principles at the network-wide level 
is also very beneficial to service providers, (Device -> Network -> Service -> 
Product -> Offer).
This vision was first realized by AT in their ECOMP framework which (along 
with Open-O) is now morphed to ONAP. This idea has now been adopted by many 
service providers as well as commercial products.  
  
adam

RE: questions asked during network engineer interview

[adamv0025]

> From: Mel Beckman 
> Sent: Tuesday, July 21, 2020 9:48 PM
> 
> The problem is your door is stuck in 2014 :)
> 
> A lot has happened in the last six years.
> 
Yeah but if you won't get the basics (or fundamental problems in networking as 
a discipline that SDN is trying to solve) as nicely illustrated by Scott 
Shenker in that preso, you won't understand what modern SDN forks are about.

adam


  

RE: questions asked during network engineer interview

[adamv0025]

> From: Mark Tinka 
> Sent: Tuesday, July 21, 2020 6:14 PM
> 
> On 21/Jul/20 18:39, adamv0...@netconsultings.com wrote:
> > Little you two know about SDN, please read the following presentation
> from Scott Shenker and then get back here arguing what it is and what it is
> not: https://inst.eecs.berkeley.edu/~cs168/fa14/lectures/lec23-public.pdf
> 
> I'll pass, thanks. Already did my time in that rabbit hole.
> 
Well  "I can only show you the door. You're the one that has to walk through 
it."  ;)

adam

RE: questions asked during network engineer interview

[adamv0025]

> William Herrin
> Sent: Monday, July 20, 2020 9:02 PM
>
> On Mon, Jul 20, 2020 at 5:09 AM Mark Tinka 
> wrote:
> > We'll probably spend 95% of the time just talking about who they are,
> > and 5% on the role. That has worked well for me in the past decade,
> > and none of those hires had any "certificates" to impress me with,
> > even though those that didn't make it, did.
> 
> I find there's a strong INVERSE correlation between the quantity of
> certificates on an applicant's resume and their ability to do the job.
>
One can't be an expert in many areas, (like CCIE-everything... folks)
Same as Usain Bolt can't swim like Michael Phelps...
 

> I still have to laugh about the guy who let me know via his resume that he
> was certified in setting up Kentrox CSU/DSUs.
> 
Nice, did he also knew how to put a loop on a T1/T3 DACS (DCX) or how to do SNA 
to mainframe and then DLSW? :) very useful skills these days indeed. 
technology used to be much simpler. 


adam

RE: questions asked during network engineer interview

[adamv0025]

Little you two know about SDN, please read the following presentation from 
Scott Shenker and then get back here arguing what it is and what it is not: 
https://inst.eecs.berkeley.edu/~cs168/fa14/lectures/lec23-public.pdf

adam

RE: BFD for long haul circuit

[adamv0025]

Well luckily we have MEF to set expectations about ones EPL/EVPL/EPLAN/EVPLAN 
performance. (and formal SLA contracts describing every single aspect of the 
service and its performance).

Anyways, when I was designing these the back in the days when it was cool and 
demand was high, customers (other carriers) were getting MTU9100 (to fit 
customers MTU9000), the whole CFM & LFM shebang (to the point made earlier in 
the thread that the link should go down on both ends -like it’s the case with a 
wave) and sub 50ms convergence in case something when wrong inside our 
backbone. 

 

We as a provider got more $$$ from single investment to our wave/fiber, but our 
customers could enjoy p2p links on par with wave for less $.

 

adam

 

From: NANOG  On Behalf Of 
Robert Raszuk
Sent: Friday, July 17, 2020 10:50 AM
To: Mark Tinka 
Cc: nanog@nanog.org
Subject: Re: BFD for long haul circuit

 

>  Unfortunately not. 

 

Fortunately  very fortunately Mark. 

 

L2VPNs running on someone's IP backbone sold by many as "circuits" has many 
issues ... stability, MTU blackhols, random drops - and that is pretty much the 
same all over the world :( 

 

Very unfortunate technology just to mux more users and get more $$$ from single 
investment.

 

Cheers,

R.

 

On Fri, Jul 17, 2020 at 8:43 AM Mark Tinka mailto:mark.ti...@seacom.com> > wrote:

 

On 17/Jul/20 02:37, Harivishnu Abhilash wrote:

  

Thanks for the update. You have any backhauls, that is running over an L2 
xconnect  ? I’m facing issue only on the backhaul link over a l2vpn ckt.  


Unfortunately not. All our backbones are either over dark fibre or EoDWDM.

Mark.

RE: questions asked during network engineer interview

[adamv0025]

> William Herrin
> Sent: Tuesday, July 14, 2020 8:32 PM
> 
> On Tue, Jul 14, 2020 at 12:17 PM Michael Thomas  wrote:
> > On 7/14/20 12:09 PM, William Herrin wrote:
> > > On Mon, Jul 13, 2020 at 3:12 PM Mehmet Akcin 
> wrote:
> > >> I am hosting a live show a few times a month about internet
> > >> infrastructure and today's topics were, your favorite questions
> > >> asked network engineers - you can watch the recording here
> > >>
> > >> https://www.youtube.com/watch?v=o3pvikTrF0M
> > >>
> > >> if you have suggestions on topics to cover helping network operations
> engineering that you want to see in here, please feel free to contact me off-
> list, and let's create unique content that can be helpful to others.
> > >
> > > "What happens when you type www.google.com in your browser bar
> and
> > > hit enter?" is one of my favorite questions. Half the field of
> > > computing happens next. Keyboard interrupts fire. Bits are poked in
> > > dram, sram, maybe even tcam. Packets are sent. Fonts are composed
> into pixels.
> > > There's a crazy amount you can talk about and the right answer is:
> > > string things together in order for 5 or 10 minutes without getting
> > > anything horribly wrong.
> >
> > Oh, I thought this was a trick question of whether it takes you
> > directly to google, or does a search.
> 
> That's a good start. First thing the browser does decide whether that's a URL
> or a search question. How does it decide? And then what happens?
> 
> I will prompt you to keep talking. After all, I'm rooting for you to succeed 
> so
> that I can hire you.
> 
The question is vague enough for the candidate to start talking just about 
anything they like. 
What happens where? In the world? In the universe? In my body?
Depending on the position you're hiring for you may want to include the "where" 
as well to narrow down the scope of the talk (to say "finger tips" if you hire 
a brain surgeon or 2020 laureate of Kavli Prize for Neuroscience for discover 
of pressure receptors, or simply to "network" if hiring network engineer, 
etc...). 
 
adam
 

RE: questions asked during network engineer interview

[adamv0025]

This is exactly why the espresso network looks like it does… 

Have you seen the advert for network architect position at google? Bunch of 
programming languages and that’s enough apparently.

adam

 

From: NANOG  On Behalf Of 
Ahmed elBorno
Sent: Tuesday, July 14, 2020 6:57 PM
To: Michael Thomas 
Cc: NANOG list 
Subject: Re: questions asked during network engineer interview

 

15 years ago, I applied to a network admin role at Google, it was for their 
corporate office, not even the production network.

 

I had less than two years experience.

 

The interviewer asked me:

 

1) What is the difference between flow balancing techniques on Cisco IOS and 
Linux?

2) If we had a 1GB file that we need to transfer between America and Europe, 
how much time do we need, knowing that we start with a TCP size of X?

 

I'll never forget this :)

 

On Tue, Jul 14, 2020 at 10:50 AM Michael Thomas mailto:m...@mtcc.com> > wrote:

 

On 7/14/20 10:33 AM, Owen DeLong wrote:

On Jul 14, 2020, at 10:20 , Michael Thomas mailto:m...@mtcc.com> > wrote:





I once failed a network engineering interview because I couldn’t recite the 
OSPF LSA types by number from memory. It was fine, the fact that was a key 
question in the interview convinced me that I had no more desire to work there 
than they had to hire me.

I once got rejected because I spaced out and didn't remember the java keyword 
"final" as a constant. you can't make this stuff up.

 

My personal method is to devise a problem and actually work with them... 
because that's what I (or others) are going to be doing. How well can they get 
the requirements? How do they zero in on how to solve it? You can take this as 
deep or shallow as you like. Often I'd give it as a homework assignment if I 
liked them.

I prefer this approach as well. Depending on the level of interviewee, I like 
to pull up a real world scenario from my past and see how they approach it. I’m 
not nearly as concerned if they get to the right solution as I want to see how 
they go about identifying and solving the problem. Do they ask questions that 
narrow their focus and identify the issue, or do they start trying random 
things hoping to stumble across a solution without understanding the problem?

I often use something that was somewhat topical to me but dumbed down enough to 
fit in an interview and possible homework assignment. My reasoning is that I'm 
not entirely sure what the solution ought to look like either, so I'm 
interested to see what their take is. I can also give them real time feedback 
just like I would if they were a co-worker. At base, an interview should answer 
the question: "can I work with this person?". 




Not being a developer (at least not for the last 25+ years), I haven’t done 
many “software” interviews, but I’ve been through network and sysadmin 
interviews that ran the gamut. Frankly, the ones that seemed to be more about 
fluffing privates were the companies I put less focus on going forward. The 
interviewers that seemed to match my style and wanted to see me do real-world 
things like troubleshooting or solving design problems or identifying 
architectural flaws in a proposed solution usually resulted in mutual respect 
and I usually moved forward through the interview processes. On the few 
occasions where I got a job out of a fluffing interview, it almost universally 
turned out to be one I wished I hadn’t taken.

I had a screening interview at Google where the screener asked some ridiculous 
question that nobody not straight out of school would know, and even then not 
likely. I was like, wtf? If that's how they treat candidates -- and from 
everything I've heard it is -- I want nothing to do with them, and flat out 
refused their recruiters a dozen time afterward even though they pleaded that 
they've changed. Sorry, interviewing is a two-way street.

Mike

YANG to Web-form/Web-UI

[adamv0025]

Hi folks,
Anyone knows of any tool that would take YANG module and use it to
automatically generate Web-form (Web-UI), other than DLUX for ODL (which
sadly has been deprecated). 
Thanks 

adam 

RE: 60 ms cross-continent

[adamv0025]

> Stephen Satchell via NANOG
> Sent: Monday, June 22, 2020 8:37 PM
> 
> On 6/22/20 12:59 AM, adamv0...@netconsultings.com wrote:
> >> William Herrin
> >>
> >> Howdy,
> >>
> >> Why is latency between the east and west coasts so bad? Speed of
> >> light accounts for about 15ms each direction for a 30ms round trip.
> >> Where does the other 30ms come from and why haven't we gotten rid of
> it?
> >>
> > Wallstreet did :)
> > https://www.wired.com/2012/08/ff_wallstreet_trading/
> 
> “Of course, you’d need a particle accelerator to make it work.”
> 
> So THAT'S why CERN wants to build an even bigger accelerator than the LHC!
>
Yep, why to go around the planet chasing a perfect geodesic with as few relay 
towers or drones if you can go through (shortest distance is always a straight 
line as opposed to an arc). 
While maintaining the speed of light in vacuum since neutrinos don't seem 
interact with regular matter, that's why they are so darn hard to detect. 
All you need is an extremely powerful neutrino detector to get you above the 
51:49 success ratio. (49% packet loss is not what we're accustomed to, but for 
these guys it's low enough to start making money). 
It's quite a fascinating networking world these guys live in, working for a HFT 
company would be my dream job, always pushing the envelope, racing to the 
bottom, it's like F1 of the networking world just without the safety and 
fairness BS to slow you down.

adam

RE: why am i in this handbasket? (was Devil's Advocate - Segment Routing, Why?)

[adamv0025]

> Masataka Ohta
> Sent: Monday, June 22, 2020 1:49 PM
> 
> Robert Raszuk wrote:
> 
> > Moreover if you have 1000 PEs and those three sites are attached only
> > to 6 of them - only those 6 PEs will need to learn those routes (Hint:
> > RTC -
> > RFC4684)
> 
> If you have 1000 PEs, you should be serving for somewhere around 1000
> customers.
> 
> And, if I understand BGP-MP correctly, all the routing information of all
the
> customers is flooded by BGP-MP in the ISP.
> 
Not quite,
The routing information is flooded by default, but the receivers will cherry
pick what they need and drop the rest. 
And even if the default flooding of all and dropping most is a concern -it
can be addressed where only the relevant subset of all the routing info is
sent to each receiver.
The key takeaway however is that no single entity in SP network, be it PE,
or RR, or ASBR, ever needs everything, you can always slice and dice
indefinitely.
So to sum it up you simply can not run into any scaling ceiling with MP-BGP
architecture.   

adam

RE: why am i in this handbasket? (was Devil's Advocate - Segment Routing, Why?)

[adamv0025]

> From: Masataka Ohta 
> Sent: Monday, June 22, 2020 2:17 PM
> 
> adamv0...@netconsultings.com wrote:
> 
> > But MPLS can be made flow driven (it can be made whatever the policy
> > dictates), for instance DSCP driven.
> 
> The point of Yakov on day one was that, flow driven approach of Ipsilon
does
> not scale and is unacceptable.
> 
> Though I agree with Yakov here, we must also eliminate all the flow driven
> approaches by MPLS or whatever.
> 
First I'd need a definition of what flow means in this discussion are we
considering 5-tuple or 4-tuple or just SRC-IP & DST-IP, is DSCP marking part
of it? 
Second, although I agree that ~1M unique identifiers is not ideal, can you
provide examples of MPLS applications where 1M is limiting?
What particular aspect?
Is it 1M interfaces per MPLS switching fabric box?
Or 1M unique flows (or better flow groups) originated by a given
VM/Container/CPE?
Or 1M destination entities (IPs or apps on those IPs) that any particular
VM/Container/CPE needs to talk to?
Or 1M customer VPNs or 1M PE-CPE links, if PE acts as a bottleneck?

adam
  

RE: Devil's Advocate - Segment Routing, Why?

[adamv0025]

Hi Baldur,

 

>From memory mx204 FIB is 10M (v4/v6) and RIB 30M for each v4 and v6.

And remember the FIB is hierarchical -so it’s the next-hops per prefix you are 
referring to with BGP FRR. And also going from memory of past scaling testing, 
if pfx1+NH1 == x, then  Pfx1+NH1+NH2 !== 2x, where x is used FIB space. 

  

adam

 

From: NANOG  On Behalf Of 
Baldur Norddahl
Sent: Saturday, June 20, 2020 9:00 PM



I can't speak for the year 2000 as I was not doing networking at this level at 
that time. But when I check the specs for the base mx204 it says something like 
32 VRFs, 2 million routes in FIB and 6 million routes in RIB. Clearly those 
numbers are the total of routes across all VRFs otherwise you arrive at silly 
numbers (64 million FIB if you multiply, 128k FIB if you divide by 32). My 
conclusion is that scale wise you are ok as long you do not try to have more 
than one VRF with a complete copy of the DFZ.

 

More worrying is that 2 million routes will soon not be enough to install all 
routes with a backup route, invalidating BGP FRR.

 

RE: 60 ms cross-continent

[adamv0025]

> William Herrin
> 
> Howdy,
> 
> Why is latency between the east and west coasts so bad? Speed of light
> accounts for about 15ms each direction for a 30ms round trip. Where does
> the other 30ms come from and why haven't we gotten rid of it?
> 
Wallstreet did :) 
https://www.wired.com/2012/08/ff_wallstreet_trading/

adam

RE: Devil's Advocate - Segment Routing, Why?

[adamv0025]

> From: NANOG  On Behalf Of Masataka Ohta
> Sent: Friday, June 19, 2020 5:01 PM
> 
> Robert Raszuk wrote:
> 
> > So I think Ohta-san's point is about scalability services not flat
> > underlay RIB and FIB sizes. Many years ago we had requests to support
> > 5M L3VPN routes while underlay was just 500K IPv4.
> 
> That is certainly a problem. However, worse problem is to know label
values
> nested deeply in MPLS label chain.
> 
> Even worse, if route near the destination expected to pop the label chain
> goes down, how can the source knows that the router goes down and
> choose alternative router near the destination?
> 
Via IGP or controller, but for sub 50ms convergence there are edge node
protection mechanisms, so the point is the source doesn't even need to know
about for the restoration to happen. 

adam
 

RE: why am i in this handbasket? (was Devil's Advocate - Segment Routing, Why?)

[adamv0025]

> Masataka Ohta
> Sent: Sunday, June 21, 2020 1:37 PM
> 
> > Whether you do it manually or use a label distribution protocol, FEC's
> > are pre-computed ahead of time.
> >
> > What am I missing?
> 
> If all the link-wise (or, worse, host-wise) information of possible
destinations
> is distributed in advance to all the possible sources, it is not
hierarchical but
> flat (host) routing, which scales poorly.
> 
> Right?
> 
On the Internet yes in controlled environments no, as in these environments
the set of possible destinations is well scoped. 

Take an MPLS enabled DC for instance, every VM does need to talk to only a
small subset of all the VMs hosted in a DC. 
Hence each VM gets flow transport labels programmed via centralized
end-to-end flow controllers on a need to know bases (not everything to
everyone).
(E.g. dear vm1 this is how you get your EF/BE flows via load-balancer and FW
to backend VMs in your local pod, this is how you get via local pod fw to
internet gw, etc..., done)
Now that you have these neat "pipes" all over the place connecting VMs it's
easy for the switching fabric controller to shuffle elephant and mice flows
around in order to avoid any link saturation. 

And now imagine a bit further doing the same as above but with CPEs on a
Service Provider network... yep, no PEs acting as chokepoints for MPLS label
switch paths to flow assignment, needing massive FIBs and even bigger, just
dumb MPLS switch fabric, all the "hard-work" is offloaded to centralized
controllers (and CPEs for label stack imposition) -but only on a need to
know bases (not everything to everyone).

Now in both cases you're free to choose to what extent should the MPLS
switch fabric be involved with the end-to-end flows by imposing hierarchies
to the MPLS stack.  

In light of the above, does it suck to have just 20bits of MPLS label space?
Absolutely.
 
Adam

RE: why am i in this handbasket? (was Devil's Advocate - Segment Routing, Why?)

[adamv0025]

But MPLS can be made flow driven (it can be made whatever the policy dictates), 
for instance DSCP driven…

 

adam

 

From: NANOG  On Behalf Of 
Robert Raszuk
Sent: Saturday, June 20, 2020 4:13 PM
To: Masataka Ohta 
Cc: North American Network Operators' Group 
Subject: Re: why am i in this handbasket? (was Devil's Advocate - Segment 
Routing, Why?)

 

 

The problem of MPLS, however, is that, it must also be flow driven,
because detailed route information at the destination is necessary
to prepare nested labels at the source, which costs a lot and should
be attempted only for detected flows.

 

MPLS is not flow driven. I sent some mail about it but perhaps it bounced. 

 

MPLS LDP or L3VPNs was NEVER flow driven. 

 

Since day one till today it was and still is purely destination based. 

 

Transport is using LSP to egress PE (dst IP). 

 

L3VPNs are using either per dst prefix, or per CE or per VRF labels. No 
implementation does anything upon "flow detection" - to prepare any nested 
labels. Even in FIBs all information is preprogrammed in hierarchical fashion 
well before any flow packet arrives. 

 

Thx,
R.

 

 

 


 > there is the argument that switching MPLS is faster than IP; when the
 > pressure points i see are more at routing (BGP/LDP/RSVP/whatever),
 > recovery, and convergence.

Routing table at IPv4 backbone today needs at most 16M entries to be
looked up by simple SRAM, which is as fast as MPLS look up, which is
one of a reason why we should obsolete IPv6.

Though resource reserved flows need their own routing table entries,
they should be charged proportional to duration of the reservation,
which can scale to afford the cost to have the entries.

Masataka Ohta

RE: Devil's Advocate - Segment Routing, Why?

[adamv0025]

> From: NANOG  On Behalf Of Mark Tinka
> Sent: Friday, June 19, 2020 7:28 PM
> 
> 
> On 19/Jun/20 17:13, Robert Raszuk wrote:
> 
> >
> > So I think Ohta-san's point is about scalability services not flat
> > underlay RIB and FIB sizes. Many years ago we had requests to support
> > 5M L3VPN routes while underlay was just 500K IPv4.
> 
> Ah, if the context, then, was l3vpn scaling, yes, that is a known issue.
> 
I wouldn't say it's known to many as not many folks are actually limited by 
only up to ~1M customer connections, or next level up, only up to ~1M customer 
VPNs.   

> Apart from the global table vs. VRF parity concerns I've always had (one of
> which was illustrated earlier this week, on this list, with RPKI in a VRF),
>
Well yeah, things work differently in VRFs, not a big surprise.  
And what about an example of bad flowspec routes/filters cutting the boxes off 
net -where having those flowspec routes/filters contained within an Internet 
VRF would not have such an effect.
See, it goes either way.  
Would be interesting to see a comparison of good vs bad for the Internet routes 
in VRF vs in Internet routes in global/default routing table.


> the
> other reason I don't do Internet in a VRF is because it was always a 
> trade-off:
> 
> - More routes per VRF = fewer VRF's.
> - More VRF's  = fewer routes per VRF.
> 
No, that's just a result of having a finite FIB/RIB size -if you want to cut 
these resources into virtual pieces you'll naturally get your equations above.
But if you actually construct your testing to showcase the delta between how 
much FIB/RIB space is taken by x prefixes with each in a VRF as opposed to all 
in a single default VRF (global routing table) the delta is negligible. 
(Yes negligible even in case of per prefix VPN label allocation method -which 
I'm assuming no one is using anyways as it inherently doesn't scale and would 
limit you to ~1M VPN prefixes though per-CE/per-next-hop VPN label allocation 
method gives one the same functionality as per-prefix one while pushing the 
limit to ~1M PE-CE links/IFLs which from my experience is sufficient for most 
folks out there). 
  
adam

RE: Devil's Advocate - Segment Routing, Why?

[adamv0025]

> Saku Ytti
> Sent: Friday, June 19, 2020 8:50 AM
> 
> On Fri, 19 Jun 2020 at 10:24, Radu-Adrian Feurdean  adrian.feurdean.net> wrote:
> 
> 
> > > I don't understand the point of SRv6. What equipment can support
> > > IPv6 routing, but can't support MPLS label switching?
> >
> Maybe these inferior technologies will win, due to the marketing strength of
> DC solutions. Or maybe DC will later figure out the fundamental aspect in
> tunneling cost, and invent even-better-MPLS, which is entirely possible now
> that we have a bit more understanding how we use MPLS.
>
Looking back at history (VXLAN or the Google's Espresso "architecture") I'm not 
holding my breath for anything reasonable coming out of the DC camp

adam
 

RE: Devil's Advocate - Segment Routing, Why?

[adamv0025]

> From: Mark Tinka 
> Sent: Thursday, June 18, 2020 12:51 PM
> 
> On 18/Jun/20 13:23, adamv0...@netconsultings.com wrote:
> 
> >  is the current state is not the end state, this is a pretty dynamic 
> > industry
> that I'm sure is converging/evolving towards a v4:v6 parity, however the pace
> may be, which is understandable considering the scope of ground to be
> covered.
> 
> Which I am fine with  - if you give me a time line to say LDPv6,
> SR-OSPFv3 and SR-ISISv6 will be available on X date, I can manage my
> operation and expectations accordingly.
> 
> But if you say, "No LDPv6, no SR-OSPFv3, no SR-ISISv6... only SRv6", then
> that's an entirely different issue.
> 
> The good news is there currently is choice on the matter, but upending
> hundreds or thousands of boxes to prove that point should really be a last
> resort, as there are more pressing things we all have to deal with.
> 
Hence our current strategy is to stay on IPv4 control-plane (and IPv4 
management plane) as it suits, and for the foreseeable future will suite, all 
our needs (which are to transport v4 data packets via L2 MPLS VPN 
services), there are simply more important projects than to experiment with v6 
control-plane, like for instance perfecting/securing the v6 customer facing 
services (delivered over the underlying v4 signalled MPLS infrastructure, that 
no customer really cares about). 

But I understand your frustrations case it seems like you're taking the bullet 
for us late adopters and in a sense you are, cause say in 10 years from now 
when I decide to migrate to v6 control-plane and management-plane as then it 
might be viewed as common courtesy, it will be all there on a silver plate 
waiting for me allowing for a relatively effortless and painless move. All 
thanks to you fighting the good fight today.

> 
> >  Yes you're right in acknowledging that we're not living in a perfect world
> and that choices are limited, but it's been like that since ever yet we
> managed to thrive by analysing our options and striving for optimal strategies
> year by year.
> 
> We can thank NAT44, CIDR, DHCP and PPPoE for that strategy over the years
> :-).
> 
> IPv6 is the future, and at some point, we'll have to stop hiding from it.
> 
And I'd say the future is now, cause there is an actual need for v6 services. 
But need for v6 control & management plane? - It's not like operators are 
losing business opportunities not having that. (they might even be viewed as 
conservative->stable, which might be preferred by some customers). 

adam 

RE: Devil's Advocate - Segment Routing, Why?

[adamv0025]

> From: NANOG  On Behalf Of Mark Tinka
> Sent: Thursday, June 18, 2020 8:13 AM
> 
> There are probably as many networks running SR-MPLS as there are running
> LDPv6, likely fewer if your SR deployment doesn't yet support OSPFv3 or SR-
> ISISv6. I concede that for some networks looking to go SR-MPLS, label
> distribution state reduction is probably higher up on the agenda than
> MPLSv6 forwarding. For me, I'd like the option to have both, and decide
> whether my network is in a position to handle the additional state required
> for LDPv6, if I feel that I'd prefer to deal with a protocol that has had more
> exposure to the sun.
> 
You do have the LDP vs SR choice (in v4 anyways) yes there's not a good 1:1 
feature parity with v6, but the important point is the current state is not the 
end state, this is a pretty dynamic industry that I'm sure is 
converging/evolving towards a v4:v6 parity, however the pace may be, which is 
understandable considering the scope of ground to be covered. Yes you're right 
in acknowledging that we're not living in a perfect world and that choices are 
limited, but it's been like that since ever yet we managed to thrive by 
analysing our options and striving for optimal strategies year by year.  

adam

RE: Devil's Advocate - Segment Routing, Why?

[adamv0025]

> From: Saku Ytti
> Sent: Thursday, June 18, 2020 6:26 AM
> 
> On Thu, 18 Jun 2020 at 01:17, Mark Tinka  wrote:
> 
> > Yes, we all love less state, I won't argue that. But it's the same question
> that is being asked less and less with each passing year - what scales better 
> in
> 2020, OSPF or IS-IS. That is becoming less relevant as control planes keep
> getting faster and cheaper.
> 
> I don't think it ever was relevant.
> 
In 99% of cases, there are cases however where supporting 1M+ routes in IGP is 
one of the viable options to consider, or running multi-100k of LSPs through a 
core node... 
But these are core MPLS networks that have no boundaries cause these literally 
wrap around the globe, and access to custom code. 
 
adam 

RE: Devil's Advocate - Segment Routing, Why?

[adamv0025]

> From: NANOG  On Behalf Of Mark Tinka
> Sent: Wednesday, June 17, 2020 6:07 PM
> 
> 
> I've heard a lot about "network programmability", e.t.c., 
First of all the "SR = network programmability" is BS, SR = MPLS, any 
programmability we've had for MPLS since ever works the same way for SR. 

> but can anyone
> point me toward a solution that actually does this in the way that it has been
> touted for years? A true flow that shows the implementation of "network
> programming" over any incarnation of SR? Perhaps one a customer can go to
> the shop and grab off the shelf?
> 
Yes anything that works for RSVP-TE (i.e. PCEP), if you want to play there's 
this free app on top of ODL(acting as PCEP+BGP-LS) to program LSPs (can't 
recall the name).
  

> I've heard about "end-to-end service chaining" as a use-case for SR. 
"service chaining" = traffic-engineering, you can do that with or without SR 
just fine.

> To
> service-chain what? 
To service-chain DC or as hipsters call it "cloud" stuff. To TE path from VM to 
FW to ...whatever, or to TE mice flows around elephant flows. 

> Classic telco's don't offer complex over-the-top services
They do via telco cloud. 

> What problems are 90% of the
> operators running MPLS having that SR will truly fix,
> 
None,  
The same point I was trying to get across in our LDPv6 (or any v6 in 
control-plane or management plane for that matter) discussion, there's no 
problem to solve. 
Personally I'll be doing SR only in brand new greenfield deployments or if I 
start running out of RSVP-TE scale on existing deployments.   


adam

favourite YANG data-store?

[adamv0025]

Hi folks,
Was just wondering what are you folks using as production YANG data store
and what do you like about the particular one you're using? Or maybe folks
using OANP what is your YANG DS of choice? 
Plan on using it as in memory DS primarily for service, network YANG
modules, (in addition to the usual use case of device modules), 
- so quite a lot of data as you can imagine storing data for higher
abstraction layers Service & Network. 
Been looking at ODL and Confd thus far. 

adam

RE: BGP FLowspec to Yang/Yaml ACL

[adamv0025]

In order to use YANG you need a device that can speak NETCONF/RESTCONF and 
understands YANG.

There’s no such thing as “The YANG ACL” -there’s IETF YANG model for ACLs, 
there’s OpenConfig one, and your switch vendor might have another YANG model 
for representing ACLs. 

Whichever model provides sufficient coverage for your use case (i.e. can use 
the model to specify SRC/DST/MASK/DENY/ACCEPT) and is supported natively by 
your device (can send the ACL config in this format to the device at it knows 
what to do) is the right for you.   

 

If your devices do not support NETCONF/RESTCONF nor understand YANG you can 
still push the ACL changes via CLI scraping (Ansible)

 

Now in either case (netconf-yang/ansible), what you’re better off with is a 
tool that allows operator to enter the details of the ACL line to be added 
(details of the flow) and just take that input and insert it into the 
pre-defined/prepared template (yang/ansible template), then the script just 
prompts the resulting config to be pushed onto the device (devices).

 

 

adam

 

From: NANOG  On Behalf Of Douglas Fischer
Sent: Tuesday, June 16, 2020 7:40 PM
To: nanog@nanog.org
Subject: BGP FLowspec to Yang/Yaml ACL

 

We were looking for some way to implement BGP Flowspec Filtering(just the 
permit/deny basic) using L3 switches  in an automated way.

Searching a bit we found   
https://github.com/ios-xr/bgpfs2acl

 

Is almost what we are looking for!
But is focused on Cisco devices.

We even considered fork it to our specific vendor.
But before reinventing the wheel, I decide to ask to colleagues if anybody 
knows some tool that converts BGP Flowspec ACLs into YAML or even to YANG.

 

If that exists, with Ansible/Netconf/RestConf(or some similar tool), it would 
be easy to delegate to Switchs doing the basic filtering that only More 
expensive Routers can do by now.


P.S.: This Idea does not include(on the first moment) more complex features of 
Flowspec like Redirect ou Rate-Limt.

 

Any suggestions or ideas? 

 

 

 

 

-- 

Douglas Fernando Fischer
Engº de Controle e Automação

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: Mark Tinka 
> Sent: Tuesday, June 16, 2020 12:09 PM
> 
> On 16/Jun/20 12:00, adamv0...@netconsultings.com wrote:
> 
> > Hence my earlier comment on why I think it's not commercially feasible
> > to switch to v6 control plane,
> 
> Personally, I've never been a fan of a single-stack backbone. I can, however,
> understand the use-case where a new or growing network is unable to
> obtain anymore IPv4 space and don't want to use RFC 1918 space (yuck!).
> 
Actually I was exactly I that situation and v4 RFC 1918 space worked out just 
fine.

> 
> >  (the only thing I'd be getting is MPLS-IPv6 which I already have (or
> > have had) via L3VPN-6VPE),
> 
> Well, not quite.
> 
> What you currently have with 6PE is IPv6 tunneled inside MPLSv4 which runs
> over IPv4. While you get the benefits of MPLS forwarding for your
> IPv6 traffic, you now create a condition where your IPv6 network is in the
> hands of your IPv4 network. Granted, there are many folk that run 6PE, so
> whether the fate-sharing is of concern to you or not is an exercise left up to
> the reader. Personally, I'd rather avoid fate-sharing whenever I can.
> 
I've been dependent solely on v4 all my life and I still am. 
But I see your fate-sharing argument, similar to my argument around separate 
iBGP infrastructure (Route-Reflector plane) for Internet vs for other customer 
private VPN prefixes. 
But in the multiplanar iBGP case one plane is statistically more likely to fail 
than the other, whereas in case of v4 vs v6 control plane I'd say it's actually 
the NEW v6 that's more likely hiding some unforeseen bug. 
So let me ask the following "devil's advocate" type of question, under the 
assumption that the LDPv6 is a new thing (not as proven as LDPv4), are you 
taking dependency away by splitting control plane to v4 and v6 or actually 
adding dependency - where the NEW v6 control plane components could negatively 
affect the existing v4 control plane components after all they share a common 
RE (or even RDP in Junos). 

> On the other hand, MPLSv6 is native, runs over IPv6 and does not depend on
> IPv4 at all.
> 
> Ultimately, plenty of energy will need to go into supporting the additional
> VPN services that go beyond plain-old MPLSv6 switching. But that can only be
> promoted with the vendors after we jump the first hurdle of deploying the
> 1st application, basic MPLSv6 switching; get that widely adopted, and create
> more awareness within the vendor community about its overall viability.
> 
> 80% of our network currently runs LDPv6 and switches IPv6 traffic in MPLSv6.
> Our immediate task is to get the remaining 20% supported (IOS
> XE) as well.
> 
> 
> > But I'm thankful to you for doing the "ice breaking" for the rest of the
> community.
> 
> As Eriq La Salle unashamedly claimed in the 1988 Eddie Murphy picture,
> Coming to America, "You know me, anything for the kids :-)".
> 
That was a good movie :D :D :D

adam

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: Mark Tinka 
> Sent: Monday, June 15, 2020 4:07 PM
> 
> On 15/Jun/20 12:13, adamv0...@netconsultings.com wrote:
> 
> > Not to mention this whole thread is focused solely on next-hop
> identification -which is just the lowest of the layers of abstraction in the
> vertical stack.
> > We haven’t talked about other "entities" that need identification like:
> VPNs, applications, policies (yes I'm looking at you VXLAN!) etc... - all of
> which are way better identified by a simple label rather than IPinIPinIP
> 
> The problem is if you want to have MPLS services signaled over IPv6, you first
> need an IPv6 control plane that will signal the labels that will support those
> services, i.e., LDPv6 and RSVPv6.
> 
> Right now, all vendors that support LDPv6 do so only for pure MPLS-IPv6
> forwarding. If you want l2vpnv6, l3vpnv6, EVPNv6, 4PE, 4VPE, TE-FRRv6,
> e.t.c., you can't get that today. You'd have to signal those services over 
> LDPv4
> or RSVPv4.
> 
Hence my earlier comment on why I think it's not commercially feasible to 
switch to v6 control plane, (the only thing I'd be getting is MPLS-IPv6 which I 
already have (or have had) via L3VPN-6VPE), time will tell if I ever need to 
make the switch. 
But I'm thankful to you for doing the "ice breaking" for the rest of the 
community.  

adam 

RE: Router Suggestions

[adamv0025]

> On 6/15/20 8:00 AM, Colton Conor wrote:
> For around $11,000 right now, you can get a brand new Juniper MX204
> router. Alternatively, you can get a used MX240 / MX480 with quad
> power supplies, redundant quad core RE's, and 2 16X10G MIC cards for
> around $12,000.
>
> My question, is there anything else worth looking at in this price
> range / port configuration? Open to both new and used options. Looking
> to take full BGP routes.

We all like our edge boxes to be not susceptible to any old garbage that might 
come at them in form of BGP advertisements from the Internet (500 AS-PATH 
prepends, 1000s of communities, or straight out exotic attributes - just to 
name a few).  So I'd recommend a vendor that has some pedigree in providing 
facilities to withstand these wild whims of the Internet and ability to 
normalize/bleach the routing information before sending it to the rest of your 
AS via iBGP.  

Examples:
RFC 7606 - Revised Error Handling for BGP UPDATE Messages
Max as path limit
Max community limit
Max prefix per session limit
Attribute filtering
Etc...

adam  

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: Saku Ytti 
> Sent: Monday, June 15, 2020 11:02 AM
> 
> On Mon, 15 Jun 2020 at 12:46,  wrote:
> 
> > Yes it can indeed, and that's moving towards the centre between the
> extreme cases that David laid out.
> > It's about how granular one wants to be in identifying an end-to-end path
> between a pair of edge nodes.
> > I agree with you that MPLS is still better than IP, and I tried to
> > illustrate that even enumerating every possible paths using deep label
> stack is not a problem (and even that can be alleviated using hierarchy of
> LSPs).
> 
> The entirety of my point is, if we were rational, we'd move towards
> increasingly efficient solutions. And technically everything we do in MPLS
> tunnels, we can do in IP tunnels and converse. Should we imagine a future
> where all features and functions are supported in both, it's clear we should
> want to do MPLS tunnels. Just the [IGP][BGP-LU] 8B overhead, compared to
> IP 40B overhead should drive the point home, and ultimately, that's the only
> difference, rest is implementation.
> 
> And I'm saddened we've been marketed snake-oil like SRv6 with fake
> promises of inherent advantages or simplicity 'just IP'.
> 
> We can do better than MPLS, absolutely. But IP is worse.
> 
Yes I absolutely agree,

Not to mention this whole thread is focused solely on next-hop identification 
-which is just the lowest of the layers of abstraction in the vertical stack. 
We haven’t talked about other "entities" that need identification like: VPNs, 
applications, policies (yes I'm looking at you VXLAN!) etc... - all of which 
are way better identified by a simple label rather than IPinIPinIP

adam  

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: Saku Ytti 
> Sent: Monday, June 15, 2020 10:31 AM
> 
> On Mon, 15 Jun 2020 at 12:24,  wrote:
> 
> 
> > Yes this is where each node needs to have a label uniquely identifying
> > every LSP passing through it.
> > Saku,
> > With IP header you don't need this,
> > Consider this:
> > PE1 to PE2 via 3 P-core nodes
> > With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2,
> > which will be load-shared across all 3 paths.
> > Using MPLS If you need to uniquely identify each path you need 3 FECs
> > (3 LSPs one via each P core node), now imagine you have 100K possible
> > paths across the fabric  -that's a lot of FECs on PE1 or any node in
> > the fabric where each has to have a unique label for every possible
> > unique path via the core that the particular node is part of.
> 
> Are we talking about specific implementations of fundamentals? It sounds
> like we are talking about a specific case where IP next-hop is unilist of N 
> next-
> hops, and MPLS next-hop is a single item without indirection? This is not a
> fundamental difference, this is implementation detail.
> There is no particular reason MPLS next-hop couldn't be unilist of N
> destinations.
> 
Yes it can indeed, and that's moving towards the centre between the extreme 
cases that David laid out.
It's about how granular one wants to be in identifying an end-to-end path 
between a pair of edge nodes.
I agree with you that MPLS is still better than IP, 
and I tried to illustrate that even enumerating every possible paths using deep 
label stack is not a problem (and even that can be alleviated using hierarchy 
of LSPs). 
  
adam

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> David Sinn
> Sent: Friday, June 12, 2020 4:42 PM
> 
> > On Jun 12, 2020, at 8:26 AM, Saku Ytti  wrote:
> >
> > On Fri, 12 Jun 2020 at 18:16, David Sinn  wrote:
> >
> The label stack question is about the comparisons between the two
> extremes of SR that you can be in. You either label your packet just for
it's
> ultimate destination or you apply the stack of the points you want to pass
> through.
> 
> In the former case you are, at the forwarding plane, equal to what you see
> with traditional MPLS today, with every node along the path needing to
> know how to reach the end-point. Yes, you have lowered label space from
> traditional MPLS, but that can be done with site-cast labels already. And,
> while the nodes don't have to actually swap labels, when you look at
> commodity implementations (across the last three generations since you
> want to do this with what is deployed, not wholesale replace the network)
a
> null swap still ends up eating a unique egress next-hop entry. So from a
> hardware perspective, you haven't improved anything. Your ECMP group
> count is high.
> 
Yes this is where each node needs to have a label uniquely identifying every
LSP passing through it.
Saku,
With IP header you don't need this, 
Consider this: 
PE1 to PE2 via 3 P-core nodes 
With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2, which
will be load-shared across all 3 paths. 
Using MPLS If you need to uniquely identify each path you need 3 FECs (3
LSPs one via each P core node), now imagine you have 100K possible paths
across the fabric
 -that's a lot of FECs on PE1 or any node in the fabric where each has to
have a unique label for every possible unique path via the core that the
particular node is part of.

> In the extreme latter case, you have to, on ingress, place the full stack
of
> every "site" you want to pass through. That has the benefits of "sites"
only
> need labels for their directly connected sites, so you have optimized the
> implications on commodity hardware. However, you now have a label stack
> that can be quite tall. At least if you want to walk the long-way around
the
> world, say due to failure. On top, that depth of label stack means devices
in
> the middle can't look at the original payload to make ECMP decisions. So
you
> can turn to entropy labels, but that sort of makes matters worse.
> 
David,
You can use hierarchy of tunnels to help with the deep label-stack
imposition problem.  

adam

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: David Sinn
> Sent: Friday, June 12, 2020 4:19 PM
>
> > On Jun 11, 2020, at 2:02 PM, Mark Tinka  wrote:
> >
> > On 11/Jun/20 17:32, David Sinn wrote:
> >
> >> Respectfully, that is deployment dependent. In a traditional SP
topology
> that focuses on large do everything boxes, where the topology is fairly
point-
> to-point and you only have a small handful of nodes at a PoP, labels can
be
> fast, cheap and easy. Given the lack of ECMP/WECMP, they remain fairly
> efficient within the hardware.
> >>
> >> However if you move away from large multi-chip systems, which hide
> internal links which can only be debugged and monitored if you know the
the
> obscure, often different ways in which they are partially exposed to the
> operator, and to a system of fixed form-factor, single chip systems,
labels fall
> apart at scale with high ECMP.
> >
> > I'm curious about this statement - have you hit practical ECMP issues
> > with label switching at scale?
> 
> Yes. Path enumeration when you use mult-tier Clos topologies within a PoP
> causes you many, many problem.
> 
Hi David, 

Can you be more specific please? Maybe some examples with numbers. 

I can see how you might run out of L2 rewrite/adjacency table space on a
particular node if you enumerate every possible path downstream of it
(especially on leaf nodes), cause that number is has a dependency on the
size of the fabric in terms of the total number of links in the fabric
(which balloons quickly). 
Let's focus on the alternate case then, (the deep label-stack one) 
At each node in the multi-tier Clos (which I assume is the Russ White's
butterfly model? But any Clos or Benes fabric needs the same) you need to
program a label to uniquely identify each egress interface ,so now there's
this nice one-to-one relationship between label and egress interface. Now
the depth of the label-stack depends on the number of hops the packet needs
to traverse across the fabric. But how deep could the fabric realistically
be? Even in the "butterfly" model with separate pods instead of leaf nodes I
counted 9 hops (that's not ultra-deep is it)? It's the VM doing label
imposition as programmed by the fabric controller (all in SW so can go as
deep as you want) and all fabric nodes are just popping top label so no big
deal.

adam 

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: Mark Tinka 
> Sent: Thursday, June 11, 2020 3:59 PM
> 
> > No, my line of reasoning is if you have MPLS LSPs signalled over v4 I see no
> point having them signalled also over v6 in parallel.
> 
> It's not about signaling IPv4 LSP's over IPv6.
> LDPv4 creates IPv4 FEC's.
> LDPv6 creates IPv6 FEC's.
> 
> The idea is to create IPv6 FEC's so that IPv6 traffic can be label-switched in
> the network natively, allowing you to remove BGPv6 in a native dual-stack
> core.
> 
Right I see what you are striving to achieve is migrate from BGP in a core to a 
BGP free core but not leveraging 6PE or 6VPE? 

> 
> As you can see, just as with IPv4, IPv6 packets are now being MPLS-switched
> in the core, allowing you to remove BGPv6 in the core and simplify
> operations in that area of the network.
> 
> So this is native MPLSv6. It's not 6PE or 6VPE.
> 
So considering you already had v4 FECs wouldn't it be simpler to do 6PE/6VPE, 
what do you see as drawbacks of these compared to native MPLSv6 please?
 
> > Apart from X months worth of functionality, performance, scalability and
> interworking testing -network wide code upgrades to address the bugs
> found during the testing process and then finally rollout across the core and
> possibly even migration from LDPv4 to LDPv6, involving dozens of people
> from Arch, Design, OPS, Project management, etc... with potential for things
> to break while making changes in live network.
> 
> Which you wouldn't have to do with SRv6, because you trust the vendors?
> 
Well my point was that if v4 FECs would be enough to carry v6 traffic then I 
wouldn't need SRv6 nor LDPv6, hence I'm curious to hear from you about the 
benefits of v6 FEC over v4 FEC (or in other words MPLSv6 vs 6PE/6VPE). 

adam

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: David Sinn
> Sent: Thursday, June 11, 2020 4:32 PM
> 
> However if you move away from large multi-chip systems, 
> to a system of fixed form-factor, single chip systems, labels fall
> apart at scale with high ECMP. Needing to enumerate every possible path
> within the network or having to have a super-deep label stack removes all
of
> the perceived benefits of cheap and simple. 
>
Looks like the deployments you describe are large DC Clos/Benes fabric, then
the potentially deep label imposition would be done by the VMs right?
On transit nodes the 64x ECMP or super-deep labels is no problem for
NPU/lookup process as it's always just top label lookup and resolving to a
single egress interface.

adam 

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: Saku Ytti 
> Sent: Thursday, June 11, 2020 3:29 PM
> 
> On Thu, 11 Jun 2020 at 16:45, Mark Tinka  wrote:
> 
> > Not sure if beating up on Job for months qualifies as "a customer
> > wanting RPKI from NTT" :-).
> 
> I'm sure we can blame Job for this, why not. But probably because of his
> lobbying some customers are _requiring_, i.e. flat out told they will stop
> accepting transit offers from providers who don't do RPKI.
> 
Case in point. 

On the other hand not sure if any of the customers would care whether LSPs are 
signalled over v4 as opposed to v6. 

adam 

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: Mark Tinka 
> Sent: Thursday, June 11, 2020 1:56 PM
> 
> On 11/Jun/20 14:17, adamv0...@netconsultings.com wrote:
> 
> > Well RPKI or DNSSEC is at least adding a value, even something you can
> brag about to your customers (we are part of the solution not part of the
> problem etc...).
> 
> Bragging doesn't bring in income, it's just PR :-).
> 
Good PR might ;)

> 
> >
> > But running MPLS over IPv6 in addition to already running it over IPv4,
> gaining new functionality or features in the process that could be ultimately
> monetized in providing better service to customers? -maybe even exposing
> the network to a new set of bugs?  -I don't know, that doesn’t sound like a
> good use of company resources especially in these uncertain times . Maybe
> I'm being overly harsh here maybe if you could please explain the drivers or
> expected net value out of this exercise please?
> 
> Oh dear, you sound like our Finance department now; "drivers" and "net
> value" :-).
> 
I thought I sound like a network architect :( 

> If I take your line of reasoning, deploying SRv6 likely requires new hardware,
> which means $$. How much money will I charge customers for my shiny new
> SRv6?
> 
No, my line of reasoning is if you have MPLS LSPs signalled over v4 I see no 
point having them signalled also over v6 in parallel.
Or if your full-mesh RSVP-TE is killing your RSVP-TE or you're in need of TE, 
then might want to look at SR MPLS. 

> LDPv6 builds on LDPv4. Just like IPv6 builds on IPv4. At best, you can remove
> BGPv6 from your core, 
I'm using VPNv6 & VPNv4 so not sure I follow how LDPv6 allows for removal of 
BGPv6 (is it BGPv6 over v4 NHs/transport?) but then again if it works over v4...

> which lowers your administration costs in that part of
> the network even further, costing you less in human time running it,
> resources you can otherwise re-deploy to other time- and money-saving
> activities. At worst, you get IPv4/IPv6 feature parity, and who doesn't like
> that :-).
> 
I knew there's a bit of OCD hidden in this at some level :)

> And how much money did LDPv6 cost you to deploy? $0.
> 
Apart from X months worth of functionality, performance, scalability and 
interworking testing -network wide code upgrades to address the bugs found 
during the testing process and then finally rollout across the core and 
possibly even migration from LDPv4 to LDPv6, involving dozens of people from 
Arch, Design, OPS, Project management, etc... with potential for things to 
break while making changes in live network. 

   
adam

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> From: Mark Tinka  
> Sent: Thursday, June 11, 2020 10:21 AM
>
>> -what additional revenue stream am I getting by enabling v6 in the
>> underlay/management plane that would offset the pain of dealing with the
>> increased bug surface?  
>
> Also, if you link every feature to a revenue stream, you'll never deploy 
> RPKI or DNSSEC :-).
>
Well RPKI or DNSSEC is at least adding a value, even something you can brag 
about to your customers (we are part of the solution not part of the problem 
etc...).

But running MPLS over IPv6 in addition to already running it over IPv4, gaining 
new functionality or features in the process that could be ultimately monetized 
in providing better service to customers? -maybe even exposing the network to a 
new set of bugs?  -I don't know, that doesn’t sound like a good use of company 
resources especially in these uncertain times . Maybe I'm being overly harsh 
here maybe if you could please explain the drivers or expected net value out of 
this exercise please?  


adam

RE: [c-nsp] LDPv6 Census Check

[adamv0025]

> Mark Tinka
> Sent: Wednesday, June 10, 2020 6:19 PM
> 
> Hi all.
> 
> Just want to sample the room and find out if anyone here - especially
those
> running an LDP-based BGPv4-free core (or something close to it) - would be
> interested in LDPv6, in order to achieve the same for BGPv6?
> 
> A discussion I've been having with Cisco on the matter is that they do not
> "see any demand" for LDPv6, and thus, won't develop it (on IOS XE).
> Meanwhile, it is actively developed, supported and maintained on IOS XR
> since 5.3.0, with new features being added to it as currently as 7.1.1.
> 
> Needless to say, a bunch of other vendors have been supporting it for a
> while now - Juniper, Nokia/ALU, Huawei, even HP.
> 
> IOS XR supporting LDPv6 notwithstanding, Cisco's argument is that "the
> world" is heavily focused on deploying SRv6 (Segment Routing). While I
know
> of one or two questionable deployments, I'm not entirely sure much of the
> world is clamouring to deploy SR, based on all the polls we've done at
various
> NOG meetings and within the general list-based operator community
> 
> So I just wanted to hear from this operator community on whether you
> would be interested in having LDPv6 support to go alongside your LDPv4
> deployments, especially if you run native dual-stack backbones. Or if your
> focus is totally on SRv6. Or if you don't care either way :-). Thanks.
> 
Hey Mark,
My stance is that should I go with anything "new" for label distribution the
MPLS SR/SPRING is getting to a point where it might be mature enough.
Also "BGP free core" means internet won't talk to your core -i.e. free to
use private addressing -so no need for v6 at all in the "underlay" (as
hipsters call it these days).
Alternatively using public "infrastructure subnet" (i.e. not advertised to
the Internet) for a "BGP free core", the aim is to make money of the core
-what additional revenue stream am I getting by enabling v6 in the
underlay/management plane that would offset the pain of dealing with the
increased bug surface?  

And with regards to the XE/XR discrepancies, I mentioned my prophecy a
number of times, I think XE future in SP products portfolio is next to none.



adam 

RE: Rate-limiting BCOP?

[adamv0025]

> Saku Ytti
> Sent: Friday, May 22, 2020 7:52 AM
> 
> On Thu, 21 May 2020 at 22:11, Bryan Holloway  wrote:
> 
> > I've done all three on some level in my travels, but in the past it's
> > also been oftentimes vendor-centric which hindered a scalable or
> > "templateable" solution. (Some things police in only one direction, or
> > only well in one direction, etc.)
> 
> Further complication, let's assume you are all-tomahawk on ASR9k.
> Let's assume TenGigE0/1/2/3/4 as a whole is pushing 6Gbps traffic across all
> VLAN, everything is in-contract, nothing is being dropped for any VLAN in any
> class. Now VLAN 200 gets DDoS attack of 20Gbps coming from single
> backbone interface. I.e. we are offering that tengig interftace 26Gbps of
> traffic. What will happen is, all VLANs start dropping packets QoS unaware,
> 12.5Gbps is being dropped by the ingress NPU which is not aware to which
> VLAN traffic is going nor is it aware of the QoS policy on the egress VLAN. 
>
Hmm, is that so?
Shouldn’t the egress FIA(NPU) be issuing fabric grants (via central Arbiters) 
to ingress FIA(NPU) for any of the VOQs all the way up till egress NPU's 
processing capacity, i.e. till the egress NPU can still cope with the overall 
pps rate (i.e. pps rate from fabric & pps rate from "edge" interfaces), subject 
to ingress NPU fairness of course?
Or in other words, shouldn't all or most of the 26Gbps end up on egress NPU, 
since it most likely has all the necessary pps processing capacity to deal with 
the packets at the rate they are arriving, and decide for each based on local 
classification and queuing policy whether to enqueue the packet or drop it?  

Looking at my notes, (from discussions with Xander and Thuijs and Aleksandar 
Vidakovic):
Each 10G entity is represented by on VQI = 4 VOQs (one VOQ for each priority 
level)
The trigger for the back-pressure is the utilisation level of RFD buffers. 
RFD buffers are holding the packets while the NP microcode is processing them. 
If you search for the BRKSPG-2904, the more feature processing the packet goes 
through, the longer it stays in RFD buffers.
RFD buffers are from-fabric feeder queues. - Fabric side backpressure kicks in 
if RFD queues are more than 60% full

So according to the above, should the egress NPU be powerful enough to deal 
with 26Gbps of traffic coming from fabric in addition to whatever business as 
usual duties its performing, (i.e RFD queues utilization is below 60%) then no 
drops should occur on ingress NPU (originating the DDoS traffic).
  

> So
> VLAN100 starts to see NC, AF, BE, LE drops, even though the offered rate in
> VLAN100 remains in-contract in all classes.
> To mitigate this to a degree on the backbone side of the ASR9k you need to
> set VoQ priority, you have 3 priorities. You could choose for example BE P2,
> NC+AF P1 and LE Pdefault. Then if the attack traffic to
> VLAN200 is recognised and classified as LE, then we will only see
> VLAN0100 LE dropping (as well as every other VLAN LE) instead of all the
> classes.
> 

YANG module designer tool

[adamv0025]

Hi folks, 
Just wanted to ask what are your favourite tools for designing YANG models
and why?
Are these two all there is:
https://www.mg-soft.com/mgYangDesigner.html
https://github.com/CiscoDevNet/yang-explorer

Or maybe some plugins for notepad++ or totalcmd/midnightcmd :)

Thanks a bunch.

adam
 

RE: Applications of MPLS in the metro area

[adamv0025]

Hi,

So where the books talk about PEs -think of your metro nodes here (basically 
converting the metro into an MPLS network -or making it part of your existing 
MPLS core) (you might not have a classic design where PEs hang off of P-Core 
nodes and might have just rings of PEs in your metro area)  

And where the books talk about various L3VPN and L2VPN services that’s 
basically what you can offer over your metro -now that it’s been converted to a 
fully-fledged MPLS network.

Ranging from multicast L3VPNs for 3PALY services through L2 p2p|p2mp|mp2mp 
services for Dat-Center-Interconect, to network-slicing buzzword (cause with 
VRFs and Traffic Engineering you can slice your metro area network whichever 
way you like).  

  

adam 

 

From: NANOG  On Behalf Of Etienne-Victor Depasquale
Sent: Tuesday, April 28, 2020 2:44 PM
To: NANOG 
Subject: Applications of MPLS in the metro area

 

Hello !

 

I'm looking for what a network operator would consider a realistic reference 
deployment of MPLS within the metro area network. 

 

By "realistic reference", I'm asking about what a network operator would 
consider to be a typical, perhaps most common, application of MPLS technology.

 

>From a bookish perspective, I understand MPLS well but have never implemented 
>it in the scope of my current field of study (metro area networks). I would 
>dearly like to get this "grounded" perspective from anyone who might care to 
>share it.

 

 

Cheers,

 

Etienne

 

-- 

Ing. Etienne-Victor Depasquale
Assistant Lecturer
Department of Communications & Computer Engineering
Faculty of Information & Communication Technology
University of Malta

Web. https://www.um.edu.mt/profile/etiennedepasquale

RE: IS-IS Error (FRR)

[adamv0025]

Interesting, so it is an MTU problem after all, I take it there’s no way to 
adjust the BPF (Berkeley Packet Filter) limit to let the 6020B sized PSNPs 
through?

 

adam

 

From: Mark Tinka  
Sent: Sunday, April 19, 2020 1:47 AM
To: Tony Li ; adamv0...@netconsultings.com
Cc: nanog@nanog.org
Subject: Re: IS-IS Error (FRR)

 

So we've been discussing this on Slack.

It seems the packet handlers for IS-IS in FRR and how they work on FreeBSD is 
not completely optimized (not an issue on Linux).

FRR-based IS-IS on FreeBSD is sending PSNP's that are 6,020 bytes large. 
However, FreeBSD's BPF (Berkeley Packet Filter) has a packet data limit of 
4,096 bytes. This is what is causing the error.

While it is possible to set the IIH MTU in FRR with the "lsp-mtu" command, 
there is not way to set the PSNP MTU.

In short, IS-IS on FRR does not understand that there is an MTU limit for 
PSNP's on FreeBSD.

I'm working with the IS-IS FRR team, but as of now, no idea on if/when there 
will be a fix for this.

While routes are being successfully exchanged between FRR and Cisco IOS XE for 
IS-IS, this is a show-stopper! So for the moment, I cannot recommend anyone 
runs IS-IS on FRR in production.

As it stands, it appears that this is the only thing standing in the way of a 
successful deployment, as far as simple Anycast services go.

Will keep you all posted as this develops.

Mark.

On 18/Apr/20 16:34, Tony Li wrote:

 

No, IIH’s are padded, hot PSNP’s.

 

Tony





On Apr 18, 2020, at 3:08 AM, adamv0...@netconsultings.com 
  wrote:

 

Just shooting in the dark, Isn’t the PSNP padded to some ISIS defined max MTU 
and that somehow fails due to mismatch on em0 L2 MTU?

 

adam

From: NANOG    On 
Behalf Of Mark Tinka
Sent: Friday, April 17, 2020 11:05 AM
To: nanog@nanog.org  
Subject: IS-IS Error (FRR)

 

Hi all.

I'm almost there getting IS-IS to work without issue. I'm now faced with the 
following error log:

2020/04/17 10:02:01 ISIS: IS-IS bpf: could not transmit packet on em0: 
Input/output error
2020/04/17 10:02:01 ISIS: [EC 67108865] ISIS-Snp (1): Send L2 PSNP on em0 failed

This repeats every second.

Anyone know what this could be?

Systems is FreeBSD-12.1-RELEASE-p3.

Despite the error, IS-IS is running and routing is good, talking to a Cisco IOS 
XE implementation on the other side. So the error is throwing me off.

No feedback from the Slack feed, Google doesn't say much, and waiting to hear 
back from the FRR list.

Mark.

 

RE: IS-IS Error (FRR)

[adamv0025]

Just shooting in the dark, Isn’t the PSNP padded to some ISIS defined max MTU 
and that somehow fails due to mismatch on em0 L2 MTU?

 

adam

From: NANOG  On Behalf Of Mark Tinka
Sent: Friday, April 17, 2020 11:05 AM
To: nanog@nanog.org
Subject: IS-IS Error (FRR)

 

Hi all.

I'm almost there getting IS-IS to work without issue. I'm now faced with the 
following error log:

2020/04/17 10:02:01 ISIS: IS-IS bpf: could not transmit packet on em0: 
Input/output error
2020/04/17 10:02:01 ISIS: [EC 67108865] ISIS-Snp (1): Send L2 PSNP on em0 failed

This repeats every second.

Anyone know what this could be?

Systems is FreeBSD-12.1-RELEASE-p3.

Despite the error, IS-IS is running and routing is good, talking to a Cisco IOS 
XE implementation on the other side. So the error is throwing me off.

No feedback from the Slack feed, Google doesn't say much, and waiting to hear 
back from the FRR list.

Mark.

RE: attribution

[adamv0025]

> Christopher Morrow
> Sent: Tuesday, April 14, 2020 2:51 AM
> 
> On Mon, Apr 13, 2020 at 7:38 PM Brandon Martin
>  wrote:
> >
> > On 4/13/20 4:31 PM, Randy Bush wrote:
> > > it seems a lot of folk think prepending acrually works.
> >
> > I mean, there's prepending and then there's prepending 50+ times...
> > Has the latter EVER been useful in any way, shape, or form?
> 
> for ~4 yrs or so there's been possible problems with as-paths longer than ~50
> (I think, i can't recall the exact vendor bug) so, folk should have already 
> been
> denying announcements with longer than ~soemthing-like-45 asn in the
> path.. right? :)
> 
>From memory this was one of the two accidents (someone prepending their AS 255 
>times and an university announcing special unheard-of attribute) that 
>triggered the good work around RFC 7606 - Revised Error Handling for BGP 
>UPDATE Messages.
And why Randy and we all can enjoy messages like
Apr 12 17:57:42 r0.iad rpd[1752]: Prefix Send failed ! 103.148.41.0/24 
bgp_rt_trace_too_big_message:1209 path attribute too big. Cannot build update.
Or
RP/0/RSP0/CPU0:Jan 18 00:22:41.029 : bgp[1058]: %ROUTING-BGP-3-MALFORM_UPDATE : 
Malformed UPDATE message received from neighbor x.x.x.x (VRF: INTERNET) - 
message length 87 bytes, error flags 0x0040, action taken "DiscardAttr". 
Error details: "Error 0x0040, Field "Attr-unexpected", Attribute 128 (Flags 
0xe0, Length 18), Data [e0801200]". NLRIs: [IPv4 Unicast] <>
While our BGP sessions keep on working just fine and either the update is 
treated as withdraw or the attribute is deleted.
 

On the point of as-path length limit, Yes I know of at least one tier-1 that 
does it and since I left some 8 years back I do it everywhere I go.
In addition to the above (best common practice, id' say)
-on junos you can do community length limiting
-and on cisco you can do attribute filtering  -hence my question to this forum 
some time back about whether folks do filter all the "experiments" for the sake 
of running a successful business (paraphrasing...)

adam

RE: interesting troubleshooting

[adamv0025]

> Saku Ytti
> Sent: Saturday, March 21, 2020 4:26 PM
> 
> On Sat, 21 Mar 2020 at 18:19, Mark Tinka  wrote:
> 
> > So the three or four times we tried to get FAT going (in a
> > multi-vendor network), it simply didn't work.
> 
> Yeah we run it in a multivendor network (JNPR, CSCO, NOK), works.
> 
> I would also recommend people exclusively using CW+FAT and disabling LSR
> payload heuristics (JNPR default, but by default won't do with CW, can do
> with CW too).
> 
And I'd add entropy labels too -for L3VPN traffic.
Using all this you know where to look (at PE edge) for any hashing related 
problems.

adam

RE: Hi-Rise Building Fiber Suggestions

[adamv0025]

> Warren Kumari
> Sent: Wednesday, February 26, 2020 4:31 PM
> 
> On Wed, Feb 26, 2020 at 11:20 AM Coy Hile  wrote:
> >
> > On 2020-02-26 11:14, Randy Bush wrote:
> > >> We use plenty of multi-mode, but only in the data centre, between
> > >> our own kit, for racks within the same cage.
> > >
> > > so you have to stock both single and multi?  hmmm
> > >
> > > randy
> >
> > I'd expect that from the ToR -> Servers would be MMF, but that other
> > infrastructure cabling would be SMF.
> > Even using aftermarket optics, putting single-mode transceivers in
> > every server and access port would quickly become cost-prohibitive,
> > would it not?
> 
> Cisco GLC-SX-MM Compatible 1000BASE-SX SFP 850nm 550m DOM
> Transceiver Module - $6.00 - https://www.fs.com/products/11774.html
> Cisco SFP-GE-L Compatible 1000BASE-LX/LH SFP 1310nm 10km DOM
> Transceiver Module - $7.00 - https://www.fs.com/products/12622.html
> 
> Yup, it is $1.00 more for SM, and you need 2 per link, but unless you are 
> doing
> *lots* that's likely not cost-prohibitive. The delta on 10G is a bit more 
> ($21 vs
> $18), but still not crazy-pants territory...
> 
Unfortunately I'm currently stuck with 40G breakout into 10G towards the floor 
switches, which is $39 vs $299 -hence my question regarding SMF vs MMF MPO/MTP 
for the risers. 
Alternatively killing a 40G port with single 10G.

adam 

RE: Hi-Rise Building Fiber Suggestions

[adamv0025]

> Joel Jaeggli
> Sent: Wednesday, February 26, 2020 4:46 AM
> 
> > There are two fiber pairs running up the building riser. I need to put a POE
> switch on each floor using this fiber.
> 
> You didn’t specify if the existing fiber is single or multi-mode however 
>
On that note would you gents recommend single-mode or multimode fiber for 
buildings?

adam

RE: Hi-Rise Building Fiber Suggestions

[adamv0025]

> Sent: Wednesday, February 26, 2020 4:46 AM
> To: Norman Jester 
> 
> Sent from my iPhone
> 
> > On Feb 25, 2020, at 18:34, Norman Jester  wrote:
> >
> > I’m in the process of choosing hardware for a 30 story building. If
> > anyone has experience with this I’d appreciate any tips.
> >
> > There are two fiber pairs running up the building riser. I need to put a POE
> switch on each floor using this fiber.
> 
> In my experience with retrofitting existing structures, if you have access to
> the riser at each floor as it sounds like you do, you would typically drop in 
> a
> new duct,  blow micro duct through it with a branch for each floor, have an
> MDF  or two In a utility spaces  and them you have the ability to reconfigure
> the fiber as necessary to meet your present and future needs.
> 
> You didn’t specify if the existing fiber is single or multi-mode however it is
> unlikely that the was enough slack built into two fiber runs to make 30
> additional splices so that approach seems dubious as a premise.
> 
> As you correctly surmise daisy chaining 30 switches is not an advisable
> network design practice.
> 
+1 to that,
Put your own fiber in and do a star topology to an MDF device.

adam

RE: Dual Homed BGP

[adamv0025]

Reading all the arguments one could generalize that choosing default/partial 
routes (instead of full feed) one is basically outsourcing all the control, 
convergence speed, security, etc.. to upstream providers.

 

adam

 

From: NANOG  On Behalf Of Amir Herzberg
Sent: Monday, January 27, 2020 1:49 PM
To: Job Snijders 
Cc: NANOG 
Subject: Re: Dual Homed BGP

 

Dear Job and NANOG,

 

Just wondering, wouldn't any of you guys consider using full tables in this 
case, for  the ability to detect and avoid prefix hijacks (using RPKI/ROV or 
other means)? 

 

Of course, I'm focused on security, and I know this is often not a high 
priority for a real network manager who has many other considerations; just 
want to know. Thanks. 

-- 
Amir 

 

 

On Fri, Jan 24, 2020 at 12:27 PM Job Snijders mailto:j...@instituut.net> > wrote:

Dear Brian,

 

On Fri, 24 Jan 2020 at 17:40, Brian mailto:brian@gmail.com> > wrote:

Hello all. I am having a hard time trying to articulate why a Dual Home ISP 
should have full tables. My understanding has always been that full tables when 
dual homed allow much more control. Especially in helping to prevent Async 
routes.

 

The advantage of receiving full routing tables from both providers is that in 
cases where one of the two providers is not yet fully converged, your routers 
will use the other provider for those missing destinations. This may happen 
during maintenance or router boot-up in your upstream’s network.

 

Another advantage of receiving full routes is that you can manipulate 
LOCAL_PREF per destination, or compose routing policy based on per-route 
attributes such as BGP communities your upstreams set. It can happen that a 
provider is great for 99% of destinations, except a few - without full tables 
such granular traffic-engineering can be cumbersome.

 

Virtually all internet routing is asymmetric, I wouldn’t consider that an 
issue. 

 

Am I crazy? 

 

I dropped out of university, never completed my psychology studies, I fear I am 
unqualified to answer this question. ;-)

 

Kind regards,

 

Job

RE: Peering/Transit eBGP sessions -pet or cattle?

[adamv0025]

> Baldur Norddahl
> Sent: Wednesday, February 12, 2020 7:57 PM
> 
> On Tue, Feb 11, 2020 at 12:33 AM Lukas Tribus  wrote:
> > Therefore, if being down for several minutes is not ok, you should 
> > invest in dual links to your transits. And connect those to two 
> > different routers. If possible with a guarantee the transits use two 
> > routers at their end and that divergent fiber paths are used etc.
> 
> That is not my experience *at all*. I have always seen my prefixes 
> converge in a couple of seconds upstream (vs 2 different Tier1's).
> 
> This is a bit old but probably still thus:
> 
> https://labs.ripe.net/Members/vastur/the-shape-of-a-bgp-update
> 
> Quote: "To conclude, we observe that BGP route updates tend to 
> converge globally in just a few minutes. The propagation of newly 
> announced prefixes happens almost instantaneously, reaching 50% 
> visibility in just under 10 seconds, revealing a highly responsive 
> global system. Prefix withdrawals take longer to converge and generate 
> nearly 4 times more BGP traffic, with the visibility dropping below 10% only 
> after approximately 2 minutes".
> 
> Unfortunately they did not test the case of withdrawal from one router 
> while having the prefix still active at another.
> 
Yes that's unfortunate,
Although I'm thinking that the convergence time would be highly dependent on 
the first-hop upstream providers involved in the "local-repair" for the 
affected AS -once that is done doesn't matter that the whole world still routes 
traffic to affected AS towards the original first-hop upstream AS, as long as 
it has a valid detour route.
And I guess the topology configuration of this first-hop outskirt from the 
affected AS involved in the "local-repair" would dictate the convergence time.
E.g. if your upstream A box happens to have a direct (usable) link/session to 
upstream B box -winner, however the higher the number of boxes involved in the 
"local-repair" detour that need to be told "A no more, now B is the way to go" 
the longer the convergence time.
-but if significant portion of the Internet gets withdraw in 2 min -wondering 
how long could it be for a typical "local-repair" string of bgp speakers to all 
get the memo.
-but realistically how many bgp speakers could that be, ranging from min 2 - to 
max... say ~6? 
   


> 
> When I saw *minutes* of brownouts in connectivity it was always 
> because of ingress prefix convergence (or the lack thereof, due to 
> slow FIB programing, then temporary internal routing loops, nasty 
> things like that, but never external).
> 
> That is also a significant problem. In the case of a single transit 
> connection per router, two routers and two providers, there will be a 
> lot of internal convergence between your two routers in the case of a 
> link failure. That is also avoided by having both routers having the same 
> provider connections.
> That way a router may still have to invalidate many routes but there 
> will be no loops and the router has loop free alternatives loaded into 
> memory already (to the other provider). Plus you can use the simple 
> trick of having a default route as a fall back.
> 
This is a very good point actually, indeed since the box has two transit 
sessions in case of a failure of only one of them it will still retain all the 
prefixes in FIB -it will just need to reprogram few next-hops to point towards 
the other eBGP/iBGP speakers, whoever offers a best path. And reprograming 
next-hops is significantly faster (with hierarchical FIBs anyways).

adam

RE: Peering/Transit eBGP sessions -pet or cattle?

[adamv0025]

> Baldur Norddahl
> Sent: Monday, February 10, 2020 3:06 PM
> 
> No matter how much money you put into your peering router, the session 
> will be no more stable that whatever the peer did to their end.
>
Agreed, that's a fair point,  

> Plus at some
> point you will need to reboot due to software upgrade or other reasons. 
>
There are ways of draining traffic for planned maintenance.

> If
> you care at all, you should be doing redundancy by having multiple 
> locations, multiple routers. You can then save the money spent on each 
> router, because a router failure will not cause any change on what the 
> internet sees through BGP.
> 
I think router failure will cause change on what the Internet sees as you 
rightly outlined below:

> Also transits are way more important than peers. Loosing a transit 
> will cause massive route changes around the globe and it will take a 
> few minutes to stabilize. Loosing a peer usually just means the peer 
> switches to the transit route, that they already had available.
> 
agreed and I suppose the questions is whether folks tend to try minimizing 
these impacts by all means possible or just take it as necessary evil that will 
eventually happen.

> Peers are not equal. You may want to ensure redundancy to your biggest 
> peers, while the small fish will be fine without.
> 
> To be explicit: Router R1 has connections to transits T1 and T2. 
> Router R2 also has connections to the same transits T1 and T2. When 
> router R1 goes down, only small internal changes at T1 and T2 happens. 
> Nobody notices and the recovery is sub second.
> 
Good point again,
Though if I had only T1 on R1 and only T2 on R2 then convergence won't happen 
inside each Transit but instead between T1 and T2 which will add to the 
convergence time. 
So thinking about it seems the optimal design pattern in a distributed 
(horizontally scaled out) edge would be to try and pair up -i.e. at least two 
edge nodes per Transit (or Peer for that matter), in order to allow for 
potentially faster intra-Transit convergence rather than arguably slower 
inter-transit convergence.  
 
adam

Peering/Transit eBGP sessions -pet or cattle?

[adamv0025]

Hi,

 

Would like to take a poll on whether you folks tend to treat your
transit/peering connections (BGP sessions in particular) as pets or rather
as cattle.

And I appreciate the answer could differ for transit vs peering connections.

However, I'd like to ask this question through a lens of redundant vs
non-redundant Internet edge devices.

To explain, 

a.  The "pet" case:

Would you rather try improving the failure rate of your transit/peering
connections by using resilient Control-Plane (REs/RSPs/RPs) or even
designing these as link bundles over separate cards and optical modules? 

Is this on the bases that doesn't matter how hard you try on your end (i.e.
distribute your traffic to multitude of transit and peering connections or
use BFD or even BGP-PIC Edge to shuffle thing around fast, any disruption to
the eBGP session itself will still hurt you in some way, (i.e. at least some
partial outage for some proportion of the traffic for not insignificant
period of time) until things converge in direction from The Internet back to
you. 

 

b.  The "cattle" case:

Or would you instead rely on small-ish non-redundant HW at your internet
edge rather than trying to enhance MTBF with big chassis full of redundant
HW? 

Is this cause eventually the MTBF figure for a particular transit/peering
eBGP session boils down to the MTBF of the single card or even single
optical module hosting the link, (and creating bundles over separate cards
-well you can never be quite sure how the setup looks like on the other end
of that connection)?

Or is it because the effects of a smaller/non-resilient border edge device
failure is not that bad in your particular (maybe horizontally scaled)
setup?

 

Would appreciate any pointers, thank you.

Thank you

 

adam

 

RE: BGP Path Attribute Filtering, YES or NO?

[adamv0025]

> From: Saku Ytti 
> Sent: Wednesday, January 8, 2020 1:09 PM
> 
> On Wed, 8 Jan 2020 at 14:46,  wrote:
> 
> > Other  might be: “These experimental work is of great value to the
> community and there’s a process now to announce and manage these
> experiments, what about net neutrality, and besides modern BGP
> implementations should handle well formatted attributes and if it’s not the
> case its good that these flaws are being exposed and fixed.”
> 
> This is my position. Unfortunately it's a pipe dream, as you only need very
> few to think filtering is needed to ruin the utility.
> 
In an ideal world that would be my position too, but I suppose it depends on 
the context,  
Imagine: 
CTO: Could you have prevented this major financial and market loss and damage 
to our reputation resulting from this major network outage, is this something 
that never happened before and couldn't be foreseen?
Me: Nah happened already and sure I could have simply dropped the offending BGP 
attribute 254 in this case since it's not used anyways.
CTO: What the ..., why haven’t you do so then?!?!?
Me: Well because "this experimental work is of great value to the community and 
there’s a process now to announce and manage these experiments, what about net 
neutrality, and besides modern BGP implementations should handle well formatted 
attributes and if it’s not the case it's good that these flaws are being 
exposed and fixed".
CTO: You mean exposed like this? Like breaking my network?!?!? Get the ... out 
of here you're fired

  

> Some specific examples
> 
> - don't clean up communities which don't belong to you (
Agreed, whatever you do only condition communities with your AS# on them.  

> - don't clean up TOS byte (I may want to communicate QoS over internet
> between my islands)
Agreed, will dump it all into best-effort or scavenger class in my MPLS 
backbone anyways, along with all the SD-WAN super high priority stuff...
Falls into do not touch transit traffic unless under DoS.

> - don't clean up BGP attributes (128 would have utility if it transit, but 
> due to
> old issues, it often does not)
Looking at https://www.iana.org/assignments/bgp-parameters/bgp-parameters.xhtml 
(don't know how well maintained it is actually), I see 128 as assigned to 
ATTR_SET [RFC6368], so if I filtered only Unassigned, Deprecated and Reserved 
from that list that shouldn't do any harm right?

> - don't drop ICMP (ICMP TS would be high utility if not filtered)
So obvious that you shouldn't even mention it, but again falls into do not 
touch transit traffic unless under DoS,
Traffic (ICMP included) destined to your infrastructure -well that's subject to 
the iACL policies.  

> I think we need specific good reason to mangle/filter and if you cannot come
> up with one, don't do it. If you can come up with one, consider if it's
> persistent or workaround to deal with specific active defect.
> 
Well the BGP attribute induced outage has a precedence and had quite a positive 
fallout in terms of BGP enhanced error handling etc...
My pipedream is to have the time to shoot random stuff at BGP to see what 
happens and then report back to vendors about my findings,...
But no, this is not part of our software certification test suite.

adam

BGP Path Attribute Filtering, YES or NO?

[adamv0025]

Would like to gather current views of a wider community on BGP Path
Attribute Filtering (discarding selected attributes in particular, not treat
as withdraw) as an addition to the long list of standard conditioning tools
like max as-path length limit, limiting number of communities all the way to
running iBGP infrastructure to carry Internet prefixes separate to the one
carrying customers' L3/L2VPN prefixes. 

 

And I appreciate the topic is somewhat contentious and there's no simple yes
or no answer either.

 

My view is that in a stub AS there should be no harm in discarding unused
BGP path attributes,

On transit AS-es I'd expect two opposing views:

One might be: "I have a business to run and don't care about some university
experiments, so unless any of my customers specifically asks for some
attribute I'll drop all reserved, unassigned and deprecated ones and might
even drop some not widely used ones just to be on the well-trodden bug free
path"

Other  might be: "These experimental work is of great value to the community
and there's a process now to announce and manage these experiments, what
about net neutrality, and besides modern BGP implementations should handle
well formatted attributes and if it's not the case its good that these flaws
are being exposed and fixed."

 

Please let me know your thoughts.

 

adam  

 

 

 

 

 

 

RE: FRR as Route-Reflector & Scaling stats

[adamv0025]

> ERCIN TORUN
> Sent: Friday, November 15, 2019 9:34 AM
> 
> Hello Rakesh,
> 
> As James said, better to ask it at FRR mailing list.
> 
> Generally chipset is what limits the scale (e.g. trident2 is 128k ipv4 lpm
> https://docs.cumulusnetworks.com/cumulus-linux/Layer-3/Routing/ ).  If
> you disable "zebra" daemon, FRR works only in control-plane then you would
> most likely have a limitation with memory/RAM only. (speed is another
> issue).
> 
Data-plane lookup memory limitations have nothing to do with the scale of a RR 
function, as you eluded to (if the RR is in path then it has to act as any 
other routing node so FIB scaling limitations apply -but that is completely 
orthogonal to the RR function). 
One would assume that NOS to be used for a crucial role in the overall BGP 
infrastructure would feature the essential ability to limit the installation 
(complete/selective) of routes to FIB/data-plane. (or in the modern virtual 
deployments lack the data-plane altogether). 

adam   

RE: BGP over TLS

[adamv0025]

> From: Christopher Morrow 
> Sent: Friday, October 25, 2019 7:08 AM
> 
> > > > So move from bilateral peering over common IX-LAN to direct
> > > > peering Or if a direct link is still not to be trusted do MACSEC.
> > > > Then it's all about you and the peer -if he/she screws you over de-peer.
> > >
> > > and it burning a 100g port on a chassis for ~1g (or less) of traffic
> > > is worthwhile...
> > Sure cause the usual math is if I have 100G to IX LAN which is half empty
> then going direct means 100G x number of peers I'd like to move to direct
> peering...
> 
> Sorry, I skipped a sentence figuring it wasn't worth typing, but ...
> "100g is the new 10g which was the new 1g ... and on 100g platforms you
> often don't get 1g as well... it's harder to delivery 10G (almost) than 100g 
> :( "
> 
> So default at the edge is moving to 100g for some folk
> 
Nah that's just advertising, 100G is still not quite there yet:
100G SR4 99,-
40G SR4 39,-
10G SR 18,-
1000BASE-T 18,-
1000BASE-SX 6,-

100G LR 799,-
40G LR 279,-
10G LR 24,-
1G LX 7,-


Yes these are just indicative transceiver prices, but it sort of speaks to the 
argument of 100G for anything cause it doesn't matter.
Also from the forwarding chip perspective,
100G being a new 10G is currently has a faint chance only on the most 
"simplest" merchant silicon, it will take a long time till this notion trickles 
down and is reflected in prices of vendor NPUs/PFEs or cards.
e.g. try comparing list prices of a 32 port 10G card to a 32 port 100G card and 
then tell me it doesn't really matter whether you'll burn a 10G or 100G port 
for the 1g peer :)

adam

RE: BGP over TLS

[adamv0025]

> From: Christopher Morrow 
> Sent: Wednesday, October 23, 2019 6:53 PM
> Subject: Re: BGP over TLS
> 
> On Wed, Oct 23, 2019 at 10:43 AM  wrote:
> >
> > > Sent: Tuesday, October 22, 2019 8:26 PM
> > > To: Keith Medcalf 
> > >
> > > No,
> > >
> > >
> > > > On Oct 22, 2019, at 2:08 PM, Keith Medcalf 
> > > wrote:
> > > >
> > > > At this point further communications are encrypted and secure
> > > > against
> > > eavesdropping.
> > >
> > > The problem isn't the protocol being eavesdropped on. The data is
> > > already published publicly by many people.
> > >
> > > The problem is one of mutual authentication and authorization of the
> > > transport.
> > >
> > Yes the information is public but if the routing information exchanged
> > over a given peering session is tempered with that could potentially
> > cause some problems right?
> 
> 'mutual authentication' CAN be the thing (via tcp md5 today) that does this
> 'do not tamper' part.
> there ARE problems with tcp-md5... some are "because we collectively didnt'
> squeak enough to get key-tables"
> some are: "err, doing low-level-kernel-muckery is gross".
> 
> > But then again, as Jeff mentioned, with GTSM this vector is limited to
> > a local link between two eBGP speakers (or whole IGP domain for iBGP
> > sessions but let's leave that one out for now).
> 
> ibgp, it turns out, is important.
> 
Well yes and that's why we all have internet plane RR infrastructure separate 
to the RR infrastructure(s) distributing information for other services right?
And is also why we all filter non-standard or even non-usual BGP attributes and 
unusually long AS paths and community lists on ingress to our AS, just to 
safeguard our BGP infrastructure.
...yeah about that. 

> > So move from bilateral peering over common IX-LAN to direct peering Or
> > if a direct link is still not to be trusted do MACSEC.
> > Then it's all about you and the peer -if he/she screws you over de-peer.
> 
> and it burning a 100g port on a chassis for ~1g (or less) of traffic is
> worthwhile...
Sure cause the usual math is if I have 100G to IX LAN which is half empty then 
going direct means 100G x number of peers I'd like to move to direct peering... 

> there's lots of variables here... options are nice to have... better security 
> for
> /not just my IX/ bgp would be nice too :)
Well yes I do agree with that statement on a sentimental level almost.
But pragmatically I'd have to ask what big of a problem is BGP over TLS going 
to solve and how much would it cost. 
Ideally the solution would tick all these 3 together: 
1) it's a panacea (solves all aspects of the problem), 
2) it's zero effort (development and getting it to the network)
3) it solves a huge problem, (operators are struggling with the effects of this 
problem on a daily bases)

adam

RE: BGP over TLS

[adamv0025]

> Sent: Tuesday, October 22, 2019 8:26 PM
> To: Keith Medcalf 
> 
> No,
> 
> 
> > On Oct 22, 2019, at 2:08 PM, Keith Medcalf 
> wrote:
> >
> > At this point further communications are encrypted and secure against
> eavesdropping.
> 
> The problem isn't the protocol being eavesdropped on. The data is already
> published publicly by many people.
> 
> The problem is one of mutual authentication and authorization of the
> transport.
> 
Yes the information is public but if the routing information exchanged over
a given peering session is tempered with that could potentially cause some
problems right?

But then again, as Jeff mentioned, with GTSM this vector is limited to a
local link between two eBGP speakers (or whole IGP domain for iBGP sessions
but let's leave that one out for now).
So move from bilateral peering over common IX-LAN to direct peering 
Or if a direct link is still not to be trusted do MACSEC.
Then it's all about you and the peer -if he/she screws you over de-peer.

adam

RE: Request comment: list of IPs to block outbound

[adamv0025]

> From: Saku Ytti 
> Sent: Tuesday, October 22, 2019 11:54 AM
> 
> On Mon, 21 Oct 2019 at 23:14,  wrote:
> 
> > The obvious drawback especially for TCAM based systems is the scale,
> > so not only we'd need to worry if our FIB can hold 800k prefixes, but
> > also if the filter memory can hold the same amount -in addition to
> > whatever additional filtering we're doing at the edge (comb filters
> > for DoS protection etc...)
> 
> This is actually somewhat cheap problem, if you optimise for it. That is rules
> are somewhat expensive, but N prefixes per rule are not, when designed
> with that requirement. Certainly the BOM effect can be entirely ignored.
> However this is of course only true if that was design goal, won't help in a
> situation where HW is in place and doesn't not scale there. Just pointing out
> that there are no technical or commercial problems getting there, should we
> so want.
> 
Well sure if BGP prefix=ACL prefix was true from the get go both scaling 
problems would be catered for in unison and we wouldn't even notice.
People here would be asking for recommendations on new/replacement edge router 
that can support 1M routes and filter entries...
But the reality is that long filters can  significantly decrease performance of 
modern (supporting 100G interfaces) NPUs/PFEs.

adam

RE: Request comment: list of IPs to block outbound

[adamv0025]

> -Original Message-
> From: NANOG  On Behalf Of Lukas Tribus
> Sent: Friday, October 18, 2019 9:45 PM
> To: Saku Ytti 
> Cc: nanog@nanog.org
> Subject: Re: Request comment: list of IPs to block outbound
> 
> Hello,
> 
> On Fri, Oct 18, 2019 at 7:40 PM Saku Ytti  wrote:
> > It's interesting to also think, when is good time to break things.
> >
> > CustomerA buys transit from ProviderB and ProviderA
> >
> > CustomerA gets new prefix, but does not appropriately register it.
> >
> > ProviderB doesn't filter anything, so it works. ProviderA does filter
> > and does not accept this new prefix. Neither Provider has ACL.
> >
> >
> > Some time passes, and ProviderB connection goes down, the new prefix,
> > which is now old prefix experiences total outage. CustomerA is not
> > happy.
> >
> >
> > Would it have been better, if ProviderA would have ACLd the traffic
> > from CustomerA? Forcing the problem to be evident when the prefix is
> > young and not in production. Or was it better that it broke later on?
> 
> That's an orthogonal problem and it's solution hopefully doesn't require a
> traffic impacting ingress ACL.
> 
> I'm saying this breaks valid configurations because even with textbook IRR
> registrations there is a race condition between the IRR registration (not a
> route-object, but a new AS in the AS-MACRO), the ACL update and the BGP
> turn-up of a new customer (on AS further down).
> 
> 
> Here's a environment for the examples below:
> 
> Customer C1 uses existing transits Provider P11 and P12 (meaning C1 is
> actually a production network; dropping traffic sourced by it in the DFZ is 
> very
> bad; P11 and P12 is otherwise irrelevant).
> Customer C1 is about to turn-up a BGP session to Provider P13.
> Provider P13 is a Tier2 and buys transit from Tier1 Providers P1 and P2
> Provider P2 deploys ingress ACLs depending on IRR data, based on P13's AS-
> MACRO.
> 
I still think that ACLs rule should go hand in hand with eBGP prefixes by 
default.  
But the ACLs should be updated based on advertised and accepted eBGP prefixes 
automatically (so not dependent on external data).
If the IRR data accuracy and AS-MACROs get solved the filtering problem would 
be solved as well.
If such mechanism was enabled by default in all vendors' implementations it 
would address the double lookup problem of uRPF while accomplishing the same 
thing and even address the source IP spoofing problem.
3 Simple rules:

Rule 1)
If you are advertising a prefix 
Then allow it as source prefix in your egress ACL
And allow it as destination prefix in you ingress ACL 
(cause why do you advertise a prefix well you expect to send traffic sourced 
from IPs covered by that prefix and you expect to get a response back right?)
And as a result
Traffic sourced from IPs haven't advertised via a particular link would be 
blocked at egress from your AS (on that link) -boundary A1
Traffic destined to IPs you haven't advertised via a particular link it will be 
blocked at ingress to you AS (on that link)  

Rule 2)
If you are accepting a prefix 
Then allow in as source in your ingress ACL 
And allow it as destination in your egress ACL
(cause why do you accept a prefix well you expect to send traffic towards IPs 
covered by that prefix and you'd want those IPs to be able to respond back 
right?) 
And as a result
Traffic sourced from IPs you haven't accepted via a particular link would be 
blocked at ingress to your AS (on that link) -boundary A2
Traffic destined to IPs you haven't accepted via a particular link would be 
blocked at egress from your AS (on that link) -required because there's already 
an egress ACL blocking everything.

Rule 3)
If interface can't be uniquely identified based on IPs used for the eBGP 
session warn the operator about the condition  

The obvious drawback especially for TCAM based systems is the scale, so not 
only we'd need to worry if our FIB can hold 800k prefixes, but also if the 
filter memory can hold the same amount -in addition to whatever additional 
filtering we're doing at the edge (comb filters for DoS protection etc...)

adam 

RE: Viability of GNS3 network simulation for testing features/configurations.

[adamv0025]

> problem I've run into is our IOS isn't supported

Not sure what you mean, like you can’t find the same exact version of IOS XRv 
9000?

Surely going with similar XRv version to your production one would be much 
closer than going with IOSv

 

adam 

 

From: NANOG  On Behalf Of rylandkremeier
Sent: Wednesday, October 16, 2019 6:12 PM
To: Yan Filyurin ; Jason Kuehl 
Cc:  
Subject: Re: Viability of GNS3 network simulation for testing 
features/configurations.

 

We have 9 ASR's so I don't think it would be too hard to host them in the GNS3 
vm insurance we're using. The main problem I've run into is our IOS isn't 
supported, which is where Cisco IOSv comes in, hoping it could be configured in 
a way to act very closely like our deployed hardware. 

 

I'm not so much worried about hardware faults, more so network configurations 
and testing of new methods. In a perfect world I would be able to copy the 
running configs from deployed hardware into GNS3. At least that's how closely I 
would like GNS3 running. 

 

Not a ton of info out there on IOSv, so I'm curious as to how it's configured. 
If it's the "universal" IOS that I would imagine it should be, then it could 
work. 

 

Thanks for the links, those are both things I didn't run across during initial 
research.

 

-- 

Ryland Kremeier

RE: Viability of GNS3 network simulation for testing features/configurations.

[adamv0025]

> From: Saku Ytti 
> Sent: Thursday, October 17, 2019 3:41 PM
> 
> On Thu, 17 Oct 2019 at 15:15,  wrote:
> 
> > But as you can see A) and B) can easily be tested with a single DUT (or some
> small topology around it) using actual HW plugged in a loop with IXIA/Spirent
> testers.
> 
> Snake topology does conserve IXIA/Spirent ports but will not allow you to
> test everything. I see no practical way of just having bunch of IXIA/Spirent
> ports to verify behaviour under various types of congestion. Unfortunately
> the 'bunch' is getting rather large, since even the smallest atom of a modern
> networking chip may contain dozens of 100GE ports.
> 
More IXIA/Spirent ports is your answer we use the "dumb" IXIA cards for NPU/PFE 
and fabric fairness testing as those are much cheaper.

adam 

RE: Viability of GNS3 network simulation for testing features/configurations.

[adamv0025]

I've been using network simulation well before GNS3 was around using
dynamips - and even when GNS3 came along it was still not good -since it
just couldn't handle the scale (~40nodes) (not on my compute resources at
that time anyways).

 

And similarly nowadays in the era of proper HW simulation through VMs
(though I miss the idle-pc), I really like virsh/libvirt along with OVS as
it allows me to programmatically generate the VM files (xmls, images, etc..)
and define the topology in OVS (talking hundreds of links) which would be
otherwise really tedious to draw by hand. 

Also spinning up a big virtual lab from scratch takes several hours (of pure
compute time) so it's better to have some meshing in between the nodes and
just spin up arbitrary L1 topologies on demand rather than spinning up the
VMs every time one needs to load a different topology.

Said that I haven't played with GNS3, EVE-NG, VIRL,. recently so I don't
know if any of these would allow me to create these massive "spreadsheets"
for programmatic generation of labs. 

 

Best approach is to have at least two virtual environments

1) closely resembling production environment -this is where designers and
Ops people can test day to day operational changes etc..

2) environment where architects can test strategic/evolution changes to the
network infrastructure, new concepts and big migration/integration projects,
etc.   

 

What is it good for:

Testing design concepts

-this is one of the biggest advantages of virtual testing

Physical labs as we all know cost a small fortune and you can simulate just
a small cross-sections of your overall topology at a time  -but in virtual
lab depending on your computing resources and depending on what you need to
test you can either simulate very large sections or complete network (at
lower resolution) or smaller sections with very high resolution or
combination of both.

This allows you to really see what happens to your traffic patterns and
assess the impact of your design changes from small to large scales.  

 

What is it not good for:

A) Scale testing 

i.e. how many bgp/bfd/vrrp/etc.. sessions how many routes/VRFs/etc. - you
need the actual HW resources to carry out these tests

B) Performance testing

How much pps I can drive through NPU with these features (QOS,filters,etc.)
what are the failover times, (fast reroute, fabric fail,RE fail, etc.)
-again you need the actual HW that will be used in production to measure
these

 

But as you can see A) and B) can easily be tested with a single DUT (or some
small topology around it) using actual HW plugged in a loop with
IXIA/Spirent testers. 

 

adam

 

From: NANOG  On Behalf Of Ryland Kremeier
Sent: Wednesday, October 16, 2019 4:31 PM
To:  
Subject: Viability of GNS3 network simulation for testing
features/configurations.

 

Hello,

 

I'm currently in the process of setting up a near identical network to our
own in GNS3 for testing purposes. Has anyone here tried this before to any
success? We need to buy the Cisco IOSv image to continue with the sim so I
figured I would inquire here first before diving in.

 

All info is appreciated,

-- 

Ryland Kremeier

RE: IPAM recommendations

[adamv0025]

> From: Owen DeLong 
> Sent: Friday, September 6, 2019 4:52 PM
> 
> You might also want to look at 6connect.
> 
> 
I actually did recently, to potentially migrate from phpipam, but I couldn't
even drive the IPAM thing :( 
Just saying, in phpipam I didn't have any trouble figuring out basic stuff
like how to name a subnet, assign IP to A-side host and B-side host on a p2p
link with /31 (or /30) subnet, or how to create children off of a parent
block in no time.
Though what I like about 6conenc is that its insanely customizable and I
like the automatic chopping of subnet to equal sized blocks and ability to
merge etc., but... as I said couldn't drive even the basic stuff... 
And regarding the automation capabilities -I actually like the full blown
ONAP style automation better. 
 
adam

RE: IPAM recommendations

[adamv0025]

I tried all 3 netbox, nipap and phpipam as a gui user and I like the phpipam 
the best as it has most knobs and features and is very intuitive, (can't 
comment on the api richness, yet)
But what is strange is that none of these tools provides RT (or 
extended/standard communities or even vlan for that matter) management the way 
they provide IP management, that is allow one to create custom pools and then 
have the next free resource allocated from a pool if needed.

adam