Re: A spammer is harvesting email addresses from the NANOG list.

[Bryan Fields]

On 6/20/20 6:42 PM, Bryan Fields wrote:
> On 6/20/20 5:33 PM, Tim Pozar wrote:
>> Looks like a spammer is harvesting email addresses from the NANOG list.  i
>> I will be unscribing as I don't need this additional noise in my mailbox.
> 
> Do you have the full headers of these emails?  Please send them along if you 
> do.
> 

Tim provided these and we've removed the interloper.  Should anyone see
anything like this again please send the complete email with headers to
ge...@nanog.org.

Thanks,
-- 
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: Devil's Advocate - Segment Routing, Why?

[Owen DeLong]

> On Jun 20, 2020, at 2:27 PM, Mark Tinka  wrote:
> 
> 
> 
> On 20/Jun/20 00:41, Anoop Ghanwani wrote:
> 
>> One of the advantages cited for SRv6 over MPLS is that the packet contains a 
>> record of where it has been.
> 
> I can't see how advantageous that is, or how possible it would be to 
> implement, especially for inter-domain traffic.
> 
> Mark.
> 

Since the packet is essentially source-routed, and the labels aren’t popped off 
the way they are in MPLS, but preserved in the hop by hop headers (AIUI), the 
implementation isn’t particularly difficult.

Owen

Re: Devil's Advocate - Segment Routing, Why?

[Sabri Berisha]

- On Jun 20, 2020, at 2:27 PM, Mark Tinka  wrote: 

Hi Mark,

> On 20/Jun/20 00:41, Anoop Ghanwani wrote:

>> One of the advantages cited for SRv6 over MPLS is that the packet contains a
>> record of where it has been.

> I can't see how advantageous that is, 

That will be very advantageous in a datacenter environment, or any other
environment dealing with a lot of ECMP paths. 

I can't tell you how often during my eBay time I've been troubleshooting
end-to-end packetloss between hosts in two datacenters where there were at least
10 or more layers of up to 16 way ECMP between them. Having a record of which
path is being taken by a packet is very helpful to determine the one with a 
crappy
transceiver.

> or how possible it would be to implement,

That work is already underway, albeit not specifically for MPLS. For example,
I've worked with an experimental version of In-Band Network Telemetry (INT)
as described in this draft: https://tools.ietf.org/html/draft-kumar-ippm-ifa-02

I even demonstrated a very basic implementatoin during SuperCompute 19 in Denver
last year. Most people who were interested in the demo were academics however,
probably because it wasn't a real networking event.

Note that there are several caveats that come with this draft and previous
versions, and that it is still very much work in progress. But the potential is
huge, at least in the DC.

> especially for inter-domain traffic.

That's a different story, but not entirely impossible. A probe packet can
be sent across AS borders, and as long as the two NOCs are cooperating, the
entire path can be reconstructed.

Thanks, 

Sabri

Re: A spammer is harvesting email addresses from the NANOG list.

[Bryan Fields]

On 6/20/20 5:33 PM, Tim Pozar wrote:
> Looks like a spammer is harvesting email addresses from the NANOG list.  i
> I will be unscribing as I don't need this additional noise in my mailbox.

Do you have the full headers of these emails?  Please send them along if you do.

-- 
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: A spammer is harvesting email addresses from the NANOG list.

[Mark Tinka]

They've been harassing me all day - I've been ignoring.

Mark.

On 20/Jun/20 23:33, Tim Pozar wrote:
> Looks like a spammer is harvesting email addresses from the NANOG list.  i
> I will be unscribing as I don't need this additional noise in my mailbox.
>
> Tim
>
> - Forwarded message from Michele Jemmi  -
>
> Date: Sat, 20 Jun 2020 16:06:05 -0500
> From: Michele Jemmi 
> To: po...@lns.com
> Subject: Re: Re: 60 ms cross-continent
>
>So, where did u leave?
>
> - End forwarded message -
>  Forwarded Message 
> Subject:  Re: Re: 60 ms cross-continent
> Date: Sat, 20 Jun 2020 15:59:05 -0500
> From: Michele Jemmi 
> Reply-To: Michele Jemmi 
> To:   po...@lns.com
>
> hiyaa u made my kitty sad
> - End forwarded message -
>  Forwarded Message 
> Subject:  Re: 60 ms cross-continent
> Date: Sat, 20 Jun 2020 15:52:05 -0500
> From: Michele Jemmi 
> Reply-To: Michele Jemmi 
> To:   po...@lns.com
>
> hey Having u so close is what I need the most, 
> =?UTF-8?Q?Tim_Po=C5=BE=C3=A1r?= I want to feel yur love completely. Do u want 
> me to go or will u come for me? ..
> - End forwarded message -
>

A spammer is harvesting email addresses from the NANOG list.

[Tim Pozar]

Looks like a spammer is harvesting email addresses from the NANOG list.  i
I will be unscribing as I don't need this additional noise in my mailbox.

Tim

- Forwarded message from Michele Jemmi  -

Date: Sat, 20 Jun 2020 16:06:05 -0500
From: Michele Jemmi 
To: po...@lns.com
Subject: Re: Re: 60 ms cross-continent

   So, where did u leave?

- End forwarded message -
 Forwarded Message 
Subject:Re: Re: 60 ms cross-continent
Date:   Sat, 20 Jun 2020 15:59:05 -0500
From:   Michele Jemmi 
Reply-To:   Michele Jemmi 
To: po...@lns.com

hiyaa u made my kitty sad
- End forwarded message -
 Forwarded Message 
Subject:Re: 60 ms cross-continent
Date:   Sat, 20 Jun 2020 15:52:05 -0500
From:   Michele Jemmi 
Reply-To:   Michele Jemmi 
To: po...@lns.com

hey Having u so close is what I need the most, =?UTF-8?Q?Tim_Po=C5=BE=C3=A1r?= 
I want to feel yur love completely. Do u want me to go or will u come for me? ..
- End forwarded message -

-- 
 GPG Fingerprint: 4821 CFDA 06E7 49F3 BF05  3F02 11E3 390F 8338 5B04
  https://pgp.mit.edu/pks/lookup?op=get=0x11E3390F83385B04
   https://keys.openpgp.org/search?q=pozar%40lns.com

Re: Devil's Advocate - Segment Routing, Why?

[Mark Tinka]

On 20/Jun/20 00:41, Anoop Ghanwani wrote:

> One of the advantages cited for SRv6 over MPLS is that the packet
> contains a record of where it has been.

I can't see how advantageous that is, or how possible it would be to
implement, especially for inter-domain traffic.

Mark.

Re: [c-nsp] Devil's Advocate - Segment Routing, Why?

[Mark Tinka]

On 20/Jun/20 19:58, Gert Doering wrote:

> The 6880/6840 products were promising at that time, but the pricing made
> it uninteresting.  So we kept our 6506Es for a time...

We haven't done anything with them since we bought and deployed them in
2014.

They are serving their purpose quite well, and when it's time for them
to go, the Arista kicks.


> ... and then went to Arista 7050SX2/SX3 for the "edge things" (small
> routing table, lots of 10G/25G ports, small power draw, EVPN/VXLAN).
>
> Nice stuff like the JSON RPC API, which is nice to work with and 
> amazingly *fast* (compared to JunOS commit times...).  And, most important,
> a good TAC and a company interested in listening to their customers.

We run the 7508E for core switching in large PoP's, and the 7280R for
data centre access aggregation in all PoP's (this replaced our Juniper
EX4550 and EX4600 platforms).

We are very happy - but these are pure Layer 2 switching use-cases.


> I'm a bit annoyed that the 7050SX* do not have MPLS-P support (because 
> we have MPLS PEs that basically "live behind" the 7050SX, and now need 
> to have vlans *through* them, to reach a MPLS P router).  
>
> I do understand that the BRCM Trident is fairly limited wrt MPLS handling, 
> so Arista decided "better no MPLS than something which is not enough 
> for people" - unfortunately for us, because we only want "LDP and 
> single-label swap/pop", but I can accept the technical arguments to 
> some extent.

You know how I feel about Broadcom chips :-).

If Arista aren't that comfortable about them, you'd do well to take them
at their word, hehe.


> (As a side note, changing our network from EIGRP to OSPF for IPv4 did
> not come for free, so I would have much preferred to stay in my self-
> chosen vendor lock-in with Cisco.  But Cisco has gone insane these days,
> and new boxes like the NCS5500 come *without* EIGRP support.  So they
> really do not want us customers locked in...)

Looking at things, it's probably cheaper for you to spend money on
getting an open IGP than spending money dealing with the indecision
Cisco sometimes goes through.

Mark.



signature.asc
Description: OpenPGP digital signature

Re: 60 ms cross-continent

[Tim Požár]

Did you not read my posting on Quora?

Tim

On 6/20/20 10:49 AM, Wayne Bouchard wrote:

And thus far, no one has mentioned switching speed and other
electronic overhead such as the transceivers (that's the big one,
IIRC.)

I also don't recall if anyone mentioned that the 30ms is as the
photon flies, not fiber distance.

-Wayne

On Sat, Jun 20, 2020 at 05:32:30PM +, Mel Beckman wrote:

An intriguing development in fiber optic media is hollow core optical fiber, 
which achieves 99.7% of the speed of light in a vacuum.

https://www.extremetech.com/computing/151498-researchers-create-fiber-network-that-operates-at-99-7-speed-of-light-smashes-speed-and-latency-records

-mel

On Jun 20, 2020, at 10:14 AM, Dave Cohen  wrote:

??? Doing some rough back of the napkin math, an ultra low-latency path from, 
say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is 
considerably longer than the I-90 driving distance would suggest because:
- Best case optical distance is more like 5500 km, in part because the path 
actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by 
right-of-way will be more like 5500 km when you account for things like 
maintenance coils, in-building wiring, etc.
- You???ll need (at least) three OEO regens on that distance, since there???s 
no value in spending 5x to deploy an optical system that wouldn???t need to 
(like the ones that would manage that distance subsea). This is in addition to 
~60 in-line amplification nodes, although that adds significantly less latency 
even in aggregate

Some of that is simply due to cost savings. In theory, you could probably spend 
a boatload of money to build a route that cuts off some of the distance 
inefficiency and gets you closer to 4500 km optical distance with minimal slack 
coil, and maybe no regens, so you get a real-world performance of 46 ms. But 
there are no algo trading sites of importance in DC, and for everybody else 
there???s not enough money in the difference between 46 and 59 ms for someone 
to go invest in that type of deployment.

Dave Cohen
craetd...@gmail.com

On Jun 20, 2020, at 12:44 PM, Tim Durack  wrote:

???
And of course in your more realistic example:

2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path

On Sat, Jun 20, 2020 at 12:36 PM Tim Durack 
mailto:tdur...@gmail.com>> wrote:
Speed of light in glass ~200 km/s

100 km rtt = 1ms

Coast-to-coast ~6000 km ~60ms

Tim:>

On Sat, Jun 20, 2020 at 12:27 PM William Herrin 
mailto:b...@herrin.us>> wrote:
Howdy,

Why is latency between the east and west coasts so bad? Speed of light
accounts for about 15ms each direction for a 30ms round trip. Where
does the other 30ms come from and why haven't we gotten rid of it?

c = 186,282 miles/second
2742 miles from Seattle to Washington DC mainly driving I-90

2742/186282 ~= 0.015 seconds

Thanks,
Bill Herrin

--
William Herrin
b...@herrin.us
https://bill.herrin.us/


--
Tim:>


--
Tim:>


---
Wayne Bouchard
w...@typo.org
Network Dude
http://www.typo.org/~web/

Re: 60 ms cross-continent

[Martin Hannigan]

On Sat, Jun 20, 2020 at 16:14 Bryan Fields  wrote:

> On 6/20/20 1:56 PM, Saku Ytti wrote:
> > On Sat, 20 Jun 2020 at 20:52, Wayne Bouchard  wrote:
> >
> >> And thus far, no one has mentioned switching speed and other
> >> electronic overhead such as the transceivers (that's the big one,
> >> IIRC.)
> > This will be something from tens of meters (low lat swich), to few
> > hundred meters (typical pipeline),  to 2km delay (NPU+FAB+NPU) per
> > active IP device. If that is a big one, I guess it depends, cross
> > atlantic, no, inside rack, maybe.
>
> I think he might be referring to the newer modulation types (QAM) on long
> haul
> transport.  There's quite a bit of time in uS that the encoding takes into
> QAM
> and adding FEC.  You typically won't see this at the plug-able level
> between
> switches and stuff.
>
> 60ms is nothing really, and I'm happy I don't need to play in the HFT space
> anymore.  I do wish my home connection wasn't 60 ms across town as spectrum
> wants takes TPA-ATL-DCA-DEN-NY to get to my rack. :-)



working on that .. :-)






> --
> Bryan Fields
>
> 727-409-1194 - Voice
> http://bryanfields.net
>

Re: 60 ms cross-continent

[Bryan Fields]

On 6/20/20 1:56 PM, Saku Ytti wrote:
> On Sat, 20 Jun 2020 at 20:52, Wayne Bouchard  wrote:
> 
>> And thus far, no one has mentioned switching speed and other
>> electronic overhead such as the transceivers (that's the big one,
>> IIRC.)
> This will be something from tens of meters (low lat swich), to few
> hundred meters (typical pipeline),  to 2km delay (NPU+FAB+NPU) per
> active IP device. If that is a big one, I guess it depends, cross
> atlantic, no, inside rack, maybe.

I think he might be referring to the newer modulation types (QAM) on long haul
transport.  There's quite a bit of time in uS that the encoding takes into QAM
and adding FEC.  You typically won't see this at the plug-able level between
switches and stuff.

60ms is nothing really, and I'm happy I don't need to play in the HFT space
anymore.  I do wish my home connection wasn't 60 ms across town as spectrum
wants takes TPA-ATL-DCA-DEN-NY to get to my rack. :-)
-- 
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: 60 ms cross-continent

[Marshall Eubanks]

This was also pitched as one of the killer-apps for the SpaceX
Starlink satellite array, particularly for cross-Atlantic and
cross-Pacific trading.

https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening-up-the-next-frontier-for-hft/

"Several commentators quickly caught onto the fact that an extremely
expensive network whose main selling point is long-distance,
low-latency coverage has a unique chance to fund its growth by
addressing the needs of a wealthy market that has a high willingness
to pay — high-frequency traders."

Regards
Marshall

On Sat, Jun 20, 2020 at 2:01 PM Carsten Bormann  wrote:
>
> On 2020-06-20, at 19:07, Joel Jaeggli  wrote:
> >
> > This is c in a vacuum. Light transmission through a medium is slower.
>
> Ob-movie: https://en.wikipedia.org/wiki/The_Hummingbird_Project
>
> Grüße, Carsten
>

Re: Devil's Advocate - Segment Routing, Why?

[Baldur Norddahl]

On Sat, Jun 20, 2020 at 12:38 PM Mark Tinka  wrote:

>
>
> On 20/Jun/20 11:27, Baldur Norddahl wrote:
>
>
>>
> We run the Internet in a VRF to get watertight separation between
> management and the Internet. I do also have a CGN vrf but that one has very
> few routes in it (99% being subscriber management created, eg. one route
> per customer). Why would this create a scaling issue? If you collapse our
> three routing tables into one, you would have exactly the same number of
> routes. All we did was separate the routes into namespaces, to establish a
> firewall that prevents traffic to flow where it shouldn't.
>
>
> It may be less of an issue in 2020 with the current control planes and how
> far the code has come, but in the early days of l3vpn's, the number of
> VRF's you could have was directly proportional to the number of routes you
> had in each one. More VRF's, less routes for each. More routes per VRF,
> less VRF's in total.
>
> I don't know if that's still an issue today, as we don't run the Internet
> in a VRF. I'd defer to those with that experience, who knew about the
> scaling limitations of the past.
>
>
I can't speak for the year 2000 as I was not doing networking at this level
at that time. But when I check the specs for the base mx204 it says
something like 32 VRFs, 2 million routes in FIB and 6 million routes in
RIB. Clearly those numbers are the total of routes across all VRFs
otherwise you arrive at silly numbers (64 million FIB if you multiply, 128k
FIB if you divide by 32). My conclusion is that scale wise you are ok as
long you do not try to have more than one VRF with a complete copy of the
DFZ.

More worrying is that 2 million routes will soon not be enough to install
all routes with a backup route, invalidating BGP FRR.

Regards,

Baldur

Re: 60 ms cross-continent

[Alejandro Acosta]

Hello,

  Taking advantage of this thread may I ask something?. I have heard of 
"wireless fiber optic", something like an antenna with a laser pointing 
from one building to the other, having said this I can assume this link 
with have lower RTT than a laser thru a fiber optic made of glass?



Thanks,


Alejandro,


On 6/20/20 1:11 PM, Dave Cohen wrote:
Doing some rough back of the napkin math, an ultra low-latency path 
from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. 
This is considerably longer than the I-90 driving distance would 
suggest because:
- Best case optical distance is more like 5500 km, in part because the 
path actually will go Chicago-NJ-WDC and in part because a distance of 
5000 km by right-of-way will be more like 5500 km when you account for 
things like maintenance coils, in-building wiring, etc.
- You’ll need (at least) three OEO regens on that distance, since 
there’s no value in spending 5x to deploy an optical system that 
wouldn’t need to (like the ones that would manage that distance 
subsea). This is in addition to ~60 in-line amplification nodes, 
although that adds significantly less latency even in aggregate


Some of that is simply due to cost savings. In theory, you could 
probably spend a boatload of money to build a route that cuts off some 
of the distance inefficiency and gets you closer to 4500 km optical 
distance with minimal slack coil, and maybe no regens, so you get a 
real-world performance of 46 ms. But there are no algo trading sites 
of importance in DC, and for everybody else there’s not enough money 
in the difference between 46 and 59 ms for someone to go invest in 
that type of deployment.


Dave Cohen
craetd...@gmail.com


On Jun 20, 2020, at 12:44 PM, Tim Durack  wrote:


And of course in your more realistic example:

2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path

On Sat, Jun 20, 2020 at 12:36 PM Tim Durack > wrote:


Speed of light in glass ~200 km/s

100 km rtt = 1ms

Coast-to-coast ~6000 km ~60ms

Tim:>

On Sat, Jun 20, 2020 at 12:27 PM William Herrin mailto:b...@herrin.us>> wrote:

Howdy,

Why is latency between the east and west coasts so bad? Speed
of light
accounts for about 15ms each direction for a 30ms round trip.
Where
does the other 30ms come from and why haven't we gotten rid
of it?

c = 186,282 miles/second
2742 miles from Seattle to Washington DC mainly driving I-90

2742/186282 ~= 0.015 seconds

Thanks,
Bill Herrin

-- 
William Herrin

b...@herrin.us 
https://bill.herrin.us/



-- 
Tim:>




--
Tim:>

Re: 60 ms cross-continent

[Carsten Bormann]

On 2020-06-20, at 19:07, Joel Jaeggli  wrote:
> 
> This is c in a vacuum. Light transmission through a medium is slower.

Ob-movie: https://en.wikipedia.org/wiki/The_Hummingbird_Project

Grüße, Carsten

Re: 60 ms cross-continent

[Saku Ytti]

On Sat, 20 Jun 2020 at 20:52, Wayne Bouchard  wrote:

> And thus far, no one has mentioned switching speed and other
> electronic overhead such as the transceivers (that's the big one,
> IIRC.)

This will be something from tens of meters (low lat swich), to few
hundred meters (typical pipeline),  to 2km delay (NPU+FAB+NPU) per
active IP device. If that is a big one, I guess it depends, cross
atlantic, no, inside rack, maybe.

-- 
  ++ytti

Re: [c-nsp] Devil's Advocate - Segment Routing, Why?

[Mark Tinka]

On 19/Jun/20 20:19, ljwob...@gmail.com wrote:

> >From the vendor standpoint, the market has never been able to agree on what 
> >makes a "core" application.  If I have five "big" customers, I guarantee you 
> >that:
>  - one of them will need really big ACLs, even though it's a "core" box to 
> them.
>  - one of them will want to terminate high speed L2VPN circuits, even though 
> it's a "core" box.
>  - one of them will need to do hierarchical shaping and granular QoS, even 
> though it's a "core" box. 
>  - one of them will want to have lots of headroom in the FIB (2-3x today's 
> global tables) ... even though it's a "core" box.
>  - one of them will want to buy something that they can "migrate out to the 
> edge" one day...
>
> Say it with me:  "even though it's a "core" box"
>
> This puts me (as the vendor) in a super weird spot... I can try to create a 
> card/PID that addresses *some subset* of this, charge a lot less for it, and 
> hopefully sell a bunch.  Or I'm left creating something that I can sell into 
> that broader market, but then that thing has to "do all the stuff" and I'm 
> going to charge for it appropriately.  
>
> The physics dictate that "Really High Speed Silicon" does not exist which can 
> solve all of those things.  I can solve SOME with a given chip design, but 
> not all. 

Well, if I'm honest, it's the vendors who created "tiers" back in the
day, so they can sell boxes in the way they did, and still do.

Ever since Ethernet became the gold standard, what a core, edge,
peering, branch, e.t.c., box is has been meaningless. It was meaningless
before Ethernet (I mean, to some, the Cisco 3640 was a core router,
while to others it was a peering router), but it's more meaningless in
the days of Ethernet.

The reason the ASR9000 sells better than the CRS or NCS 6000, is the
same reason the MX sells better than the PTX or T-series. It's all about
Ethernet.

So now, vendors who are able to build boxes that can license Ethernet
ports will be the winners, because I don't have to pay the upfront capex
for the entire card, and can just grow as I need. The Transport boys
have been doing it for so long, why can't the IP boys do the same? The
good news is, they have started.

Secondly, and perhaps more importantly, if vendors can build around a
single ASIC across a multitude of platforms, then there is hope. If you
have to build a different ASIC for every tier of a network, you end up
with the issues you raise, above. The reason the MX does so well is
because in addition to being an Ethernet platform, it uses the same ASIC
(since Trio, to be clear), which gives Juniper and its customers plenty
of flexibility across a variety of applications.

There is a lesson, here, to be learned.

Mark.

Re: 60 ms cross-continent

[Wayne Bouchard]

And thus far, no one has mentioned switching speed and other
electronic overhead such as the transceivers (that's the big one,
IIRC.)

I also don't recall if anyone mentioned that the 30ms is as the
photon flies, not fiber distance.

-Wayne

On Sat, Jun 20, 2020 at 05:32:30PM +, Mel Beckman wrote:
> An intriguing development in fiber optic media is hollow core optical fiber, 
> which achieves 99.7% of the speed of light in a vacuum.
> 
> https://www.extremetech.com/computing/151498-researchers-create-fiber-network-that-operates-at-99-7-speed-of-light-smashes-speed-and-latency-records
> 
> -mel
> 
> On Jun 20, 2020, at 10:14 AM, Dave Cohen  wrote:
> 
> ??? Doing some rough back of the napkin math, an ultra low-latency path from, 
> say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is 
> considerably longer than the I-90 driving distance would suggest because:
> - Best case optical distance is more like 5500 km, in part because the path 
> actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by 
> right-of-way will be more like 5500 km when you account for things like 
> maintenance coils, in-building wiring, etc.
> - You???ll need (at least) three OEO regens on that distance, since there???s 
> no value in spending 5x to deploy an optical system that wouldn???t need to 
> (like the ones that would manage that distance subsea). This is in addition 
> to ~60 in-line amplification nodes, although that adds significantly less 
> latency even in aggregate
> 
> Some of that is simply due to cost savings. In theory, you could probably 
> spend a boatload of money to build a route that cuts off some of the distance 
> inefficiency and gets you closer to 4500 km optical distance with minimal 
> slack coil, and maybe no regens, so you get a real-world performance of 46 
> ms. But there are no algo trading sites of importance in DC, and for 
> everybody else there???s not enough money in the difference between 46 and 59 
> ms for someone to go invest in that type of deployment.
> 
> Dave Cohen
> craetd...@gmail.com
> 
> On Jun 20, 2020, at 12:44 PM, Tim Durack  wrote:
> 
> ???
> And of course in your more realistic example:
> 
> 2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
> 
> On Sat, Jun 20, 2020 at 12:36 PM Tim Durack 
> mailto:tdur...@gmail.com>> wrote:
> Speed of light in glass ~200 km/s
> 
> 100 km rtt = 1ms
> 
> Coast-to-coast ~6000 km ~60ms
> 
> Tim:>
> 
> On Sat, Jun 20, 2020 at 12:27 PM William Herrin 
> mailto:b...@herrin.us>> wrote:
> Howdy,
> 
> Why is latency between the east and west coasts so bad? Speed of light
> accounts for about 15ms each direction for a 30ms round trip. Where
> does the other 30ms come from and why haven't we gotten rid of it?
> 
> c = 186,282 miles/second
> 2742 miles from Seattle to Washington DC mainly driving I-90
> 
> 2742/186282 ~= 0.015 seconds
> 
> Thanks,
> Bill Herrin
> 
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
> 
> 
> --
> Tim:>
> 
> 
> --
> Tim:>

---
Wayne Bouchard
w...@typo.org
Network Dude
http://www.typo.org/~web/

Re: [c-nsp] Devil's Advocate - Segment Routing, Why?

[Mark Tinka]

On 19/Jun/20 17:26, Gert Doering wrote:

> There's a special place in hell for people re-using the "Catalyst" brand
> name and then putting yearly renewable licenses on it.  Or IOS XE.
>
> I'm not actually sure *which* BU is doing "Catalyst" these days, but
> we're so annoyed about Cisco these days that I haven't really looked at
> all these new and shiny products from all these new and shiny BUs with
> all these new and shiny operating systems in quite a while.

We bought the C6880-X for our core switch back in 2014. It's still
humming, running plain old IOS.

The upgrade path is Arista for this. I'm not sure what switching is
doing over at Cisco, these days.

Mark.



signature.asc
Description: OpenPGP digital signature

Re: 60 ms cross-continent

[Mel Beckman]

An intriguing development in fiber optic media is hollow core optical fiber, 
which achieves 99.7% of the speed of light in a vacuum.

https://www.extremetech.com/computing/151498-researchers-create-fiber-network-that-operates-at-99-7-speed-of-light-smashes-speed-and-latency-records

-mel

On Jun 20, 2020, at 10:14 AM, Dave Cohen  wrote:

 Doing some rough back of the napkin math, an ultra low-latency path from, 
say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is 
considerably longer than the I-90 driving distance would suggest because:
- Best case optical distance is more like 5500 km, in part because the path 
actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by 
right-of-way will be more like 5500 km when you account for things like 
maintenance coils, in-building wiring, etc.
- You’ll need (at least) three OEO regens on that distance, since there’s no 
value in spending 5x to deploy an optical system that wouldn’t need to (like 
the ones that would manage that distance subsea). This is in addition to ~60 
in-line amplification nodes, although that adds significantly less latency even 
in aggregate

Some of that is simply due to cost savings. In theory, you could probably spend 
a boatload of money to build a route that cuts off some of the distance 
inefficiency and gets you closer to 4500 km optical distance with minimal slack 
coil, and maybe no regens, so you get a real-world performance of 46 ms. But 
there are no algo trading sites of importance in DC, and for everybody else 
there’s not enough money in the difference between 46 and 59 ms for someone to 
go invest in that type of deployment.

Dave Cohen
craetd...@gmail.com

On Jun 20, 2020, at 12:44 PM, Tim Durack  wrote:


And of course in your more realistic example:

2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path

On Sat, Jun 20, 2020 at 12:36 PM Tim Durack 
mailto:tdur...@gmail.com>> wrote:
Speed of light in glass ~200 km/s

100 km rtt = 1ms

Coast-to-coast ~6000 km ~60ms

Tim:>

On Sat, Jun 20, 2020 at 12:27 PM William Herrin 
mailto:b...@herrin.us>> wrote:
Howdy,

Why is latency between the east and west coasts so bad? Speed of light
accounts for about 15ms each direction for a 30ms round trip. Where
does the other 30ms come from and why haven't we gotten rid of it?

c = 186,282 miles/second
2742 miles from Seattle to Washington DC mainly driving I-90

2742/186282 ~= 0.015 seconds

Thanks,
Bill Herrin

--
William Herrin
b...@herrin.us
https://bill.herrin.us/


--
Tim:>


--
Tim:>

Re: Devil's Advocate - Segment Routing, Why?

[Mark Tinka]

On 20/Jun/20 14:41, Masataka Ohta wrote:

>  
> There are many. So, our research group tried to improve RSVP.

I'm a lot younger than the Internet, but I read a fair bit about its
history. I can't remember ever coming across an implementation of RSVP
between a host and the network in a commercial setting. If I missed it,
kindly share, as I'd be keen to see how that went.


>
> Practically, the most serious problem of RSVP is, like OSPF, using
> unreliable link multicast to reliably exchange signalling messages
> between routers, making specification and implementations very
> complicated.
>
> So, we developed SRSVP (Simple RSVP) replacing link multicast by,
> like BGP, link local TCP mesh (thanks to the CATENET model, unlike
> BGP, there is no scalability concern). Then, it was not so difficult
> to remove other problems.

Was "S-RSVP" ever implemented, and deployed?


>
> However, perhaps, most people think show stopper to RSVP is lack
> of scalability of weighted fair queueing, though, it is not a
> problem specific to RSVP and MPLS shares the same problem.

QoS has nothing to do with MPLS. You can do QoS with or without MPLS.

I should probably point out, also, that RSVP (or RSVP-TE) is not MPLS.
They collaborate, yes, but we'd be doing the community a disservice by
interchanging them for one another.


>
> Obviously, weighted fair queueing does not scale because it is
> based on deterministic traffic model of token bucket model
> and, these days, people just use some ad-hoc ways for BW
> guarantee implicitly assuming stochastic traffic model. I
> even developed a little formal theory on scalable queueing
> with stochastic traffic model.

Maybe so, but I still don't see the relation to MPLS.

All MPLS can do is convey IPP or DSCP values as an EXP code point in the
core. I'm not sure how that creates a scaling problem within MPLS itself.

If you didn't have MPLS, you'd be encoding those values in IPP or DSCP.
So what's the issue?


>
> So, we have specification and working implementation of
> hop-by-hop, scalable, stable unicast/multicast interdomain
> QoS routing protocol supporting routing hierarchy without
> clank back.
>
> See
>
> http://www.isoc.org/inet2000/cdproceedings/1c/1c_1.htm
>
> for rough description of design guideline.
>

If I understand this correctly, would this be the IntServ QoS model?

 
>
> I didn't attempt to standardize our result in IETF, partly
> because optical packet switching was a lot more interesting.

Still is, even today :-)?


> That should be a reasonable way of practical operation, though I'm
> not very interested in OCS (optical circuit switching) of GMPLS

Design goals are often what they are, and then the real world hits you.



> For IP layer, that should be enough. For ASON, so complicated
> GMPLS is actually overkill.
>
> When I was playing with ATM switches, I established control
> plain network with VPI/VCI=0/0 and assign control plain IP
> addresses to ATM switches. To control other VCs, simple UDP
> packets are sent to switches from controlling hosts.
>
> Similar technology should be applicable to ASON. Maintaining
> integrity between wavelength switches is responsibility
> of controllers.

Well, GMPLS and ASON is basically skinny OSPF, IS-IS and RSVP running in
a DWDM node's control plane.


>
> No, I just explained what was advertised to be MPLS by people
> around Cisco against Ipsilon.
>
> According to the advertisements, you should call what you
> are using LS or GLS, not MPLS or GMPLS.

It takes a while for new technology to be fully understood, which is why
I'm not rushing on to the SR bandwagon :-).

I can't blame the sales droids or the customers of the day. It probably
sounded like dark magic.


> Assuming a central controller (and its collocated or distributed
> back up controllers), we don't need complicated protocols in
> the network to maintain integrity of the entire network.

Well, that's a point of view, I suppose.

I still can't walk into a shop and "buy a controller". I don't know what
this controller thing is, 10 years on.

IGP's, BGP and label distribution protocols have proven themselves, in
the interim.


> What if, an inner label becomes invalidated around the
> destination, which is hidden, for route scalability,
> from the equipments around the source?

I can't say I've ever come across that scenario running MPLS since 2004.

Do you have an example from a production network that you can share with
us? I'd really like to understand this better.


> No, as "the destination expected to pop the label" is located somewhere
> around the final destination end-host.
>
> If, at the destination site, connectivity between a router to pop nested
> label and the fine destination end-host is lost, we are at a loss,
> unless source side changes inner label.

Maybe a diagram would help, as I still don't get this failure scenario.

If a host lost connectivity with the service provider network, getting
label switching to work is pretty low on 

Re: 60 ms cross-continent

[Dave Cohen]

Doing some rough back of the napkin math, an ultra low-latency path from, say, 
the Westin to 1275 K in Seattle will be in the 59 ms range. This is 
considerably longer than the I-90 driving distance would suggest because:
- Best case optical distance is more like 5500 km, in part because the path 
actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by 
right-of-way will be more like 5500 km when you account for things like 
maintenance coils, in-building wiring, etc.
- You’ll need (at least) three OEO regens on that distance, since there’s no 
value in spending 5x to deploy an optical system that wouldn’t need to (like 
the ones that would manage that distance subsea). This is in addition to ~60 
in-line amplification nodes, although that adds significantly less latency even 
in aggregate

Some of that is simply due to cost savings. In theory, you could probably spend 
a boatload of money to build a route that cuts off some of the distance 
inefficiency and gets you closer to 4500 km optical distance with minimal slack 
coil, and maybe no regens, so you get a real-world performance of 46 ms. But 
there are no algo trading sites of importance in DC, and for everybody else 
there’s not enough money in the difference between 46 and 59 ms for someone to 
go invest in that type of deployment. 

Dave Cohen
craetd...@gmail.com

> On Jun 20, 2020, at 12:44 PM, Tim Durack  wrote:
> 
> 
> And of course in your more realistic example:
> 
> 2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
> 
>> On Sat, Jun 20, 2020 at 12:36 PM Tim Durack  wrote:
>> Speed of light in glass ~200 km/s
>> 
>> 100 km rtt = 1ms
>> 
>> Coast-to-coast ~6000 km ~60ms
>> 
>> Tim:>
>> 
>>> On Sat, Jun 20, 2020 at 12:27 PM William Herrin  wrote:
>>> Howdy,
>>> 
>>> Why is latency between the east and west coasts so bad? Speed of light
>>> accounts for about 15ms each direction for a 30ms round trip. Where
>>> does the other 30ms come from and why haven't we gotten rid of it?
>>> 
>>> c = 186,282 miles/second
>>> 2742 miles from Seattle to Washington DC mainly driving I-90
>>> 
>>> 2742/186282 ~= 0.015 seconds
>>> 
>>> Thanks,
>>> Bill Herrin
>>> 
>>> -- 
>>> William Herrin
>>> b...@herrin.us
>>> https://bill.herrin.us/
>> 
>> 
>> -- 
>> Tim:>
> 
> 
> -- 
> Tim:>

Re: 60 ms cross-continent

[Joel Jaeggli]

Sent from my iPhone

> On Jun 20, 2020, at 9:27 AM, William Herrin  wrote:
> 
> Howdy,
> 
> Why is latency between the east and west coasts so bad? Speed of light
> accounts for about 15ms each direction for a 30ms round trip. Where
> does the other 30ms come from and why haven't we gotten rid of it?
> 
> c = 186,282 miles/second

This is c in a vacuum. Light transmission through a medium is slower. In the 
case of an optical fiber about 31% slower.

My lowest latency transit paths Palo Alto to the ashburn area are around 58ms.  
the great circle route for the two dcs involved is a distance 2408 miles which 
gives you a 39.6ms Lower bound.
 
The path isn’t quite a straight as that, but if you  eliminate the 6 routers in 
the path and count up the oeo regens I’m sure you can account most of the extra 
in the form of distance.

> 2742 miles from Seattle to Washington DC mainly driving I-90
> 
> 2742/186282 ~= 0.015 seconds
> 
> Thanks,
> Bill Herrin
> 
> -- 
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
> 

Re: 60 ms cross-continent

[Tim Požár]

Besides the refractive index of glass that makes like go about 2/3rds it 
can in a vacuum, "Stuff" also includes many other things like 
modulation/demodulation, buffers, etc.  I did a quora answer on this you 
can find at:


https://www.quora.com/How-can-one-describe-the-delay-characteristics-of-ping-and-traceroute-commands/answer/Tim-Pozar

On 6/20/20 9:29 AM, Joe Greco wrote:

On Sat, Jun 20, 2020 at 09:24:11AM -0700, William Herrin wrote:

Howdy,

Why is latency between the east and west coasts so bad? Speed of light
accounts for about 15ms each direction for a 30ms round trip. Where
does the other 30ms come from and why haven't we gotten rid of it?

c = 186,282 miles/second
2742 miles from Seattle to Washington DC mainly driving I-90

2742/186282 ~= 0.015 seconds


Speed of light in a fiber is more like 124K miles per second.  It
depends on the refractive index.  And of course amplifiers and stuff.

... JG

Re: 60 ms cross-continent

[Mike Hammett]

The speed of light in fiber is only about 2/3 the speed of light in a vacuum, 
so that 15 ms is really about 22.5 ms. That brings the total to about 45 ms. 


Some would come from how many miles of extra glass in that 2,742 miles in the 
form of slack loops. 


Some would come from fiber routes not being straight lines. Allied Fiber's 
formerly planned route from the Westin Building to Equinix Ashburn was about 
4,464 miles. That's about 38% longer than your 2,742 miles. Add that 38% to the 
previous 45 ms and you're at 62.1 ms. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message -

From: "William Herrin"  
To: nanog@nanog.org 
Sent: Saturday, June 20, 2020 11:24:11 AM 
Subject: 60 ms cross-continent 

Howdy, 

Why is latency between the east and west coasts so bad? Speed of light 
accounts for about 15ms each direction for a 30ms round trip. Where 
does the other 30ms come from and why haven't we gotten rid of it? 

c = 186,282 miles/second 
2742 miles from Seattle to Washington DC mainly driving I-90 

2742/186282 ~= 0.015 seconds 

Thanks, 
Bill Herrin 

-- 
William Herrin 
b...@herrin.us 
https://bill.herrin.us/ 

Re: 60 ms cross-continent

[Tim Durack]

And of course in your more realistic example:

2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path

On Sat, Jun 20, 2020 at 12:36 PM Tim Durack  wrote:

> Speed of light in glass ~200 km/s
>
> 100 km rtt = 1ms
>
> Coast-to-coast ~6000 km ~60ms
>
> Tim:>
>
> On Sat, Jun 20, 2020 at 12:27 PM William Herrin  wrote:
>
>> Howdy,
>>
>> Why is latency between the east and west coasts so bad? Speed of light
>> accounts for about 15ms each direction for a 30ms round trip. Where
>> does the other 30ms come from and why haven't we gotten rid of it?
>>
>> c = 186,282 miles/second
>> 2742 miles from Seattle to Washington DC mainly driving I-90
>>
>> 2742/186282 ~= 0.015 seconds
>>
>> Thanks,
>> Bill Herrin
>>
>> --
>> William Herrin
>> b...@herrin.us
>> https://bill.herrin.us/
>>
>
>
> --
> Tim:>
>


-- 
Tim:>

Re: 60 ms cross-continent

[Tim Durack]

Speed of light in glass ~200 km/s

100 km rtt = 1ms

Coast-to-coast ~6000 km ~60ms

Tim:>

On Sat, Jun 20, 2020 at 12:27 PM William Herrin  wrote:

> Howdy,
>
> Why is latency between the east and west coasts so bad? Speed of light
> accounts for about 15ms each direction for a 30ms round trip. Where
> does the other 30ms come from and why haven't we gotten rid of it?
>
> c = 186,282 miles/second
> 2742 miles from Seattle to Washington DC mainly driving I-90
>
> 2742/186282 ~= 0.015 seconds
>
> Thanks,
> Bill Herrin
>
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
>


-- 
Tim:>

Re: 60 ms cross-continent

[Joe Greco]

On Sat, Jun 20, 2020 at 09:24:11AM -0700, William Herrin wrote:
> Howdy,
> 
> Why is latency between the east and west coasts so bad? Speed of light
> accounts for about 15ms each direction for a 30ms round trip. Where
> does the other 30ms come from and why haven't we gotten rid of it?
> 
> c = 186,282 miles/second
> 2742 miles from Seattle to Washington DC mainly driving I-90
> 
> 2742/186282 ~= 0.015 seconds

Speed of light in a fiber is more like 124K miles per second.  It 
depends on the refractive index.  And of course amplifiers and stuff.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"The strain of anti-intellectualism has been a constant thread winding its way
through our political and cultural life, nurtured by the false notion that
democracy means that 'my ignorance is just as good as your knowledge.'"-Asimov

60 ms cross-continent

[William Herrin]

Howdy,

Why is latency between the east and west coasts so bad? Speed of light
accounts for about 15ms each direction for a 30ms round trip. Where
does the other 30ms come from and why haven't we gotten rid of it?

c = 186,282 miles/second
2742 miles from Seattle to Washington DC mainly driving I-90

2742/186282 ~= 0.015 seconds

Thanks,
Bill Herrin

-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: why am i in this handbasket? (was Devil's Advocate - Segment Routing, Why?)

[Robert Raszuk]

> The problem of MPLS, however, is that, it must also be flow driven,
> because detailed route information at the destination is necessary
> to prepare nested labels at the source, which costs a lot and should
> be attempted only for detected flows.
>

MPLS is not flow driven. I sent some mail about it but perhaps it bounced.

MPLS LDP or L3VPNs was NEVER flow driven.

Since day one till today it was and still is purely destination based.

Transport is using LSP to egress PE (dst IP).

L3VPNs are using either per dst prefix, or per CE or per VRF labels. No
implementation does anything upon "flow detection" - to prepare any nested
labels. Even in FIBs all information is preprogrammed in hierarchical
fashion well before any flow packet arrives.

Thx,
R.




>
>  > there is the argument that switching MPLS is faster than IP; when the
>  > pressure points i see are more at routing (BGP/LDP/RSVP/whatever),
>  > recovery, and convergence.
>
> Routing table at IPv4 backbone today needs at most 16M entries to be
> looked up by simple SRAM, which is as fast as MPLS look up, which is
> one of a reason why we should obsolete IPv6.
>
> Though resource reserved flows need their own routing table entries,
> they should be charged proportional to duration of the reservation,
> which can scale to afford the cost to have the entries.
>
> Masataka Ohta
>
>

Re: why am i in this handbasket? (was Devil's Advocate - Segment Routing, Why?)

[Robert Raszuk]

> there is saku's point of distributing labels in IGP TLVs/LSAs.  i
> suspect he is correct, but good luck getting that anywhere in the
> internet vendor task force.

Perhaps I will surprise a few but this is not only already in RFC formats -
it is also shipping already across vendors for some time now.

SR-MPLS (as part of its spec) does exactly that. You do not need to use any
SR if you do not want, you still can encapsulate your packets with
transport label corresponding to your exit at any ingress and forget about
LDP for good.

But with that let's not forget that aggregation here is still not spec-ed
out well and to the best of my knowledge it is also not shipping yet. I
recently proposed an idea how to aggregate SRGBs .. one vendor is analyzing
it.

Best,
R.



On Sat, Jun 20, 2020 at 1:33 AM Randy Bush  wrote:

> < ranting of a curmudgeonly old privileged white male >
>
> >>> MPLS was since day one proposed as enabler for services originally
> >>> L3VPNs and RSVP-TE.
> >> MPLS day one was mike o'dell wanting to move his city/city traffic
> >> matrix from ATM to tag switching and open cascade's hold on tags.
> > And IIRC, Tag switching day one was Cisco overreacting to Ipsilon.
>
> i had not thought of it as overreacting; more embrace and devour.  mo
> and yakov, aided and abetted by sob and other ietf illuminati, helped
> cisco take the ball away from Ipsilon, Force10, ...
>
> but that is water over the damn, and my head is hurting a bit from
> thinking on too many levels at once.
>
> there is saku's point of distributing labels in IGP TLVs/LSAs.  i
> suspect he is correct, but good luck getting that anywhere in the
> internet vendor task force.  and that tells us a lot about whether we
> can actually effect useful simplification and change.
>
> is a significant part of the perception that there is a forwarding
> problem the result of the vendors, 25 years later, still not
> designing for v4/v6 parity?
>
> there is the argument that switching MPLS is faster than IP; when the
> pressure points i see are more at routing (BGP/LDP/RSVP/whatever),
> recovery, and convergence.
>
> did we really learn so little from IP routing that we need to
> recreate analogous complexity and fragility in the MPLS control
> plane?  ( sound of steam eminating from saku's ears :)
>
> and then there is buffering; which seems more serious than simple
> forwarding rate.  get it there faster so it can wait in a queue?  my
> principal impression of the Stanford/Google workshops was the parable
> of the blind men and the elephant.  though maybe Matt had the main
> point: given scaling 4x, Moore's law can not save us and it will all
> become paced protocols.  will we now have a decade+ of BBR evolution
> and tuning?  if so, how do we engineer our networks for that?
>
> and up 10,000m, we watch vendor software engineers hand crafting in
> an assembler language with if/then/case/for, and running a chain of
> checking software to look for horrors in their assembler programs.
> it's the bleeping 21st century.  why are the protocol specs not
> formal and verified, and the code formally generated and verified?
> and don't give me too slow given that the hardware folk seem to be
> able to do 10x in the time it takes to run valgrind a few dozen
> times.
>
> we're extracting ore with hammers and chisels, and then hammering it
> into shiny objects rather than safe and securable network design and
> construction tools.
>
> apologies.  i hope you did not read this far.
>
> randy
>

Re: why am i in this handbasket? (was Devil's Advocate - Segment Routing, Why?)

[Masataka Ohta]

Randy Bush wrote:


MPLS was since day one proposed as enabler for services originally
L3VPNs and RSVP-TE.

MPLS day one was mike o'dell wanting to move his city/city traffic
matrix from ATM to tag switching and open cascade's hold on tags.

And IIRC, Tag switching day one was Cisco overreacting to Ipsilon.


i had not thought of it as overreacting; more embrace and devour.  mo
and yakov, aided and abetted by sob and other ietf illuminati, helped
cisco take the ball away from Ipsilon, Force10, ...


Ipsilon was hopeless because, as Yakov correctly pointed out, flow
driven approach to automatically detect flows does not scale.

The problem of MPLS, however, is that, it must also be flow driven,
because detailed route information at the destination is necessary
to prepare nested labels at the source, which costs a lot and should
be attempted only for detected flows.

> there is the argument that switching MPLS is faster than IP; when the
> pressure points i see are more at routing (BGP/LDP/RSVP/whatever),
> recovery, and convergence.

Routing table at IPv4 backbone today needs at most 16M entries to be
looked up by simple SRAM, which is as fast as MPLS look up, which is
one of a reason why we should obsolete IPv6.

Though resource reserved flows need their own routing table entries,
they should be charged proportional to duration of the reservation,
which can scale to afford the cost to have the entries.

Masataka Ohta

Re: Devil's Advocate - Segment Routing, Why?

[Masataka Ohta]

Mark Tinka wrote:


At the time I proposed label switching, there already was RSVP
but RSVP-TE was proposed long after MPLS was proposed.


RSVP failed to take off, for whatever reason (I can think of many).


There are many. So, our research group tried to improve RSVP.

Practically, the most serious problem of RSVP is, like OSPF, using
unreliable link multicast to reliably exchange signalling messages
between routers, making specification and implementations very
complicated.

So, we developed SRSVP (Simple RSVP) replacing link multicast by,
like BGP, link local TCP mesh (thanks to the CATENET model, unlike
BGP, there is no scalability concern). Then, it was not so difficult
to remove other problems.

However, perhaps, most people think show stopper to RSVP is lack
of scalability of weighted fair queueing, though, it is not a
problem specific to RSVP and MPLS shares the same problem.

Obviously, weighted fair queueing does not scale because it is
based on deterministic traffic model of token bucket model
and, these days, people just use some ad-hoc ways for BW
guarantee implicitly assuming stochastic traffic model. I
even developed a little formal theory on scalable queueing
with stochastic traffic model.

So, we have specification and working implementation of
hop-by-hop, scalable, stable unicast/multicast interdomain
QoS routing protocol supporting routing hierarchy without
clank back.

See

http://www.isoc.org/inet2000/cdproceedings/1c/1c_1.htm

for rough description of design guideline.


I'm not sure any network operator, today, would allow an end-host to
make reservation requests in their core.


I didn't attempt to standardize our result in IETF, partly
because optical packet switching was a lot more interesting.


Even in the Transport world, this was the whole point of GMPLS. After
they saw how terrible that idea was, it shifted from customers to being
an internal fight between the IP teams and the Transport teams.
Ultimately, I don't think anybody really cared about routers
automatically using GMPLS to reserve and direct the DWDM network.


That should be a reasonable way of practical operation, though I'm
not very interested in OCS (optical circuit switching) of GMPLS


In our Transport network, we use GMPLS/ASON in the Transport network
only. When the IP team needs capacity, it's a telephone job :-).


For IP layer, that should be enough. For ASON, so complicated
GMPLS is actually overkill.

When I was playing with ATM switches, I established control
plain network with VPI/VCI=0/0 and assign control plain IP
addresses to ATM switches. To control other VCs, simple UDP
packets are sent to switches from controlling hosts.

Similar technology should be applicable to ASON. Maintaining
integrity between wavelength switches is responsibility
of controllers.


Remember that the original point of MPLS was that it should work
scalably without a lot of configuration, which is not the reality
recognized by people on this thread.


Well, you get the choice of LDP (low-touch) or RSVP-TE (high-touch).


No, I just explained what was advertised to be MPLS by people
around Cisco against Ipsilon.

According to the advertisements, you should call what you
are using LS or GLS, not MPLS or GMPLS.


We don't use RSVP-TE because of the issugaes you describe above.

We use LDP to avoid the issues you describe above.


Good.


In the end, SR-MPLS is meant to solve this issue for TE requirements. So
the signaling state-of-the-art improves with time.

Assuming a central controller (and its collocated or distributed
back up controllers), we don't need complicated protocols in
the network to maintain integrity of the entire network.


That is certainly a problem. However, worse problem is to know
label values nested deeply in MPLS label chain.


Why, how, is that a problem? For load balancing?

What if, an inner label becomes invalidated around the
destination, which is hidden, for route scalability,
from the equipments around the source?


Even worse, if route near the destination expected to pop the label
chain goes down, how can the source knows that the router goes down
and choose alternative router near the destination?


If by source you mean end-host, if the edge router they are connected to
only ran IP and they were single-homed, they'd still go down.


No, as "the destination expected to pop the label" is located somewhere
around the final destination end-host.

If, at the destination site, connectivity between a router to pop nested
label and the fine destination end-host is lost, we are at a loss,
unless source side changes inner label.


MPLS with hierarchical routing just does not scale.


With Internet in a VRF, I truly agree.

But if you run a simple global BGP table and no VRF's, I don't see an
issue. This is what we do, and our scaling concerns are exactly the same
whether we run plain IP or IP/MPLS.


If you are using intra-domain hierarchical routing for
scalability within the domain, you 

Re: Devil's Advocate - Segment Routing, Why?

[Mark Tinka]

On 20/Jun/20 11:27, Baldur Norddahl wrote:

>
>
> We run the Internet in a VRF to get watertight separation between
> management and the Internet. I do also have a CGN vrf but that one has
> very few routes in it (99% being subscriber management created, eg.
> one route per customer). Why would this create a scaling issue? If you
> collapse our three routing tables into one, you would have exactly the
> same number of routes. All we did was separate the routes into
> namespaces, to establish a firewall that prevents traffic to flow
> where it shouldn't.

It may be less of an issue in 2020 with the current control planes and
how far the code has come, but in the early days of l3vpn's, the number
of VRF's you could have was directly proportional to the number of
routes you had in each one. More VRF's, less routes for each. More
routes per VRF, less VRF's in total.

I don't know if that's still an issue today, as we don't run the
Internet in a VRF. I'd defer to those with that experience, who knew
about the scaling limitations of the past.

Mark.

Re: Devil's Advocate - Segment Routing, Why?

[Baldur Norddahl]

On Sat, Jun 20, 2020 at 11:08 AM Mark Tinka  wrote:

> > MPLS with hierarchical routing just does not scale.
>
> With Internet in a VRF, I truly agree.
>
> But if you run a simple global BGP table and no VRF's, I don't see an
> issue. This is what we do, and our scaling concerns are exactly the same
> whether we run plain IP or IP/MPLS.
>
> Mark.
>
>
We run the Internet in a VRF to get watertight separation between
management and the Internet. I do also have a CGN vrf but that one has very
few routes in it (99% being subscriber management created, eg. one route
per customer). Why would this create a scaling issue? If you collapse our
three routing tables into one, you would have exactly the same number of
routes. All we did was separate the routes into namespaces, to establish a
firewall that prevents traffic to flow where it shouldn't.

Regards,

Baldur

Re: Devil's Advocate - Segment Routing, Why?

[Mark Tinka]

On 19/Jun/20 18:00, Masataka Ohta wrote:
 
> There seems to be serious confusions between label switching
> with explicit flows and MPLS, which was believed to scale
> without detecting/configuring flows.
>
> At the time I proposed label switching, there already was RSVP
> but RSVP-TE was proposed long after MPLS was proposed.

RSVP failed to take off, for whatever reason (I can think of many).

I'm not sure any network operator, today, would allow an end-host to
make reservation requests in their core.

Even in the Transport world, this was the whole point of GMPLS. After
they saw how terrible that idea was, it shifted from customers to being
an internal fight between the IP teams and the Transport teams.
Ultimately, I don't think anybody really cared about routers
automatically using GMPLS to reserve and direct the DWDM network.

In our Transport network, we use GMPLS/ASON in the Transport network
only. When the IP team needs capacity, it's a telephone job :-).


>
> But, today, people are seems to be using, so called, MPLS, with
> explicitly configured flows, administration of which does not
> scale and is annoying.
>
> Remember that the original point of MPLS was that it should work
> scalably without a lot of configuration, which is not the reality
> recognized by people on this thread.

Well, you get the choice of LDP (low-touch) or RSVP-TE (high-touch).

Pick your poison.

We don't use RSVP-TE because of the issues you describe above.

We use LDP to avoid the issues you describe above.

In the end, SR-MPLS is meant to solve this issue for TE requirements. So
the signaling state-of-the-art improves with time.


> That is certainly a problem. However, worse problem is to know
> label values nested deeply in MPLS label chain.

Why, how, is that a problem? For load balancing?


>
> Even worse, if route near the destination expected to pop the label
> chain goes down, how can the source knows that the router goes down
> and choose alternative router near the destination?

If by source you mean end-host, if the edge router they are connected to
only ran IP and they were single-homed, they'd still go down.

If the end-host were multi-homed to two edge routers, one of them
failing won't cause an outage for the host.

Unless I misunderstand.


> MPLS with hierarchical routing just does not scale.

With Internet in a VRF, I truly agree.

But if you run a simple global BGP table and no VRF's, I don't see an
issue. This is what we do, and our scaling concerns are exactly the same
whether we run plain IP or IP/MPLS.

Mark.

Re: Devil's Advocate - Segment Routing, Why?

[Mark Tinka]

On 19/Jun/20 17:40, Masataka Ohta wrote:
 
>
> As the first person to have proposed the forwarding paradigm of
> label switching, I have been fully aware from the beginning that:
>
>    https://tools.ietf.org/html/draft-ohta-ip-over-atm-01
>
>    Conventional Communication over ATM in a Internetwork Layer
>
>    The conventional communication, that is communication that does not
>    assume connectivity, is no different from that of the existing IP, of
>    course.
>
> special, prioritized forwarding should be done only by special
> request by end users (by properly designed signaling mechanism, for
> which RSVP failed to be) or administration does not scale.

I could be wrong, but I get the feeling that you are speaking about RSVP
in its original form, where hosts were meant to make calls (CAC) into
the network to reserve resources on their behalf.

As we all know, that never took off, even though I saw some ideas about
it being proposed for mobile phones as well.

I don't think there ever was another attempt to get hosts to reserve
resources within the network, since the RSVP failure.



>
> Not. Even without MPLS, fine tuning of BGP does not scale.

We all know this, and like I said, that is a current concern.


>
> However, just as using plain IP router costs less than using
> MPLS capable IP routers, BGP-only administration costs less than
> BGP and MPLS administration.
>
> For better networking infrastructure, extra cost should be spent
> for L1, not MPLS or very complicated technologies around it.

In the early 2000's, I would have agreed with that.

Nowadays, there is a very good chance that a box you require a BGP DFZ
on inherently supports MPLS, likely without extra licensing.

Mark.