Re: [c-nsp] LDPv6 Census Check

[Saku Ytti]

On Fri, 12 Jun 2020 at 23:25, David Sinn  wrote:

> > Should we design a rational cost-efficient solution, we should choose
> > the lowest overhead and narrowest working keys.
>
> In the abstract, sure. But if you want a practical, deployable, production 
> network, it's multi-dimensioned.

We have probably largely converged to the same place. Your vantage
point sees practical offerings where IPIP may make more sense to you
than MPLS, my vantage point definitely only implements the rich
features I need in MPLS tunnels (RSVP-TE, L2 pseudowires, FRR, L3 MPLS
VPN, all of which technically could be done of course with IPIPIP
tunnels) And the theory we agree that less is more.

ECMP appears to be your main pain point, the rich features are not
relevant, and you mentioned commodity hardware being able to hash on
IPIP. I feel this may be a very special case where HW can do IPIP hash
but not MPLSIP hash. Out of curiosity, what is this hardware? Jericho
can do MPLSIP, I know JNPR's pipeline offering, Paradise, can.  Or
perhaps it's not even that the underlaying hardware you have, cannot
do, it's that the NOS you are being offered is so focused on your
use-case, it doesn't do anything else reasonably, then of course that
use-case is best by default.

-- 
  ++ytti

Re: [c-nsp] LDPv6 Census Check

[Randy Bush]

< note that i am wearing arrcus hat >

> I'll be honest, none of our customers ever asked us to deploy RPKI and
> ROV. Will they benefit from it, sure? Is it about to become part of an
> RFP requirements document? Probably not.

we have rov in rfps received from paying customers

randy

Re: ARIN

[John Sweeting]

Thank you for taking time out to recognize the great team in RSD Mehmet! The 
fact is they go above and beyond each and every day and it's great to see that 
pointed out. Lisa and her team are amazing. Thank you again.

Sent from my iPhone

On Jun 12, 2020, at 6:45 PM, Mehmet Akcin  wrote:

?
hey there,

I just wanted to share my experience dealing with ARIN support this week. I 
think it's not very common to see people taking the time to write something 
like this but I personally think we should do more often.

It has been long time since I had to deal with RIRs and this week I had to do 
several things in ARIN. The support has team was very quick responding, very 
useful with their recommendations to my questions, and had a great attitude 
towards solving problems. I can certainly say they went above and beyond when I 
opened a ticket with wrong request type and they asked me if they can close it 
and open a new one for me? I mean.. this is called going Above and Beyond!

thank you all ARIN support desk and especially Lisa Liedel for this great 
experience. and @John Curran please accept this 
sincere thanks on your team's behalf!

Mehmet

Re: ARIN

[John Curran]

Mehmet -

I shall pass along your praise to the team that does all the real work - and I 
very glad we could help out!

/John

John Curran
President and CEO
American Registry for Internet Numbers

On Jun 12, 2020, at 6:56 PM, Mehmet Akcin  wrote:


hey there,

I just wanted to share my experience dealing with ARIN support this week. I 
think it's not very common to see people taking the time to write something 
like this but I personally think we should do more often.

It has been long time since I had to deal with RIRs and this week I had to do 
several things in ARIN. The support has team was very quick responding, very 
useful with their recommendations to my questions, and had a great attitude 
towards solving problems. I can certainly say they went above and beyond when I 
opened a ticket with wrong request type and they asked me if they can close it 
and open a new one for me? I mean.. this is called going Above and Beyond!

thank you all ARIN support desk and especially Lisa Liedel for this great 
experience. and @John Curran please accept this 
sincere thanks on your team's behalf!

Mehmet

ARIN

[Mehmet Akcin]

hey there,

I just wanted to share my experience dealing with ARIN support this week. I
think it's not very common to see people taking the time to write something
like this but I personally think we should do more often.

It has been long time since I had to deal with RIRs and this week I had to
do several things in ARIN. The support has team was very quick responding,
very useful with their recommendations to my questions, and had a great
attitude towards solving problems. I can certainly say they went above and
beyond when I opened a ticket with wrong request type and they asked me if
they can close it and open a new one for me? I mean.. this is called going
Above and Beyond!

thank you all ARIN support desk and especially Lisa Liedel for this great
experience. and @John Curran  please accept this sincere
thanks on your team's behalf!

Mehmet

Re: [c-nsp] LDPv6 Census Check

[Mark Tinka]

On 12/Jun/20 17:19, David Sinn wrote:

> Yes. Path enumeration when you use mult-tier Clos topologies within a PoP 
> causes you many, many problem.

Okay, got you. I thought you were running into these problems on the
"usual suspect" platforms.

Yes, commodity hardware certainly has a number of trade-offs for the
cost benefit. We've been evaluating this path since 2013, and for our
use-case, it will actually make less sense because we are more large
capacity, small footprint, i.e., typical network service provider.

If we were a cloud provider operating at scale in several data centres,
our current model of running Cisco's, Juniper's, Arista's, e.t.c., may
not necessarily be the right choice in 2020, particularly in the edge.

Mark.

Weekly Routing Table Report

[Routing Analysis Role Account]

This is an automated weekly mailing describing the state of the Internet
Routing Table as seen from APNIC's router in Japan.

The posting is sent to APOPS, NANOG, AfNOG, SANOG, PacNOG, SAFNOG
TZNOG, MENOG, BJNOG, SDNOG, CMNOG, LACNOG and the RIPE Routing WG.

Daily listings are sent to bgp-st...@lists.apnic.net

For historical data, please see http://thyme.rand.apnic.net.

If you have any comments please contact Philip Smith .

Routing Table Report   04:00 +10GMT Sat 13 Jun, 2020

Report Website: http://thyme.rand.apnic.net
Detailed Analysis:  http://thyme.rand.apnic.net/current/

Analysis Summary


BGP routing table entries examined:  815015
Prefixes after maximum aggregation (per Origin AS):  309321
Deaggregation factor:  2.63
Unique aggregates announced (without unneeded subnets):  397070
Total ASes present in the Internet Routing Table: 68301
Prefixes per ASN: 11.93
Origin-only ASes present in the Internet Routing Table:   58705
Origin ASes announcing only one prefix:   24458
Transit ASes present in the Internet Routing Table:9596
Transit-only ASes present in the Internet Routing Table:311
Average AS path length visible in the Internet Routing Table:   4.4
Max AS path length visible:  46
Max AS path prepend of ASN ( 22394)  38
Prefixes from unregistered ASNs in the Routing Table:   830
Number of instances of unregistered ASNs:   832
Number of 32-bit ASNs allocated by the RIRs:  32024
Number of 32-bit ASNs visible in the Routing Table:   26402
Prefixes from 32-bit ASNs in the Routing Table:  121590
Number of bogon 32-bit ASNs visible in the Routing Table:12
Special use prefixes present in the Routing Table:1
Prefixes being announced from unallocated address space:544
Number of addresses announced to Internet:   2843328512
Equivalent to 169 /8s, 121 /16s and 192 /24s
Percentage of available address space announced:   76.8
Percentage of allocated address space announced:   76.8
Percentage of available address space allocated:  100.0
Percentage of address space in use by end-sites:   99.5
Total number of prefixes smaller than registry allocations:  274923

APNIC Region Analysis Summary
-

Prefixes being announced by APNIC Region ASes:   214406
Total APNIC prefixes after maximum aggregation:   62465
APNIC Deaggregation factor:3.43
Prefixes being announced from the APNIC address blocks:  208016
Unique aggregates announced from the APNIC address blocks:86032
APNIC Region origin ASes present in the Internet Routing Table:   10615
APNIC Prefixes per ASN:   19.60
APNIC Region origin ASes announcing only one prefix:   2963
APNIC Region transit ASes present in the Internet Routing Table:   1582
Average APNIC Region AS path length visible:4.7
Max APNIC Region AS path length visible: 27
Number of APNIC region 32-bit ASNs visible in the Routing Table:   5695
Number of APNIC addresses announced to Internet:  769451904
Equivalent to 45 /8s, 220 /16s and 231 /24s
APNIC AS Blocks4608-4864, 7467-7722, 9216-10239, 17408-18431
(pre-ERX allocations)  23552-24575, 37888-38911, 45056-46079, 55296-56319,
   58368-59391, 63488-64098, 64297-64395, 131072-141625
APNIC Address Blocks 1/8,  14/8,  27/8,  36/8,  39/8,  42/8,  43/8,
49/8,  58/8,  59/8,  60/8,  61/8, 101/8, 103/8,
   106/8, 110/8, 111/8, 112/8, 113/8, 114/8, 115/8,
   116/8, 117/8, 118/8, 119/8, 120/8, 121/8, 122/8,
   123/8, 124/8, 125/8, 126/8, 133/8, 150/8, 153/8,
   163/8, 171/8, 175/8, 180/8, 182/8, 183/8, 202/8,
   203/8, 210/8, 211/8, 218/8, 219/8, 220/8, 221/8,
   222/8, 223/8,

ARIN Region Analysis Summary


Prefixes being announced by ARIN Region ASes:238331
Total ARIN prefixes after maximum aggregation:   109319
ARIN Deaggregation factor: 2.18
Prefixes being announced from the ARIN address blocks:   235619
Unique aggregates announced from the ARIN address blocks:119621
ARIN Region origin ASes present in the Internet Routing Table:18513
ARIN Prefixes per ASN:12.73
ARIN Reg

Re: [c-nsp] LDPv6 Census Check

[Saku Ytti]

On Fri, 12 Jun 2020 at 20:12, Andrey Kostin  wrote:

> Sorry for jumping in in the mddle of discussion, as a side note, in case
> of IPIP tunneling, shouldn't another protocol type be utilized in MAC
> header? As I understand, in VXLAN VTEP ip is dedicated for this purpose,
> so receiving a packet with VTEP DST IP already means "decapsulate and
> lookup the next header". But in traditional routers loopback IPs are
> used for multiple purposes and usually receiving a packet with lo0 IP
> means punt it to control plane. Isn't additional differentiator is
> needed here to tell a router which type of action it has to do? Or, as
> alternative, if dedicated stack of IPs is used for tunneling, then
> another lookup table is needed for it, isn't it? And now looks like we
> are coming to the header structure and forwarding process similar that
> we already have in MPLS, only with different label format. Please
> correct me if I went off track somewhere in this logical chain.

I don't think new etherType is mandatory by any means. Biggest gain is
security. SRv6 will necessarily have a lot of issues where
unauthorised packet gets treated as SRv6, which is much harder in MPLS
network. Many real-life devices work very differently with EH chains
(with massive performance drop, like can be 90%!). JNPR Trio will
parse up-to N EH, then drop if it cannot finish parsing. NOK FP wil
parse up-to N EH, then forward if it cannot finish parsing (i.e. now
it bypasses TCP/UDP denies, as it didn't know it is TCP/IP, or it
could have SRv6 EH, which it couldn't drop, as it didn't know it had
it).

But in terms of the big MPLS advantage, of having guaranteed exact
match lookups on small space, compared to LPM lookups on large space.
We could guarantee this in IPIP tunnels too, without having any
difference in the headers, other than obligation/guarantee that all
LSR packets are IPIP encapsulated with a small amount of outer packet
DADDRs.

-- 
  ++ytti

Re: [c-nsp] LDPv6 Census Check

[Andrey Kostin]

Saku Ytti писал 2020-06-12 12:10:

On Fri, 12 Jun 2020 at 18:52, David Sinn  wrote:

Unless you want ECMP then it VERY much matters. But I guess since we 
are only talking about theoretical instead of building an actual 
practical network, it doesn't matter.


Well blatantly we are, because in the real world most of the value of
MPLS tunnels is not available as IP tunnels. Again technically
entirely possible to replace MPLS tunnels with IP tunnels, just
question how much overhead you have in transporting the tunnel key and
how wide they are.

Should we design a rational cost-efficient solution, we should choose
the lowest overhead and narrowest working keys.


Sorry for jumping in in the mddle of discussion, as a side note, in case 
of IPIP tunneling, shouldn't another protocol type be utilized in MAC 
header? As I understand, in VXLAN VTEP ip is dedicated for this purpose, 
so receiving a packet with VTEP DST IP already means "decapsulate and 
lookup the next header". But in traditional routers loopback IPs are 
used for multiple purposes and usually receiving a packet with lo0 IP 
means punt it to control plane. Isn't additional differentiator is 
needed here to tell a router which type of action it has to do? Or, as 
alternative, if dedicated stack of IPs is used for tunneling, then 
another lookup table is needed for it, isn't it? And now looks like we 
are coming to the header structure and forwarding process similar that 
we already have in MPLS, only with different label format. Please  
correct me if I went off track somewhere in this logical chain.


To David's point about ECMP I'd like to mention that in WAN networks 
number of diverse paths is always limited, so having multiple links 
taking same path doesn't make much sense. With current economics 4x10G 
and 1x100G are usually close from price POV. Obviously, there are 
different situations when multiple links are the only option, but how 
many, usually 4-8. Sure if you need multiple 400G then there is 
currently no option to go to higher speeds, but that's more DC use case 
than WAN network. So ECMP in WAN network isn't that big scale problem 
imho, also there are existing and proposed solutions, like SR, for it.


Kind regards,
Andrey

Re: [c-nsp] LDPv6 Census Check

[Saku Ytti]

On Fri, 12 Jun 2020 at 18:52, David Sinn  wrote:

> Unless you want ECMP then it VERY much matters. But I guess since we are only 
> talking about theoretical instead of building an actual practical network, it 
> doesn't matter.

Well blatantly we are, because in the real world most of the value of
MPLS tunnels is not available as IP tunnels. Again technically
entirely possible to replace MPLS tunnels with IP tunnels, just
question how much overhead you have in transporting the tunnel key and
how wide they are.

Should we design a rational cost-efficient solution, we should choose
the lowest overhead and narrowest working keys.

-- 
  ++ytti

Re: [c-nsp] LDPv6 Census Check

[Christian Meutes]

Salve,

On Thu, Jun 11, 2020 at 8:08 PM David Sinn  wrote:

Rewrites on MPLS is horrible from a memory perspective as maintaining the
> state and label transition to explore all possible discrete paths across
> the overall end-to-end path you are trying to take is hugely in-efficient.
> Applying circuit switching to a packet network was bad from the start. SR
> doesn't resolve that, as you are stuck with a global label problem and the
> associated lack of being able to engineer your paths, or a label stack
> problem on ingress that means you need a massive ASIC's and memories there.
>

I don't think rewrites are horrible, but just very flexible and this *can*
come up with a certain price. Irt to your memory argument that path
engineering takes in vanilla TE a lot of forwarding slots we should remind
us that this is not a design principle of MPLS. Discrete paths could also
be signalled in MPLS with shared link-labels so that you will end up with
the same big instructional headend packet as in SR. There are even
implementations offering this.

IP at least gives you rewrite sharing, so in a lite-core you have way
> better trade-off on resources, especially in a heavily ECMP'ed network.
> Such as one build of massive number of open small boxes vs. a small number
> of huge opaque ones. Pick your poison but saying one is inheriantly better
> then another in all cases is just plane false.
>

If I understand this argument correctly then it shouldn't be one because of
"rewrite sharing" being irrelevant for the addressability of single nodes
in a BGP network. Why a header lookup depth of 4B per label in engineered
and non-engineered paths should be a bad requisite for h/w designers of
modern networks is beyond me. In most MPLS networks (unengineered L3VPN)
you need to read less of headers than in a eg. VXLAN fabric to make ECMP
work (24B vs. 20B).

Re: [c-nsp] LDPv6 Census Check

[Saku Ytti]

On Fri, 12 Jun 2020 at 18:42, David Sinn  wrote:

> But why do you need full table lookup in the middle of the network? Why place 
> that class of gear where it's not needed?

Some people do collapsed core. But this is getting bit theoretical,
because we definitely could do this on IP, we could do some lookups on
on-chip and some off-chip to do both, should market want it.

> The label stack question is about the comparisons between the two extremes of 
> SR that you can be in. You either label your packet just for it's ultimate 
> destination or you apply the stack of the points you want to pass through.

Quite, but transit won't inspect the stack, it doesn't have to care
about it, so it can be very deep.

> In the former case you are, at the forwarding plane, equal to what you see 
> with traditional MPLS today, with every node along the path needing to know 
> how to reach the end-point. Yes, you have lowered label space from 
> traditional MPLS, but that can be done with site-cast labels already. And, 
> while the nodes don't have to actually swap labels, when you look at 
> commodity implementations (across the last three generations since you want 
> to do this with what is deployed, not wholesale replace the network) a null 
> swap still ends up eating a unique egress next-hop entry. So from a hardware 
> perspective, you haven't improved anything. Your ECMP group count is high.

I don't disagree. What I'm trying to say, however you tunnel, you have
the same issues. If you need to tunnel, then MPLS is better tunnel
than IP. Ultimately both can be made LEM on-chip should market want
it, so difference left is what is the overhead of the tunnel, and MPLS
wins here handsdown. This is objectively true, now what practical
market reality is, that may be different, because market doesn't
optimise for best solution.

> So, yes, MPLS works fine if you want to buy big iron boxes. But that come at 
> a cost. So the point about MPLS is always better is not accurate. Engineering 
> is about trade-offs and there are trade-offs to be made when you optimize in 
> a different direction and that points away from MPLS and back to IPIP

Always if you need tunnel. Because the fundamental question is how
much overhead we have, and what is our key width. In both IP tunnel
and MPLS tunnel cases we will assume LEM lookup, to keep the lookup
cheap.

-- 
  ++ytti

Re: IPv4 Broker / Service -

[Bryan Holloway]

+1

On 6/11/20 9:38 PM, b...@theworld.com wrote:


Addrex.net

I know some of the principles personally and would vouch for them.


On June 11, 2020 at 14:27 edwin.malle...@gmail.com (edwin.malle...@gmail.com) 
wrote:
  > Hi Nanog,
  >
  >
  >
  > I have need of a reputable IPv4 broker or service  ? personal experience 
with
  > said broker would be preferred.  These would be for small blocks - /23, 24s 
?
  > in the US, so ARIN.  I know, I know, IPv6 for life and all that and I agree,
  > but ? you know, the business.  I?m happy to take responses off-list, but I
  > would really appreciate any recommendations.
  >
  >
  >
  > Thanks!
  >
  >
  >
  > Ed
  >

Re: [c-nsp] LDPv6 Census Check

[Mark Tinka]

On 11/Jun/20 19:19, Saku Ytti wrote:

> The demand is, we need tunneling, then the question is what are the
> metrics of a good tunneling solution. By answering this honestly, MPLS
> is better. We could do better surely, but IP is not that.

One unexpected benefit, I will say, with going native LDPv6 is that
MTR's for IPv6 destinations no longer report packet loss on the
intermediary core routers (CRS-X).

I know this was due to the control plane, and nothing to do with the
actual data plane, but it was always a tool explaining to customers why
MTR's for IPv4 destinations have 0% packet loss in our core, while IPv6
ones have 30% - 50% (in spite of the final end-host reporting 0% packet
loss).

Since going LDPv6, IPv6 traffic is now label-switched in the core, in
lieu of hop-by-hop IPv6 forwarding. The unforeseen-but-welcome side
effect is that customer packet loss MTR's for IPv6 destinations that
traverse the CRS-X core are as 0% as they are for IPv4 (even though we
haven't yet removed BGPv6 from the core due to IOS XE platforms that
don't yet run LDPv6).

One less trouble ticket to have to explain for our NOC; I'll gladly take
that...

As my Swedish friend would say, "That gives me an avenue of pleasure and
joy" :-).

Mark.

Re: [c-nsp] LDPv6 Census Check

[Saku Ytti]

On Fri, 12 Jun 2020 at 18:16, David Sinn  wrote:

> I'm not sure what implementation you are saying doesn't exist. The Broadcom 
> XGS line is all on-die. The two largest cloud providers are using them in 
> their transport network (to the best of my understanding). So I'm not sure if 
> your saying that no one is using small boxes like I'm describing or what. And 
> it doesn't have to be MPLS over IP. That is one option, but IPIP is another.

I'm saying implementation which has off-chip and supports putting some
on-chip. So that you could have full table lookup for edge packets and
and fast exact lookup for others. Of course we do have platforms which
do have large LEM tables off-chip.

> Again, feel free to look at only one small aspect and say that it is 
> completely better in all cases. MPLS is not better in wide ECMP cases, full 
> stop. SR doesn't help that when you actually look at the problems at massive 
> scale as I have done. You continually are on the trade-off spectrum of 
> irrationally deep label stacks or enumeration of all of the possible paths 
> through the network and burn all of your next-hop re-writes. At least if you 
> want high-radiux, single chip systems. So this is not sentimentally around a 
> protocol, it's the practical reality when you look at the problems at scale 
> using commodity components. So if you want to optimize for costs and power 
> (which is operational costs), MPLS is not where it is at.

I'm not sure why this deep label stack keeps popping, if we need
multiple levels of tunneling, we need it in IP too, and it's almost
more expensive in IP. Cases I can think of in SR, you'll only loop top
label or two, even if you might have 10 labels.
For every apples to apples cases MPLS tunnels are superior to IP
tunnels. If you want cheap very small FIB backbone, then all traffic
will need to be IP tunneled to egress, and you get all the MPLS
problems, and you get more overhead and larger keys (larger keys is
not a big deal).

Now if discussion is do we need tunnelling at all, the discussion is
very different.

-- 
  ++ytti

RE: Partial vs Full tables

[Brian Turnbow via NANOG]

Hi all,

> 
> Loose mode RPF will essentially drop traffic received on the interface if the
> router does not have any route for. (will not match a default or a discard
> route, at least in IOS-XR)
> 
> As Bill has pointed out, this may drop traffic from some peering networks that
> are not in the global routing table. Though one could argue that if a packet
> needs to be fragged it's typically closer to the edges rather than the
> transit/peering links.
> 

No one has mentioned it , but you can also use an acl combined with urpf.
You could even go so far as permiting everything and just using urpf for rtbh 
purposes.


Brian

Re: IPv4 Broker / Service -

[DIEGO ZORRILLA (diefierr) via NANOG]

I have good knowledge about best reputation of https://www.ipbroker.es/

Regards
Diego

On 11/06/20 14:40, "NANOG on behalf of b...@theworld.com" 
 wrote:


Addrex.net

I know some of the principles personally and would vouch for them.


On June 11, 2020 at 14:27 edwin.malle...@gmail.com 
(edwin.malle...@gmail.com) wrote:
 > Hi Nanog,
 > 
 >  
 > 
 > I have need of a reputable IPv4 broker or service  ? personal experience 
with
 > said broker would be preferred.  These would be for small blocks - /23, 
24s ?
 > in the US, so ARIN.  I know, I know, IPv6 for life and all that and I 
agree,
 > but ? you know, the business.  I?m happy to take responses off-list, but 
I
 > would really appreciate any recommendations.
 > 
 >  
 > 
 > Thanks!
 > 
 >  
 > 
 > Ed
 > 

-- 
-Barry Shein

Software Tool & Die| b...@theworld.com | 
http://www.TheWorld.com
Purveyors to the Trade | Voice: +1 617-STD-WRLD   | 800-THE-WRLD
The World: Since 1989  | A Public Information Utility | *oo*