Re: AWS WAF list

[Owen DeLong via NANOG]

Here’s the usual problem:

Victim is a customer Q  of ISP A.

WAF provided by provider X is chosen by website Y.

A has no business relationship with X or Y.

A’s requests to X are rebuffed because A is not a customer of X.
A’s requests to Y are rebuffed because A is not a customer of Y.

A tells Q to reach out to Y directly and attempts to explain the situation to Q.
Q does not understand half of the words coming out of A’s mouth and asks “can’t 
you just fix this?”

Sucks for everyone involved.

Sucks most for Q, second most for A because A often loses customer Q over the 
issue (where Q is often multiplied to several customers).

Sucks for Y (to a much lesser degree) because some fraction of customer Qs will 
be smart enough to be angry with Y.

OK, maybe it doesn’t really suck much for X, but they do get a bunch of calls 
they have to blow off, so it still costs them a little bit of money.

Owen


> On Feb 20, 2024, at 16:05, Tom Beecher  wrote:
> 
>> and it's affecting our customers' access to various ===>> websites.<<===
> 
> 
> On Tue, Feb 20, 2024 at 6:15 PM Pui Ee Luun Edylie  <mailto:em...@edylie.net>> wrote:
>> There must be a reason why the web site chooses the WAF list to block out 
>> the victim? If so why not the victim to contact the website to request them 
>> to talk to the waf list provider to remove victim ip block?
>> 
>>  
>> 
>> Edy
>> 
>>  
>> 
>> From: NANOG > <mailto:edylie@nanog.org>> On Behalf Of Owen DeLong via NANOG
>> Sent: Wednesday, 21 February 2024 7:04 am
>> To: j...@joelesler.net <mailto:j...@joelesler.net>
>> Cc: NANOG mailto:nanog@nanog.org>>
>> Subject: Re: AWS WAF list
>> 
>>  
>> 
>> Unfortunately, the victim doesn’t chose the WAF list, the web site that is 
>> causing the victim grief chooses the WAF list.
>> 
>>  
>> 
>> Owen
>> 
>>  
>> 
>> 
>> 
>> 
>> On Feb 20, 2024, at 14:15, j...@joelesler.net <mailto:j...@joelesler.net> 
>> wrote:
>> 
>>  
>> 
>> There are other WAF lists available on AWS besides their native one.  Ones 
>> that have support.
>> 
>> 
>> 
>> 
>> On Feb 20, 2024, at 16:18, George Herbert > <mailto:george.herb...@gmail.com>> wrote:
>> 
>>  
>> 
>> This is terrible advice, but you might need another netblock for the 
>> eyeballs.  Possibly a small one with enterprise NAT, but something outside 
>> the AWS list ranges...
>> 
>>  
>> 
>>  
>> 
>> -George
>> 
>>  
>> 
>> On Mon, Feb 19, 2024 at 7:35 PM Justin H. > <mailto:justindh...@gmail.com>> wrote:
>> 
>> That matches my experience with these types of problems in the past.  
>> Especially when the end-users don't have a process for white-listing.  
>> We actually got a response from one WAF user to "connect to another 
>> network to log in, then you should be able to use the site, because it's 
>> just the login page that's protected".
>> 
>> I am working with someone off-list, so I have hope this can be resolved 
>> without account gymnastics. :)
>> 
>> Justin H.
>> 
>> Owen DeLong wrote:
>> > The whole situation with these WAF as a service setups is a nightmare for 
>> > the affected (afflicted) parties.
>> >
>> > I saw this problem from both sides when I was at Akamai. It’s not great 
>> > from the service provider side, but it’s an absolute shit show for anyone 
>> > on the wrong side of a block. There’s no accountability or process for 
>> > redress of errors whatsoever. The impacted party isn’t a customer of the 
>> > WAF publisher, so they cant get any traction there. The WAF subscriber 
>> > blindly applies the WAF and it’s virtually impossible to track down anyone 
>> > there who even knows that they subscribe to such a thing, let alone get 
>> > them to take useful action.
>> >
>> > Best of luck.  The only thing I saw that worked while I was at Akamai was 
>> > a few entities subscribed to the WAF service and then complained about 
>> > getting blocked from their own web sites. Since they were then Akamai WAF 
>> > customers, they could get Akamai to take action.
>> >
>> > Crazy.
>> >
>> > Owen
>> >
>> >
>> >> On Feb 16, 2024, at 09:19, Justin H. > >> <mailto:justindh...@gmail.com>> wrote:
>> >>
>> >> Justin H. wrote:
>> >>> Hello,
>> >>>
>> >>> We found ou

Re: AWS WAF list

[Owen DeLong via NANOG]

Unfortunately, the victim doesn’t chose the WAF list, the web site that is 
causing the victim grief chooses the WAF list.

Owen


> On Feb 20, 2024, at 14:15, j...@joelesler.net wrote:
> 
> There are other WAF lists available on AWS besides their native one.  Ones 
> that have support.
> 
>> On Feb 20, 2024, at 16:18, George Herbert  wrote:
>> 
>> This is terrible advice, but you might need another netblock for the 
>> eyeballs.  Possibly a small one with enterprise NAT, but something outside 
>> the AWS list ranges...
>> 
>> 
>> -George
>> 
>> On Mon, Feb 19, 2024 at 7:35 PM Justin H. > <mailto:justindh...@gmail.com>> wrote:
>>> That matches my experience with these types of problems in the past.  
>>> Especially when the end-users don't have a process for white-listing.  
>>> We actually got a response from one WAF user to "connect to another 
>>> network to log in, then you should be able to use the site, because it's 
>>> just the login page that's protected".
>>> 
>>> I am working with someone off-list, so I have hope this can be resolved 
>>> without account gymnastics. :)
>>> 
>>> Justin H.
>>> 
>>> Owen DeLong wrote:
>>> > The whole situation with these WAF as a service setups is a nightmare for 
>>> > the affected (afflicted) parties.
>>> >
>>> > I saw this problem from both sides when I was at Akamai. It’s not great 
>>> > from the service provider side, but it’s an absolute shit show for anyone 
>>> > on the wrong side of a block. There’s no accountability or process for 
>>> > redress of errors whatsoever. The impacted party isn’t a customer of the 
>>> > WAF publisher, so they cant get any traction there. The WAF subscriber 
>>> > blindly applies the WAF and it’s virtually impossible to track down 
>>> > anyone there who even knows that they subscribe to such a thing, let 
>>> > alone get them to take useful action.
>>> >
>>> > Best of luck.  The only thing I saw that worked while I was at Akamai was 
>>> > a few entities subscribed to the WAF service and then complained about 
>>> > getting blocked from their own web sites. Since they were then Akamai WAF 
>>> > customers, they could get Akamai to take action.
>>> >
>>> > Crazy.
>>> >
>>> > Owen
>>> >
>>> >
>>> >> On Feb 16, 2024, at 09:19, Justin H. >> >> <mailto:justindh...@gmail.com>> wrote:
>>> >>
>>> >> Justin H. wrote:
>>> >>> Hello,
>>> >>>
>>> >>> We found out recently that we are on the HostingProviderIPList (found 
>>> >>> here 
>>> >>> https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups-ip-rep.html)
>>> >>>  at AWS and it's affecting our customers' access to various websites.  
>>> >>> We are a datacenter, and a hosting provider, but we have plenty of 
>>> >>> enterprise customers with eyeballs.
>>> >>>
>>> >>> We're finding it difficult to find a technical contact that we can 
>>> >>> reach since we're not an AWS customer.  Does anyone have a contact or 
>>> >>> advice on a solution?
>>> >> Sadly we're not getting any traction from standard AWS support, and end 
>>> >> users of the WAF list like Reddit and Eventbrite are refusing to 
>>> >> whitelist anyone.  Does anyone have any AWS contacts that might be able 
>>> >> to assist?  Our enterprise customers are becoming more and more impacted.
>>> >>
>>> >> Justin H.
>>> 
>> 
>> 
>> --
>> -george william herbert
>> george.herb...@gmail.com <mailto:george.herb...@gmail.com>

Re: AWS WAF list

[Owen DeLong via NANOG]

The whole situation with these WAF as a service setups is a nightmare for the 
affected (afflicted) parties. 

I saw this problem from both sides when I was at Akamai. It’s not great from 
the service provider side, but it’s an absolute shit show for anyone on the 
wrong side of a block. There’s no accountability or process for redress of 
errors whatsoever. The impacted party isn’t a customer of the WAF publisher, so 
they cant get any traction there. The WAF subscriber blindly applies the WAF 
and it’s virtually impossible to track down anyone there who even knows that 
they subscribe to such a thing, let alone get them to take useful action. 

Best of luck.  The only thing I saw that worked while I was at Akamai was a few 
entities subscribed to the WAF service and then complained about getting 
blocked from their own web sites. Since they were then Akamai WAF customers, 
they could get Akamai to take action. 

Crazy.

Owen


> On Feb 16, 2024, at 09:19, Justin H.  wrote:
> 
> Justin H. wrote:
>> Hello,
>> 
>> We found out recently that we are on the HostingProviderIPList (found here 
>> https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups-ip-rep.html)
>>  at AWS and it's affecting our customers' access to various websites.  We 
>> are a datacenter, and a hosting provider, but we have plenty of enterprise 
>> customers with eyeballs.
>> 
>> We're finding it difficult to find a technical contact that we can reach 
>> since we're not an AWS customer.  Does anyone have a contact or advice on a 
>> solution?
> Sadly we're not getting any traction from standard AWS support, and end users 
> of the WAF list like Reddit and Eventbrite are refusing to whitelist anyone.  
> Does anyone have any AWS contacts that might be able to assist?  Our 
> enterprise customers are becoming more and more impacted.
> 
> Justin H.

Re: IPv6 uptake (was: The Reg does 240/4)

[Owen DeLong via NANOG]

I can’t speak to Cisco as I don’t have recent experience there. Juniper, Linux, Palo Alto, and most others I’ve dealt with in the last 5 years pose no significant difference in writing policy for IPv6 vs. the process for IPv4. OwenOn Feb 17, 2024, at 10:23, Justin Streiner  wrote:We went pretty deep into the weeds on NAT in this thread - far deeper than I expected ;)Getting back to the recently revised topic of this thread - IPv6 uptake - what have peoples' experiences been related to crafting sane v6 firewall rulesets in recent products from the major firewall players (Palo Alto, Cisco, Fortinet, etc)?  On the last major v6 deployment I did, working with the firewalls was definitely one of the major pain points because the support / stability was really lacking, or there wasn't full feature parity between their v4 and v6 capabilities.Thank youjmsOn Fri, Feb 16, 2024 at 11:04 PM William Herrin  wrote:On Fri, Feb 16, 2024 at 7:41 PM John R. Levine  wrote:
> > That it's possible to implement network security well without using
> > NAT does not contradict the claim that NAT enhances network security.
>
> I think we're each overgeneralizing from our individual expeience.
>
> You can configure a V6 firewall to be default closed as easily as you can
> configure a NAT.

Hi John,

We're probably not speaking the same language. You're talking about
configuring the function of one layer in a security stack. I'm talking
about adding or removing a layer in a security stack. Address
overloaded NAT in conjunction with private internal addresses is an
additional layer in a security stack. It has security-relevant
properties that the other layers don't duplicate. Regardless of how
you configure it.

Also, you can't "configure" a layer to be default closed. That's a
property of the security layer. It either is or it is not.

You can configure a layer to be "default deny," which I assume is what
you meant. The issue is that anything that can be configured can be
accidentally unconfigured. When default-deny is accidentally
unconfigured, the network becomes wide open. When NAT is accidentally
unconfigured, the network stops functioning entirely. The gate is
closed.

Regards,
Bill Herrin


-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: IPv6 uptake (was: The Reg does 240/4)

[Owen DeLong via NANOG]

> Think of it like this: you have a guard, you have a fence and you have
> barbed wire on top of the fence. Can you secure the place without the
> barbed wire? Of course. Can an intruder defeat the barbed wire? Of
> course. Is it more secure -with- the barbed wire? Obviously.
> 
NAT is like the barbed wire. Anyone with a pair of wire cutters doesn’t need to 
defeat the barbed wire, they just cut the fence instead. 

But I understand how the barbed wire makes you and management feel warm and 
fuzzy. 

The problem here is that NAT is also like a big blind spot in the video cameras 
that should be helping you spot the guy cutting the fence. 

Owen

Re: IPv6 uptake (was: The Reg does 240/4)

[Owen DeLong via NANOG]

Bill, same scenario, but instead of fat fingering an outbound rule, you fat 
finger a port map for inbound connections to a different host and get the 
destination address wrong. 

Still hacked. 

NAT doesn’t prevent fat fingers from getting you hacked, it just changes the 
nature of the required fat fingering. 

Care to talk about trying to track down a compromised host through the audit 
trail given an abuse report that doesn’t include the source port number? 
(Oracle even one that happens to include it)?

Owen


> On Feb 16, 2024, at 17:05, William Herrin  wrote:
> 
> On Fri, Feb 16, 2024 at 3:13 PM Michael Thomas  wrote:
>> If you know which subnets need to be NAT'd don't you also know which
>> ones shouldn't exposed to incoming connections (or conversely, which
>> should be permitted)? It seems to me that all you're doing is moving
>> around where that knowledge is stored? Ie, DHCP so it can give it
>> private address rather than at the firewall knowing which subnets not to
>> allow access? Yes, DHCP can be easily configured to make everything
>> private, but DHCP for static reachable addresses is pretty handy too.
> 
> Hi Mike,
> 
> Suppose I have a firewall at 2602:815:6000::1 with an internal network
> of 2602:815:6001::/64. Inside the network on 2602:815:6001::4 I have a
> switch that accepts telnet connections with a user/password of
> admin/admin. On the firewall, I program it to disallow all Internet
> packets to 2602:815:6001::/64 that are not part of an established
> connection.
> 
> Someone tries to telnet to 2602:815:6001::4. What happens? Blocked.
> 
> Now, I make a mistake on my firewall. I insert a rule intended to
> allow packets outbound from 2602:815:6001::4 but I fat-finger it and
> so it allows them inbound to that address instead. Someone tries to
> telnet to 2602:815:6001::4. What happens? Hacked.
> 
> Now suppose I have a firewall at 199.33.225.1 with an internal network
> of 192.168.55.0/24. Inside the network on 192.168.55.4 I have a switch
> that accepts telnet connections with a user/password of admin/admin.
> On the firewall, I program it to do NAT translation from
> 192.168.55.0/24 to 199.33.225.1 when sending packets outbound, which
> also has the effect of disallowing inbound packets to 192.168.55.0/24
> which are not part of an established connection.
> 
> Someone tries to telnet to 192.168.55.4. What happens? The packet
> never even reaches my firewall because that IP address doesn't go
> anywhere on the Internet.
> 
> Now I make a mistake on my firewall. I insert a rule intended to allow
> packets outbound from 192.168.55.4 but I fat-finger it and so it
> allows them inbound to that address instead. Someone tries to telnet
> to 192.168.55.4. What happens? The packet STILL doesn't reach my
> firewall because that IP address doesn't go anywhere on the Internet.
> 
> See the difference? Accessible versus accessible and addressable. Not
> addressable enhances security.
> 
> Regards,
> Bill Herrin
> 
> 
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/

Re: IPv6 uptake (was: The Reg does 240/4)

[Owen DeLong via NANOG]

Most firewalls are default deny. Routers are default allow unless you put a 
filter on the interface.

NAT adds nothing to security (Bill and I agree to disagree on this), but at 
best, it complicates the audit trail. 

Owen


> On Feb 16, 2024, at 15:19, Jay R. Ashworth  wrote:
> 
> - Original Message -
>> From: "William Herrin" 
> 
>> On Fri, Feb 16, 2024 at 2:19 PM Jay R. Ashworth  wrote:
 From: "Justin Streiner" 
 4. Getting people to unlearn the "NAT=Security" mindset that we were forced
 to accept in the v4 world.
>>> 
>>> NAT doesn't "equal" security.
>>> 
>>> But it is certainly a *component* of security, placing control of what 
>>> internal
>>> nodes are accessible from the outside in the hands of the people inside.
>> 
>> Every firewall does that. What NAT does above and beyond is place
>> control of what internal nodes are -addressable- from the outside in
>> the hands of the people inside -- so that most of the common mistakes
>> with firewall configuration don't cause the internal hosts to -become-
>> accessible.
>> 
>> The distinction doesn't seem that subtle to me, but a lot of folks
>> making statements about network security on this list don't appear to
>> grasp it.
> 
> You bet.  I knew someone would chime in, but whether they'd be agreeing
> with me -- as you are -- or yelling at me, wasn't clear.
> 
> It's a default deny (NAT) vs default allow (firewall) question, and
> I prefer default deny -- at least inbound.  You *can* run NAT as default
> deny outbound, too, but it's much less tolerable for general internet
> connectivity -- in some dedicated circumstances, it can be workable.
> 
> Cheers,
> -- jra
> --
> Jay R. Ashworth  Baylink   
> j...@baylink.com
> Designer The Things I Think   RFC 2100
> Ashworth & Associates   http://www.bcp38.info  2000 Land Rover DII
> St Petersburg FL USA  BCP38: Ask For It By Name!   +1 727 647 1274

Re: IPv6 uptake (was: The Reg does 240/4)

[Owen DeLong via NANOG]

> On Feb 16, 2024, at 14:20, Jay R. Ashworth  wrote:
> 
> - Original Message -
>> From: "Justin Streiner" 
> 
>> 4. Getting people to unlearn the "NAT=Security" mindset that we were forced
>> to accept in the v4 world.
> 
> NAT doesn't "equal" security.
> 
> But it is certainly a *component* of security, placing control of what 
> internal
> nodes are accessible from the outside in the hands of the people inside.

Uh, no… no it is not. Stateful inspection (which the kind of NAT (actually 
NAPT) you are assuming here depends on) is a component of security. You can do 
stateful inspection without mutilating the header and have all the same 
security benefits without losing or complicating the audit trail. 

Owen

Re: The Reg does 240/4

[Owen DeLong via NANOG]

Mike, it’s true that Google used to be a lot less strict on IPv4 email than 
IPv6, but they want SPF and /or DKIM on everything now, so it’s mostly the 
same. There is less reputation data available for IPv6 and server reputation is 
a harder problem in IPv6, but reputation systems are becoming less relevant. 

YMMV, but if your mail server is properly configured for SPF and DKIM, you 
shouldn’t have any difference in SMTP experience with Google for either 
protocol. 

Owen


> On Feb 16, 2024, at 07:20, Mike Hammett  wrote:
> 
> 
> "Does any IPv6 enabled ISP provide PTR records for mail servers?"
> 
> I think people will conflate doing so at ISP-scale and doing so at 
> residential hobbiyst scale (and everything in between). One would expect 
> differences in outcomes of attempting PTR records in DIA vs. broadband.
> 
> "How does Google handle mail from an IPv6 server?"
> 
> A few people have posted that it works for them, but unless it has changed 
> recently, per conversations on the mailop mailing list, Google does not treat 
> IPv6 and IPv4 mail the same and that causes non-null issues.
> 
> 
> 
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
> 
> Midwest-IX
> http://www.midwest-ix.com
> 
> From: "Stephen Satchell" 
> To: nanog@nanog.org
> Sent: Wednesday, February 14, 2024 8:25:03 PM
> Subject: Re: The Reg does 240/4
> 
> On 2/14/24 4:23 PM, Tom Samplonius wrote:
> > The best option is what is happening right now:  you can’t get new IPv4
> > addresses, so you have to either buy them, or use IPv6.  The free market
> >   is solving the problem right now.  Another solution isn’t needed.
> 
> Really?  How many mail servers are up on IPv6?  How many legacy mail
> clients can handle IPv6?  How many MTA software packages can handle IPv6
> today "right out of the box" without specific configuration?
> 
> Does any IPv6 enabled ISP provide PTR records for mail servers?
> 
> How does Google handle mail from an IPv6 server?
> 
> The Internet is not just the Web.
> 

Re: The Reg does 240/4

[Owen DeLong via NANOG]

For everyone’s amusement:
[root@owen log]# grep 'IPv6' maillog | wc -l
2648
[root@owen log]# grep 'IPv4' maillog | wc -l
0


Now admittedly, this isn’t really a fair report because sendmail doesn’t tag 
IPv4 address as “IPv4” like it does IPv6 addresses.

e.g.: Feb 15 19:22:59 owen sendmail[1545111]: STARTTLS=server, relay=localhost 
[IPv6:0:0:0:0:0:0:0:1], version=TLSv1.3, verify=NOT, 
cipher=TLS_AES_256_GCM_SHA384, bits=256/256

A slightly more fair version:
[root@owen log]# grep 'connect from' maillog | wc -l
14547
[root@owen log]# grep 'connect from' maillog | grep IPv6 | wc -l
431


Which shows that 431 of 14547 total connections came via IPv6 during the log 
period (which begins 00:00:39 UTC Feb. 11) and continues to the time of this 
writing.

However, that is overly generous to IPv4 because a much higher percentage of 
the connections on IPv6 result in actual mail transfer while many of the IPv4 
connections are various failed authentication attempts, attempts to deliver 
rejected (SPAM, other) messages, and other various failures to complete the 
delivery process (disconnects after EHLO, etc.).

As stated earlier, approximately 40% of all mail received by my MTA arrives 
over IPv6.

FWIW, most of my netflix viewing is done via IPv6 as well.

turning off IPv4 is a tall order and a huge risk for Netflix to take, so I 
don’t see that happening. You’re not wrong about the likely impact, but it 
would be a rough contest between ISPs telling their customers “Netflix turned 
us off, blame them” and Netflix telling its customers “We’re no longer 
supporting the legacy internet protocol and your ISP needs to modernize.”. In 
the end it likely turns into a pox on both their houses and the ISPs in 
question and Netflix both lose a bunch of customers in the process.

OTOH, as new products come out that are unable to get IPv4 and are delivered 
over IPv6 only, this will eventually have roughly the same effect without the 
avoidable business risk involved in Netflix leading the way. this is my primary 
argument against the proposal, it will further delay this inevitability which, 
in turn, prolongs the pain period of this transition. While a handful of new 
entrants might benefit in some way in the short term from such a thing, in the 
long term, it’s actually harmful to everyone overall.

Owen


> On Feb 15, 2024, at 11:10, Lyndon Nerenberg (VE7TFX/VE6BBM) 
>  wrote:
> 
> I've said it before, and I'll say it again:
> 
>  The only thing stopping global IPv6 deployment is
>  Netflix continuing to offer services over IPv4.
> 
> If Netflix dropped IPv4, you would see IPv6 available *everywhere*
> within a month.
> 
> --lyndon

Re: The Reg does 240/4

[Owen DeLong via NANOG]

> On Feb 15, 2024, at 03:29, Christopher Hawker  wrote:
> 
> 
> Owen,
> 
> This is the first time we've presented this case so I'm uncertain as to how 
> you've come to the conclusion that I've "presented [my] case numerous times" 
> and that we "continue to persist".
> 
It may be your first time at bat, but this proposal has been rejected in the 
IETF many times before over at least 2 decades. 

> I also don't know how us diverting energy from 240/4 towards IPv6 deployment 
> in privately-owned networks will help. People cannot be made to adopt IPv6 
> (although IMO they should) and until they are ready to do so we must continue 
> to support IPv4, for new and existing networks. While we can encourage and 
> help people move towards IPv6 we can't force adoption through prevention of 
> access to IPv4.

Actually, no,  no we should not continue to support IPv4. The sooner there are 
real world consequences to those networks that have failed to implement IPv6, 
the sooner they will finally do so. 

Unfortunately, yes, this will be temporarily painful to new entrants that are 
IPv6 only until there is a sufficient critical mass of them to drive the 
remaining (and ever decreasing) IPv4 only networks to finally act. 

Delaying that inevitability only prolongs this pain and does not improve or 
promote any common good. 

Owen

Re: The Reg does 240/4

[Owen DeLong via NANOG]

> On Feb 14, 2024, at 18:25, Stephen Satchell  wrote:
> 
> On 2/14/24 4:23 PM, Tom Samplonius wrote:
>> The best option is what is happening right now:  you can’t get new IPv4
>> addresses, so you have to either buy them, or use IPv6.  The free market
>>  is solving the problem right now.  Another solution isn’t needed.
> 
> Really?  How many mail servers are up on IPv6?  How many legacy mail clients 
> can handle IPv6?  How many MTA software packages can handle IPv6 today "right 
> out of the box" without specific configuration?

Quite a few, actually. About 40% of my email comes and goes via IPv6. 

Sendai, postfix, outlook, and several others all handle IPv6 without need for 
any more IPv6 specific configuration than is required for IPv4. 

> 
> Does any IPv6 enabled ISP provide PTR records for mail servers?

Yes. Most of the transit providers I deal with offer ip6.arpa delegation at 
least. You can either stand up your own NS or use any of a variety of free DNS 
providers to host that delegation. 

> 
> How does Google handle mail from an IPv6 server?

So far I’ve had no issues exchanging mail with Google, Yahoo, or MSN (former 
Hotmail) on IPv6. 

> 
> The Internet is not just the Web.

True. Guess what… SSH, VNC, SMTP, IMAP, and many other things are working just 
fine on IPv6. 

IPv6 isn’t just the web either. IPv6 is the modern internet. 

Owen

Re: The Reg does 240/4

[Owen DeLong via NANOG]

There is one other mechanism available that has not yet come into play. One 
which this proposal seeks to further delay. In fact IMHO, the one that is most 
likely to ultimately succeed…

At some point new entrants will be unable to obtain IPv4. When there is a 
sufficient critical mass of those that IPv4 only sites cannot reach, those 
sites will be faced with an ROI on IPv6 deployment they can no longer ignore. 

Hence, not only is this bad idea a waste of effort, but it’s actually harmful 
in the short, medium, and long terms. 

Owen


> On Feb 14, 2024, at 15:35, Christopher Hawker  wrote:
> 
> 
> John,
> 
> If you feel that it is wasted time, you are welcome to not partake in the 
> discussion. Your remarks have been noted.
> 
> It's all well and good to say that "more sites could have IPv6 if time wasn't 
> being wasted on 240/4" however we can only do so much regarding the 
> deployment of v6 within networks we manage. All we can do is educate people 
> on the importance of IPv6 uptake, we can not force people to adopt it. The 
> only way to rapidly accelerate the uptake of IPv6 is for networks is to 
> either offer better rates for v6 transit, or disable v4 connectivity 
> completely.
> 
> Otherwise v6 connectivity is going to dawdle at the current rate it is.
> 
> Regards,
> Christopher Hawker
> From: NANOG  on behalf of John 
> Levine 
> Sent: Thursday, February 15, 2024 10:11 AM
> To: nanog@nanog.org 
> Subject: Re: The Reg does 240/4
>  
> It appears that William Herrin  said:
> >On Wed, Feb 14, 2024 at 9:23 AM Owen DeLong via NANOG  
> >wrote:
> >> Think how many more sites could have IPv6 capability already if this 
> >> wasted effort had been put into that, instead.
> >
> >"Zero-sum bias is a cognitive bias towards zero-sum thinking;
> 
> Well, OK, think how many more sites could hav IPv6 if people weren't
> wasting time arguing about this nonsense.
> 
> R's,
> John
> 
> 

Re: The Reg does 240/4

[Owen DeLong via NANOG]

> 
> 1. RIRs, following 
> https://www.icann.org/resources/pages/allocation-ipv4-rirs-2012-02-25-en, 
> would request new /8s, and receive those allocations.

I don’t think this applies any more. I could be wrong, but I think based on 
current practice, IANA would simply distribute 3 of the 16 /8s to each of the 
RIRs. 

That’s been the process for recovered blocks since the last 5 /8s from the free 
pool were distributed. 

> 2. Entities[*] with pent up demand would submit requests and have those 
> requests filled by the RIRs

Which would rapidly deplete that space in most RIRs and leave an abundance of 
wasted space sitting on the shelf in a couple of RIRs with policies that 
prolong the shortage on the pretense that it enhances the useful life of IPv4. 

> 3. While more /8s in 240/4 remain, go to step 1

Or not. (See my comment on step 1)

> 4. Return to status quo ante.

Which happens almost immediately for IANA and soon thereafter in most RIRs. 

> 
> In other words, while the IANA free pool is not (again) empty, network 
> operators would be able to get IPv4 address space at a fraction of the market 
> price, and then we’d go back to the way things are now.
> 
> This suggests the length of time the primary benefit (cheap IPv4 addresses) 
> would be enjoyed depends on RIR allocation policies.  ISTR a comment from you 
> earlier suggesting that based on current consumption rates, 240/4 would 
> fulfill needs for 50 years.  However, this appears to assume that current 
> “soft landing” (etc) policies would remain in place.  Why would you assume 
> that?  I would imagine there would be non-trivial pressure from the RIR 
> memberships to return to the pre-runout policy regime which was burning 
> through multiple /8s in months. In particular, I’d think the large scale 
> buyers of address space (as well as IP market speculators) who tend to be the 
> most active in RIR policy forums would jump at the opportunity to get “huge 
> tracts of land” at bargain basement prices again.
> 
> This doesn’t seem all that positive to me, particularly because it’s 
> temporary since the underlying problem (limited resource, unlimited demand) 
> cannot be addressed.  What positive impact do you predict?

Here, I 100% agree with David. (Which is quite rare)

Owen

Re: The Reg does 240/4

[Owen DeLong via NANOG]

That experiment already failed with the original v6 adoption process. It’s been 
more than 20 years and all we have proven is that as long as people can have an 
excuse to avoid v6 deployment, they will continue to do so. 

Giving them another 20 years of excuses is a step against the collective good 
IMHO. 

Owen


> On Feb 13, 2024, at 14:43, Christopher Hawker  wrote:
> 
> 
> Per my original email, looking at current exhaustion rates in the APNIC 
> service region, if we stuck to allocating space to new entities and 
> maintained allocating a maximum of a /22 to networks, just 3 x /8 would last 
> over 20 years. This should be a more than sufficient timeframe for a much 
> wider v6 adoption and deployment.
> 
> Regards,
> Christopher Hawker
> From: NANOG  on behalf of 
> Lyndon Nerenberg (VE7TFX/VE6BBM) 
> Sent: Wednesday, February 14, 2024 7:42 AM
> To: North American Operators' Group 
> Subject: Re: The Reg does 240/4
>  
> And what are they going to do when 240/4 runs out?

Re: The Reg does 240/4

[Owen DeLong via NANOG]

This gift from the bad idea fairy just keeps on giving. You’ve presented your case numerous times. The IETF has repeatedly found no consensus for it and yet you persist. Think how many more sites could have IPv6 capability already if this wasted effort had been put into that, instead. OwenOn Feb 13, 2024, at 14:16, Christopher Hawker  wrote:






Hi Tom,




We aren't trying to have a debate on this. All we can do is present our case, explain our reasons and hope that we can gain a consensus from the community.




I understand that some peers don't like the idea of this happening and yes we understand the technical work behind getting this across the line. It's easy enough for us to say "this will never happen" or to put it into the "too hard" basket, however, the one
 thing I can guarantee is that will never happen, if nothing is done.




Let's not think about ourselves for a moment, and think about the potential positive impact that this could bring.




Regards,

Christopher Hawker


From: Tom Beecher 
Sent: Wednesday, February 14, 2024 1:23 AM
To: Christopher Hawker 
Cc: North American Operators' Group ; aus...@lists.ausnog.net ; Christopher Hawker via sanog ; apnic-t...@lists.apnic.net 
Subject: Re: The Reg does 240/4
 




Now, we know there's definitely going to be some pushback on this. This won't be easy to accomplish and it will take some time. 


 It won't ever be 'accomplished' by trying to debate this in the media.



On Tue, Feb 13, 2024 at 5:05 AM Christopher Hawker  wrote:





Hello all,




[Note: I have cross-posted this reply to a thread from NANOG on AusNOG, SANOG and APNIC-Talk in order to invite more peers to engage in the discussion on their respective forums.]




Just to shed some light on the article and our involvement...



Since September 1981, 240/4 has been reserved for future use, see

https://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xhtml. This space has always been reserved for future use and given the global shortage of available space for new network operators we feel it is appropriate for this space to be reclassified
 as Unicast space available for delegation by IANA/PTI to RIRs on behalf of ICANN.



At present, the IP space currently available for RIRs to delegate to new members is minimal, if any at all. The primary
 goal of our call for change is to afford smaller players who are wanting to enter the industry the opportunity to do so without having to shell out the big dollars for space. Although I do not agree with IP space being treated as a commodity (as this was not
 what it was intended to be), those who can afford to purchase space may do so and those who cannot should be able to obtain space from their respective RIR without having to wait over a year in some cases just to obtain space. It's not intended to flood the
 market with resources that can be sold off to the highest bidder, and this can very well be a way for network operators to plan to properly roll out IPv6. At this point in time, the uptake and implementation of IPv6 is far too low (only 37% according to

https://stats.labs.apnic.net/ipv6) for new networks to deploy IPv6 single-stack, meaning that we need to continue supporting IPv4 deployments.


The reallocation of IPv4 space marked as Future Use would not restrict or inhibit the deployment of IPv6, if anything,
 in our view it will help the deployment through allowing these networks to service a greater number of customers than what a single /24 v4 prefix will allow. Entire regions of an economy have the potential to be serviced by a single /23 IPv4 prefix when used
 in conjunction with IPv6 space.


Now, some have argued that we should not do anything with IPv4 and simply let it die out. IPv4 will be around for the
 foreseeable future and while it is, we need to allow new operators to continue deploying networks. It is unfair of us to say "Let's all move towards IPv6 and just let IPv4 die" however the reality of the situation is that while we continue to treat it as a
 commodity and allow v6 uptake to progress as slowly as it is, we need to continue supporting it v4. Some have also argued that networks use this space internally within their infrastructure. 240/4 was always marked as Reserved for Future Use and if network
 operators elect to squat on reserved space instead of electing to deploy v6 across their internal networks then that is an issue they need to resolve, and it should not affect how it is reallocated. It goes against the bottom-up approach of policy development
 by allowing larger network operators to state that this space cannot be made unicast because they are using it internally (even though it's not listed in RFC1918), and its reallocation would affect their networks.


In the APNIC region, there is a policy which only allows for a maximum of a /23 IPv4 prefix to be allocated/assigned to
 new members and any more space required must be acquired through other means. If (as 

IPv6 Test Pages for Fortune 500 and Top 100 web sites are back

[Owen DeLong via NANOG]

Don’t know how much anyone will still care about these pages as there are lots 
of other sources of similar data these days.

However, I finally got around to fixing the two pages I maintain:

http://www.delong.com/ipv6_fortune500.html and
http://www.delong.com/ipv6_alexa500.html

In the case of Alexa, the page is no longer based on Alexa since Amazon 
discontinued that service and now uses the Majestic 1,000,000 as a source 
(grabs the first 500 entries from their list). This page was broken since 
Amazon discontinued the Alexa service.

The Fortune 500 site still uses the same datasource, but the script was 
crashing due to sites with borked SSL implementations which caused PERL to 
abort on an exception that I never figured out how to trap or ignore. As such, 
I’m now manually maintaining an exception list of such sites in the script and 
testing them is bypassed to prevent the script from crashing. Obviously, this 
is not ideal, but I found no better solution so far.

We now return you to your regularly scheduled NANOG chatter.

Owen

 

Re: route: 0.0.0.0/32 in LEVEL3 IRR

[Owen DeLong via NANOG]

> On Jan 31, 2024, at 23:19, Frank Habicht  wrote:
> 
> On 01/02/2024 01:45, Tom Beecher wrote:
>> Seems a bit dramatic. Companies all over the world have been using other 
>> people's public IPs internally for decades. I worked at a place 20 odd years 
>> ago that had an odd numbering scheme internally, and it was someone else's 
>> public space. When I asked why, the guy who built it said "Well I just liked 
>> the pattern."
>> If you're not announcing someone else's space into the DFZ, or otherwise 
>> trying to do anything shady, the three letter agencies aren't likely to come 
>> knocking. Doesn't mean anyone SHOULD be doing it, but still.
> 
> Well...
> 
> If you're using 20.20.20.0/24 which is not "yours" (as I've seen happen), 
> then certainly your customers can't get to the real 20.20.20.x
> And even if that's not announced and used /today/ - this can change quickly...
> 
> Frank

You are repeating exactly the argument I made at the time.

Owen

Re: route: 0.0.0.0/32 in LEVEL3 IRR

[Owen DeLong via NANOG]

For many years, a large customer (telco/VOIP/ISP carrier that should have known 
better) of a former employer was using 11.0.0.0/8 as an extension of 10.0.0.0/8 
and literally forced said employer to carry their routes to those prefixes in 
those tables (or lose an extremely lucrative contract). At the time, 11/8 was 
IANA resrved, and my point that it was likely to be allocated to an RIR and 
subsequently some real entitie(s) on the internet was utterly lost in the 
pursuit of the almighty dollar. I left that job for greener pastures before 
IANA allocated that prefix, but I’m sure there were some definite interesting 
results there when it happened.

Owen


> On Jan 31, 2024, at 14:45, Tom Beecher  wrote:
> 
>> Even though it is very risky to steal resources from an organization
>> that can deploy a black helicopter or a nuclear warhead over you
> 
> Seems a bit dramatic. Companies all over the world have been using other 
> people's public IPs internally for decades. I worked at a place 20 odd years 
> ago that had an odd numbering scheme internally, and it was someone else's 
> public space. When I asked why, the guy who built it said "Well I just liked 
> the pattern." 
> 
> If you're not announcing someone else's space into the DFZ, or otherwise 
> trying to do anything shady, the three letter agencies aren't likely to come 
> knocking. Doesn't mean anyone SHOULD be doing it, but still.  
> 
> On Wed, Jan 31, 2024 at 12:49 AM Rubens Kuhl  > wrote:
>> DoD's /8s are usually squatted by networks that run out of private IPv4 
>> space.
>> Even though it is very risky to steal resources from an organization
>> that can deploy a black helicopter or a nuclear warhead over you, for
>> some reason like it not appearing in the DFZ people seem to like it.
>> 
>> 
>> Rubens
>> 
>> On Tue, Jan 30, 2024 at 11:40 PM Dave Taht > > wrote:
>> >
>> > That's pretty cool, actually. I keep wondering when someone will offer
>> > up a 0.0.0.0/8. ..
>> >
>> > https://www.ietf.org/archive/id/draft-schoen-intarea-unicast-0-00.html
>> >
>> > There must be more people out there than just amazon and google that
>> > ran out of 10/8.
>> >
>> > On Tue, Jan 30, 2024 at 11:29 AM Frank Habicht > > > wrote:
>> > >
>> > > Hi,
>> > >
>> > > I got 2 bounces for the email addresses seen below for an email similar
>> > > to the below...
>> > >
>> > > Anyone want to remove this IRR entry before anyone notices...???   ;-)
>> > >
>> > > Frank
>> > >
>> > >
>> > > I believe that the entry of
>> > > route:  0.0.0.0/32 
>> > >
>> > > does not serve any good purpose?
>> > >
>> > > I was surprised to see it in a list of prefixes from bgpq4 and the very
>> > > good https://irrexplorer.nlnog.net/prefix/0.0.0.0 guided me that it's in
>> > > "Level3".
>> > >
>> > > I'm wondering how many auto-generated filters contain this unnecessary
>> > > prefix
>> > >
>> > > PS: Oh, just seen - it's from TODAY. Maybe remove before anyone sees 
>> > > it...?
>> > >
>> > > Thanks for looking into this,
>> > > Frank
>> > >
>> > >
>> > > [frank@fisi ~]$ whois -h rr.level3.com  
>> > > 0.0.0.0/32 
>> > > [Querying rr.level3.com ]
>> > > [rr.level3.com ]
>> > > route:  0.0.0.0/32 
>> > > origin: AS10753
>> > > mnt-by: TCCGlobalNV-MNT
>> > > changed:ankita.gre...@lumen.com 
>> > > source: LEVEL3
>> > > last-modified:  2024-01-30T11:04:49Z
>> > >
>> > >
>> > > [frank@fisi ~]$ whois -h rr.level3.com  
>> > > TCCGlobalNV-MNT
>> > > [Querying rr.level3.com ]
>> > > [rr.level3.com ]
>> > > mntner: TCCGlobalNV-MNT
>> > > descr:  TCC Global N.V.
>> > > auth:   CRYPT-PW DummyValue  # Filtered for security
>> > > upd-to: ripehostmas...@eu.centurylink.net 
>> > > 
>> > > tech-c: LTHM
>> > > admin-c:LTHM
>> > > mnt-by: TCCGlobalNV-MNT
>> > > changed:ankita.gre...@lumen.com 
>> > > source: LEVEL3
>> > > last-modified:  2024-01-30T11:01:52Z
>> > >
>> >
>> >
>> > --
>> > 40 years of net history, a couple songs:
>> > https://www.youtube.com/watch?v=D9RGX6QFm5E
>> > Dave Täht CSO, LibreQos

Re: SOVC - BGp RPKI

[Owen DeLong via NANOG]

SOVC appears to be a Cisco-specific acronym and it’s pretty certain that the 
OVC stands for Origin Validation Cache. My best intuition based on the research 
I’ve been able to do is that the S stands for Secure (on the pretense that RPKI 
and Origin Validation have something to do with security and because X.509 
certificate and encryption and marketing buzzwords YAY!)

Juniper refers to their equivalent database simply as “Route Validation (RV) 
Records in the RV Database.

Hope that helps.

Owen


> On Jan 31, 2024, at 14:32, Tom Beecher  wrote:
> 
>> I see it mentioned in this doc:
>> https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/15-s/irg-15-s-book/irg-origin-as.pdf
> 
> You see SOVC mentioned, yes. But you don't see the word 'stale'. 
> 
> 
> 
> Please don't just paste what ChatGPT says. It's not an authoritative source.  
> I can find no Cisco document stating what the acronym MEANS. But the context 
> they use it seems to imply the word 'stale' isn't appropriate.
> 
> 
> 
>> A prefix or prefix range and the origin-AS corresponding to it are 
>> considered an SOVC record. Overlapping prefix ranges are allowed. An SOVC 
>> table containing three records might look like this:
>  
>>  Valid—Indicates the prefix and AS pair are found in the SOVC table.
> 
>> If more than one RPKI server is configured, the router will connect to all 
>> configured servers and download prefix information from all of them. The 
>> SOVC table will be made of the union of all the records received from the 
>> different servers.
> 
>  
>>  In the following example, the router is configured to connect to two RPKI 
>> servers, from which it will receive SOVC records of BGP prefixes and AS 
>> numbers.
> 
> On Wed, Jan 31, 2024 at 3:34 PM Compton, Rich via NANOG  > wrote:
>> ChatGPT says:
>> 
>> SOVC in the context of RPKI (Resource Public Key Infrastructure) on a Cisco 
>> router stands for "Stale Origin Validation Cache". RPKI is a security 
>> framework designed to secure the Internet's routing infrastructure, 
>> primarily through route origin validation. It ensures that the Internet 
>> number resources (like IP addresses and AS numbers) are used by the 
>> legitimate owners or authorized AS (Autonomous System).
>> 
>> In RPKI, Route Origin Authorizations (ROAs) are used to define which AS is 
>> authorized to announce a specific IP address block. Network devices, like 
>> Cisco routers, use these ROAs to validate the authenticity of BGP (Border 
>> Gateway Protocol) route announcements.
>> 
>> The term "stale" in SOVC refers to a situation where the router's 
>> RPKI-to-Router protocol client has lost its connection to the RPKI server, 
>> or when the RPKI cache data is outdated and not refreshed for some reason. 
>> This can happen due to network issues, configuration errors, or problems 
>> with the RPKI server itself. When the RPKI cache is stale, the router cannot 
>> reliably validate BGP route announcements against the latest ROA data, 
>> potentially affecting routing decisions.
>> 
>> In a network security context, maintaining an up-to-date RPKI cache is 
>> crucial for ensuring that the network only accepts legitimate routing 
>> announcements, thereby reducing the risk of routing hijacks or 
>> misconfigurations. As a network security engineer, managing and monitoring 
>> the RPKI status on routers is an important aspect of ensuring network 
>> security and integrity.
>> 
>>  
>> 
>>  
>> 
>>  
>> 
>> I see it mentioned in this doc:
>> 
>> https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/15-s/irg-15-s-book/irg-origin-as.pdf
>> 
>>  
>> 
>>  
>> 
>> From: NANOG > > on behalf of Mohammad Khalil 
>> mailto:eng.m...@gmail.com>>
>> Date: Wednesday, January 31, 2024 at 10:35 AM
>> To: NANOG list mailto:nanog@nanog.org>>
>> Subject: SOVC - BGp RPKI
>> 
>> Greetings Am have tried to find out what is the abbreviation for SOVC with 
>> no luck. #sh bgp ipv4 unicast rpki servers  BGP SOVC neighbor is X. X. X. 
>> 47/323 connected to port 323 Anyone have encountered this? Thanks! ‍ ‍ ‍ ‍ ‍ 
>> ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
>> 
>> Greetings
>> 
>> Am have tried to find out what is the abbreviation for SOVC with no luck.
>> 
>>  
>> 
>> #sh bgp ipv4 unicast rpki servers 
>> 
>> BGP SOVC neighbor is X.X.X.47/323 connected to port 323
>> 
>>  
>> 
>> Anyone have encountered this?
>> 
>>  
>> 
>> Thanks!
>> 

Re: If I announce 192.0.2.0/24, do I need a discard route? (Looking for a reference…)

[Owen DeLong via NANOG]

> On Jan 31, 2024, at 13:46, Warren Kumari  wrote:
> 
> 
> 
> 
> 
> On Wed, Jan 31, 2024 at 3:56 PM, William Herrin  > wrote:
>> On Wed, Jan 31, 2024 at 12:30 PM Warren Kumari > > wrote:
>> 
>> So, let's say I'm announcing some address space (e.g 192.0.2.0/24 
>> ), but I'm only using part of it internally (e.g 
>> 192.0.2.0/25 ). I've always understood that it's best 
>> practice[0] to have a discard route (eg static to null0/discard or 
>> similar[1]) for what I'm announcing.
>> 
>> Hi Warren,
>> 
>> Your router won't announce 192.0.2.0/24  unless it 
>> knows a route to 192.0.2.0/24  or has been configured 
>> to aggregate any internal routes inside 192.0.2.0/24  
>> to 192.0.2.0/24 .
>> 
> 
> 
> It that always true? I'd started off thinking that, but a friend of mine 
> (yes, the same one that started this  argument) convinced me that some forms 
> of BGP summarization/aggregation don't always generate a "local" route…

It’s not ALWAYS true. For example, aggregate-address commands and equivalent on 
many platforms will cause the route to be announced if any component prefixes 
are present in the RIB on some platforms.

There are also some cases where “auto-summary” (does anyone actually use this? 
EVER?) will have a similar effect.

There are also some implementations where disabling synchronization or in many 
cases where synchronization has simply been deprecated by the implementor where 
any bap “network” statement (or equivalent) will result in announcement 
regardless of what’s in the RIB.

> I'd also thought that I'd seen this when redistributing an IGP into BGP, and 
> using that as a contributor to 'aggregate-address' on Cisco devices. This is 
> from a long time ago, and really hazy now, but I'd thought that any 
> contributor would cause that the aggregate-address route to be announced, and 
> a local hold down not to be created.  It's possible that a: I'm just wrong b: 
> this is not longer true, c: both of the above.

Redistributing an IGP into BGP is almost always a bad idea. However, that 
aside, yes, as mentioned above, exactly what you are saying here is true with 
or without the redistribution on most platforms IIRC.

a: you are not wrong
b: it’s still true on many platforms at least (though may be implementation 
dependent)
c: no, but it’s certainly not best practice to behave in this way or depend on 
either of these behaviors for the other original reason you stated among other 
reasons.

With BGP, you really want to have a deterministic clean definition of what your 
router will do. As a general rule, if your peer is reachable, you usually want 
to advertise your originated routes to them and make damn sure your router can 
reach those some how no matter what.

> There are also some more inventive ways of getting routes into BGP, like 
> using ExaBGP as an example.

Sure, but using ExaBGP really amounts to originating the prefix from another 
router. This discussion was (at least theoretically) about local origination on 
the router in question.

Owen

> 
> W
> 
> 
> 
>> 192.0.2.0/25  doesn't count; it needs to know a route 
>> to 192.0.2.0/24 . Sending 192.0.2.0/24 
>>  to discard guarantees that the router has a route to 
>> 192.0.2.0/24 .
>> 
>> Historically, folks would put 192.0.2.0/24 on the ethernet port. Then, when 
>> carrier was lost on the ethernet port for a moment, the router would no 
>> longer have a route to 192.0.2.0/24 , so it'd withdraw 
>> the announcement for 192.0.2.0/24 . This is a bad idea 
>> for obvious reasons, so best practice was to put a low priority route to 
>> discard as a fall-back if the ethernet port briefly lost carrier.
>> 
>> Regards, 
>> Bill Herrin
>> 
>> -- 
>> William Herrin 
>> b...@herrin.us  
>> https://bill.herrin.us/
>> 
> 
> 

Re: If I announce 192.0.2.0/24, do I need a discard route? (Looking for a reference…)

[Owen DeLong via NANOG]

> On Jan 31, 2024, at 12:30, Warren Kumari  wrote:
> 
> Hey all,
> 
> This falls into the "Somebody is wrong on the Internet …" category.

Doesn’t everything eventually end up there?

> So, let's say I'm announcing some address space (e.g 192.0.2.0/24 
> ), but I'm only using part of it internally (e.g 
> 192.0.2.0/25 ). I've always understood that it's best 
> practice[0] to have a discard route (eg static to null0/discard or 
> similar[1]) for what I'm announcing.

Usually, but it’s not always necessary.

> There are a bunch of reasons for this, but the standard (or easiest to 
> explain one!) is what happens if this comes from some provider space, and 
> they announce a supernet/covering route. If I *don't* have a 
> discard/hold-down route, and a packet is sent to part of the space I'm not 
> using (e.g 192.0.2.200), I would send it to the covering route, they would 
> just send it back to the more specific, I'd return it to them, etc…

Well… Unless you have some other more specific covering the rest of the space, 
yes, this is a risk.

> Many, but not all mechanisms that people use for advertising a route in BGP 
> automagically create this sort of discard route (e.g Juniper's 'aggregate'), 
> but I wasn't really able to find any useful documentation suggesting that if 
> you announce a route, you should make sure that you have some route covering 
> all of the space… 

I don’t know if it’s documented, but it’s certainly common sense.

> Perhaps there isn't really anything saying this (because it's obvious), but 
> I'd really like to find something so that I can point at it…. 

I think your “reason” paragraph above is quite sufficient explanation, no?

> Can anyone help me win this somewhat pointless argument?

No, because if you have an opponent who won’t buy the content of your reason 
paragraph above, you are arguing with someone who isn’t going to take any 
amount of fact or documentation over whatever ill conceived reality they have 
decided to live in.

You simply cannot win an argument when confronted with an opponent living in an 
alternative reality.

Owen

Re: SOVC - BGp RPKI

[Owen DeLong via NANOG]

I’m not sure what the acronym is, but I believe it’s an origin validator 
connection.

(bap rpki server)

Owen


> On Jan 31, 2024, at 05:16, Mohammad Khalil  wrote:
> 
> Greetings
> Am have tried to find out what is the abbreviation for SOVC with no luck.
> 
> #sh bgp ipv4 unicast rpki servers 
> BGP SOVC neighbor is X.X.X.47/323 connected to port 323
> 
> Anyone have encountered this?
> 
> Thanks!

Re: Networks ignoring prepends?

[Owen DeLong via NANOG]

> 
> When you twist a policy knob to move BGP off its defaults, you take
> responsibility for making a better routing choice. And for correcting
> that choice if it should prove faulty. What I've seen here in this
> thread is a bunch of folks abdicating that responsibility. That's not
> unexpected, but it is disappointing.

Better is in the eye of the beholder. From your perspective, better is the 
lowest latency. From almost any ISPs perspective, better is the revenue 
positive path, followed by the revenue neutral path, with last choice being the 
revenue negative path. 

From 3356 perspective, they ARE choosing the best route… the route that pays 
them. 

Owen

Re: Networks ignoring prepends?

[Owen DeLong via NANOG]

BGP is more of a PDVP (Policy Distance Vector Protocol). Policy will always 
override Distance in BGP and is pretty much the key difference between an EGP 
and an IGP. 

Once you recognize that, the rest makes much more sense. 

Owen


> On Jan 23, 2024, at 14:29, William Herrin  wrote:
> 
> On Tue, Jan 23, 2024 at 12:34 PM Niels Bakker  wrote:
>> BGP, while a distance vector protocol, famously does not take
>> latency into account when making routing decisions.
> 
> Unless overridden, BGP takes -distance- into account where distance =
> AS path length.
> 
> Centurylink has overridden that with a localpref so that it DOES NOT
> take distance into account. Which rather defeats the function of a
> distance vector protocol.
> 
> Regards,
> Bill Herrin
> 
> 
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/

Re: Networks ignoring prepends?

[Owen DeLong via NANOG]

> On Jan 23, 2024, at 10:47, Jay Borkenhagen  wrote:
> 
> William Herrin writes:
>> 
>> The best path to me from Centurylink is: 3356 1299 20473 11875
>> 
>> The path Centurylink chose is: 3356 47787 47787 47787 47787 53356
>> 11875 11875 11875
>> 
>> Do you want to tell me again how that's a reasonable path selection,
>> or how I'm supposed to pass communities to either 20473 or 53356 which
>> tell 3356 to behave itself?
>> 
> 
> What you want to do is pass communities to 3356 so they apply the same
> local-pref to routes from both paths, enabling as-path-length-based
> path selection to work.  That means lowering their local-pref on the
> currently-chosen customer path via 47787 to match the local-pref on
> the their 1299 peer path.
> 
> as3356's TE communities are listed in their IRR aut-num: AS3356
> object: 
> 
> remarks: 
> remarks: customer traffic engineering communities - LocalPref
> remarks: 
> remarks:3356:70 - set local preference to 70
> remarks:3356:80 - set local preference to 80
> remarks:3356:90 - set local preference to 90
> remarks: 
> 
> Those communities look like RFC1998.  Thus presumably 3356's peer
> local-pref is 80, and you'll want to signal using 3356:80.  As you
> make signaling changes you should use as3356's looking glass to
> confirm.
> 
> as47787 and as53356 should pass your 3356:80 community along to
> as3356.  If they don't do so, complain to them or vote with your
> feet.

The catch to all of that, however, is that he’s not directly peered with 3356 
and many AS operators strip communities.

Owen

Re: Networks ignoring prepends?

[Owen DeLong via NANOG]

I’d bet that 47787 is a paying century link customer. As such, despite the 
ugliness of the path, CL probably local prefs everything advertised by them 
higher than any non-paying link. I’m willing to bet 1299 is peered and not 
paying CL. 

Sending bits for revenue is almost always preferable to sending bits for free, 
so…

Owen


> On Jan 22, 2024, at 12:37, William Herrin  wrote:
> 
> On Mon, Jan 22, 2024 at 10:19 AM James Jun  wrote:
>> So, as a customer, you actually SHOULD be demanding your ISPs
>> to positively identify and categorize their routes using local-pref
>> and communities.
> 
> Hi James,
> 
> The best path to me from Centurylink is: 3356 1299 20473 11875
> 
> The path Centurylink chose is: 3356 47787 47787 47787 47787 53356
> 11875 11875 11875
> 
> Do you want to tell me again how that's a reasonable path selection,
> or how I'm supposed to pass communities to either 20473 or 53356 which
> tell 3356 to behave itself?
> 
> Regards,
> Bill Herrin
> 
> 
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/

Re: Networks ignoring prepends?

[Owen DeLong via NANOG]

And now you are faced with an object lesson as to why TE routes are so 
prevalent. 

Less specifics are your only functional alternative here. In most cases, you 
shouldn’t need more than 2 per prefix. 

Owen


> On Jan 22, 2024, at 12:16, William Herrin  wrote:
> 
> On Mon, Jan 22, 2024 at 5:23 AM Jon Lewis  wrote:
>> You may be limited to seeing if your backup providers have community
>> controls that would let you tell them "don't share with Centurylink"
> 
> As I already explained, neither the primary nor any of the backup
> providers directly peer with Centurylink, thus have no communities for
> controlling announcements to Centurylink.
> 
> I hate to litter the table with a batch of more-specifics that only
> originate from the short, preferred link but I'm not hearing any
> practical alternatives. Treating my distant links as equivalent even
> though I told you with prepends that they are not leaves me with few
> knobs I can turn.
> 
> Regards,
> Bill Herrin
> 
> 
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/

Re: IRR information & BYOIP (Bring Your Own IP) with Cloud Providers

[Owen DeLong via NANOG]

On Jan 21, 2024, at 16:10, Christopher Morrow  wrote:On Sun, Jan 21, 2024, 5:39 PM Owen DeLong <o...@delong.com> wrote:On Jan 21, 2024, at 12:07, Christopher Morrow <morrowc.li...@gmail.com> wrote:On Fri, Jan 19, 2024, 4:55 PM Owen DeLong via NANOG <nanog@nanog.org> wrote:Sounds like you’ve got a weird mix of route origination. Why wouldn’t you advertise to Google via BGP and have your prefix originate from your own ASN?I think in this case the customer has their own disconnected deployment, and they are asking 396982 to announce some subset of their prefixes such that gcp gets that traffic.If that’s the case, IMHO, the better solution is to obtain a second ASN and announce that to GCP. Create ROAs (if you want to use RPKI) accordingly. That's not the product today, and really this is all accomplished with cloud goop inside gcp. (aws and azure have similar offerings, I believe)Interesting. It’s what several of my consulting clients are doing. They simply treat the cloud provider as an upstream peer on their cloud virtual router and bgp as normal (ish, those cloud virtual routers are pretty strange). Owen

Re: IRR information & BYOIP (Bring Your Own IP) with Cloud Providers

[Owen DeLong via NANOG]

On Jan 21, 2024, at 12:07, Christopher Morrow  wrote:On Fri, Jan 19, 2024, 4:55 PM Owen DeLong via NANOG <nanog@nanog.org> wrote:Sounds like you’ve got a weird mix of route origination. Why wouldn’t you advertise to Google via BGP and have your prefix originate from your own ASN?I think in this case the customer has their own disconnected deployment, and they are asking 396982 to announce some subset of their prefixes such that gcp gets that traffic.If that’s the case, IMHO, the better solution is to obtain a second ASN and announce that to GCP. Create ROAs (if you want to use RPKI) accordingly. Having Google originate prefixes on your behalf that are a subset of what you are announcing is just asking for difficulties you don’t need. Owen

Re: Stealthy Overlay Network Re: 202401100645.AYC Re: IPv4 address block

[Owen DeLong via NANOG]

> 2)Philosophically, IPv6 and IPv4 are kind of like two religions, each 
> with its own believers. As long as the devotees of each focus on their 
> respective passion, the world will be peaceful. As soon as one camp imposes 
> its preference onto the other, friction starts. Unchecked, it can go even 
> worse. ... But, I digressed. 

Think of IPv4 like 8-track tapes and IPv6 like a modern streaming service. Yes, 
you need more recent hardware than 1975 to play IPv6, but it still delivers the 
same basic services as IPv4, just better and with a bigger address space.

IPv6 (or something) has to replace IPv4. Continuing to pretend that we can 
cobble IPv4 into a sustainable solution for a global internet going forward is 
just prolonging the pain. 

EZIP is just another failed attempt at pretending math can be overcome. 

Owen

Re: One Can't Have It Both Ways Re: Streamline the CG-NAT Re: EzIP Re: IPv4 address block

[Owen DeLong via NANOG]

No. No matter how you cobble it, IPv4 doesn’t have enough addresses to restore proper end to end connectivity. OwenOn Jan 20, 2024, at 07:36, Abraham Y. Chen  wrote:

  

  
  
Hi, Owen:

  
1)    "  ...  IPv4 used
to work before NAT made everything horrible.  ":

  
    Utilizing 240/4, RAN
is a flat space which should support this kind of rudimentary
end-to-end connectivity within each RAN. (called L2 routing,
correct?)

  
Regards,

  

  
Abe (2024-01-20 10:35)




On 2024-01-19 04:02, Owen DeLong wrote:


  
  Any host connected to a reasonably well peered ISP (e.g. NOT
  Cogent) with IPv6 should be able to communicate with any other
  such host so long as the administrative policies on both sides
  permit it.
  
  
  I have no difficulty directly reaching a variety of IPv6
hosts from the /48 in my home.
  
  
  However, it’s not like dial-up modem operations in the PSTN
in that IP is an inherently connectionless packet switched
service while modems were an inherently circuit switched
connection oriented service.
  
  
  However, it does work like IPv4 used to work before NAT made
everything horrible.
  
  
  Owen
  

  
On Jan 15, 2024, at 12:20, Abraham Y. Chen 
  wrote:


  
   Hi, Forrest:

  
1)    I have
a question: 

  
    If I
subscribe to IPv6, can I contact another similar
subscriber to communicate (voice and data) directly
between two homes in private like the dial-up modem
operations in the PSTN? If so, is it available
anywhere right now?
  

   
Regards,

  

  
Abe
(2024-01-15 15:20)




  

  


om
  

  
   
  

  


  



  

Re: Backward Compatibility Re: 202401100645.AYC Re: IPv4 address block

[Owen DeLong via NANOG]

> On Jan 19, 2024, at 09:21, Charles Polisher  wrote:
> 
> Owen DeLong wrote:
> 
> > Some, but not a lot. In the case of the DTMF transition, the
> > network and handsets were all under the central control of a
> > single provider at a time when they could have forced the change
> > if they really wanted to. After all, nobody was going to cancel
> > their phone service altogether (or such a small fraction of
> > subscribers as to count as a rounding error anyway) over the
> > issue and AT could simply have shipped replacement phones
> > with instructions for returning the older phone and done a
> > retrofit operation if they really wanted to drive the transition.
> 
> True, yet there's a missing piece to that description: ROI.
> In the regulated environment with a mandated X% Return On Invest-
> ment (X ≈ 15 IIRC) a bigger expense pie was a better pie because
> a bigger expense pie meant a bigger return. This was an inexorable
> force that influenced every substantive decision. An expanding
> rate base was the One True Path to advancing against the demon
> competitors: AT and other RBOCs.

You’re missing the fact that this particular set of events predates the 
formation of RBOCS or competitors in general. There was AT, there was GTE, 
and there were a handful of other ILECs sprinkled around the country, but each 
had 100% territorial exclusivity and monopoly and AT at the time was pretty 
much the only LD carrier, period.

> In the Bell System setting, before and after Divestiture, a
> perpetual and costly migration from IPv4 to IPv6 with all the
> attendant cost burdens would have been well tolerated, even
> welcomed, in the "C Suite" anyways.

Absolutely, I’m actually surprised that the DTMF forced conversion and its 
attendant cost wasn’t foisted on the unsuspecting public, TBH. I really don’t 
understand how AT missed that opportunity. Sure, it would have lowered costs 
long term to a small extent, but the handset replacement process alone would 
have been a huge cost for several years.

Let’s face it, those old AT phones were rock solid and in a pinch you could 
use one as a forging hammer. ;-)

Owen

Re: IRR information & BYOIP (Bring Your Own IP) with Cloud Providers

[Owen DeLong via NANOG]

Sounds like you’ve got a weird mix of route origination. Why wouldn’t you 
advertise to Google via BGP and have your prefix originate from your own ASN?

Owen


> On Jan 19, 2024, at 02:39, kubanowy  wrote:
> 
> Hi,
> We have our own prefix assignment from ARIN. We have our infrastructure in 
> GCP (Google Cloud Platform) where we started using BYOIP functionality 
> (Google advertises our IPs). We followed their recommendation with ROA 
> configuration in ARIN 
> https://cloud.google.com/vpc/docs/bring-your-own-ip#live-migration-recommendations
>  but they don't mention if IRR (whois database) should be updated as well. 
> I've checked with their support and they said no additional changes need to 
> be done there.
> But currently we are in situation where ARIN's whois contains entry for our 
> prefix with our own ASN and Google advertised to RADb entry for our prefixes 
> with their own ASN. When we use online tools like 
> https://irrexplorer.nlnog.net/ or Cisco's CrossWork Cloud (former BGPmon), 
> they mark our prefixes due to mismatch of ASN in those 2 databases.
> We haven't observed any routing issues so far (i.e. ISP not importing our 
> prefixes), but we aim to sort this out for better credibility. I'm wondering 
> what's community approach for updating whois databases when using BYOIP 
> functionality with Cloud providers and if there is a risk of any potential 
> impact if we were to change information in ARIN.
> Thanks
> 

Re: okta probing

[Owen DeLong via NANOG]

I think this is a new form of dDOS and it is sometimes performed by Bots.

What they are achieving is annoying you. From your post, it appears to be 
working.

It’s also possible (depending on the account name) that it’s a mistype.

Owen


> On Jan 12, 2024, at 17:50, Randy Bush  wrote:
> 
> can someone explain what some child out there hopes to gain by
> repeatedly failing to authenticate to okta in my accound name?
> a couple of times a day, i have to take 40 seconds to unlock the
> account the kiddie has triggered.  seems silly as they do not
> have the 2fa.
> 
> it's -3c here, so i guess the clue level is going down as well
> as the temp.
> 
> randy

Re: Backward Compatibility Re: 202401100645.AYC Re: IPv4 address block

[Owen DeLong via NANOG]

> On Jan 15, 2024, at 09:37, Abraham Y. Chen  wrote:
> 
> Hi, Christopher"
> 
> 1)"  IPv6 is designed to replace IPv4.  ":
> Correct. But, this is not like Ten Commandments that God gave to his 
> children. Even such had not worked out in most cases. In real life, technical 
> backward compatibility is the only known approach to achieve graceful 
> replacement of the old. Failing to observe such discipline, one should not 
> blame others for the disappointment in the transition. I am an outsider to 
> the Internet development history. But, the overall appearance at this moment 
> is that somehow IPv6 design failed to properly execute the backward 
> compatibility requirement. So, IPv6 replacing IPv4 is not given.
> 

This isn’t entirely true… Cassette tapes were not particularly backwards 
compatible with LPs or 8-tracks. CDs were not backwards compatible with LPs, 
Casettes, or 8-tracks. iPods/etc. were not backwards compatible with any of the 
above.

USB-C is not backwards compatible with Lightning is not backwards compatible 
with Dock.

What I think has been shown is that the new needs to provide something 
compelling to the user being forced to migrate in order to motivate them to 
suffer the cost and inconvenience. Unfortunately, between NAT and Microsoft, 
instead of demand for an end-to-end network solution, we have consumers that 
have come to accept, nay expect the degraded level of service that is Windows 
and the Natternet that we have today. Application developers have all coded to 
this lowest possible state of network capability, and even written code which 
breaks absent NAT in some cases (I’m pointing at you Philips Hue).

For a little while, there was a bunch of free porn available on IPv6-only that 
some hoped would drive IPv6 adoption. Unfortunately, all it really drove was a 
large number of IPv4-only free porn sites.

Other apps that were supposed to be v6-only and thus drive adoption included 
IPSEC (rapidly back ported as a terrible hack on v4, not only reducing the 
incentive to migrate to v6, but giving IPSEC a horrible reputation for 
complexity and dysfunction in the process because of how hacky the v4 
implementation has to be) and DHCP-PD (which remains IPv6-only, but failure to 
put forth standard mechanisms for the DHCP server to communicate the necessary 
delegation data to the router that need to forward the delegated prefixes 
reduced the utility of that particular solution so far).

> 2)Allow me to share with you an almost parallel event in the PSTN, to 
> illustrate how tough is to achieve the replacement of a working service, even 
> under an environment with very strict backward compatibility disicpline:
> 
> A.The Decadic (rotary) Dialing (DD) technique worked well on the 
> telephone loop to a certain limit of distance for many years that enabled the 
> automatic telephone switching systems. But, it could not go beyond the CO 
> (Central Office). 
> 
> B.So, Bell Labs studied the use of DTMF (Dual Tone Multi-Frequency) 
> or commonly known as Touch-Tone as trademarked in US, etc. The work started 
> in mid 1940s. 
> 
> c.By late 1960s, working demos became available. During the 
> mid-1970s, it was deployed across the Bell System (and beyond upon requests 
> from other countries). 
> 
> D.With end-to-end signally capability of the DTMF signalling, a lot 
> of services such as remote purchase, airline status checking , etc., grew 
> drastically.
> 
> E.However, DTMF was a complete technology from Decadic Dialing and 
> most phones in the field were still the latter type. COs had to install dual 
> function line cards on the loop that subscriber liked to enjoy the DTMF 
> capability. Obviously, the dual function line cards costed more than that of 
> the basic DD version.
> 
> F.Initially, AT offered the DTMF service for free (well, covered by 
> the rental of the new phone) to encourage that adoption. Then, they raised 
> the monthly service rate for lines still requiring DD receiver hoping to 
> gracefully forcing the subscribes to wean from using DD phones. 
> 

Actually, I recall that if you wanted DTMF capability on your line, you had to 
pay extra for a time, then when they decided to deprecate DD, they dropped that 
surcharge. I don’t remember ever having to pay extra for DD, but I do remember 
getting notices telling me that they were turning off “pulse dialing” as of 
some particular date.

This led to amusing solutions like phones you could buy at Radio Shack and 
similar with an easily accessible switch that allowed you to call whatever 
service you wanted using pulse dialing, then flip the switch and use DTMF to 
talk to said service.
> G.Guess what, the inertia of the huge DD phones out there in the 
> field accumulated through near one century made the strategy impossible. That 
> is, many subscribers would rather to pay one extra dollar or so a month to 
> enjoy having the old DD 

Re: One Can't Have It Both Ways Re: Streamline the CG-NAT Re: EzIP Re: IPv4 address block

[Owen DeLong via NANOG]

Any host connected to a reasonably well peered ISP (e.g. NOT Cogent) with IPv6 
should be able to communicate with any other such host so long as the 
administrative policies on both sides permit it.

I have no difficulty directly reaching a variety of IPv6 hosts from the /48 in 
my home.

However, it’s not like dial-up modem operations in the PSTN in that IP is an 
inherently connectionless packet switched service while modems were an 
inherently circuit switched connection oriented service.

However, it does work like IPv4 used to work before NAT made everything 
horrible.

Owen


> On Jan 15, 2024, at 12:20, Abraham Y. Chen  wrote:
> 
> Hi, Forrest:
> 
> 1)I have a question: 
> 
> If I subscribe to IPv6, can I contact another similar subscriber to 
> communicate (voice and data) directly between two homes in private like the 
> dial-up modem operations in the PSTN? If so, is it available anywhere right 
> now?
> 
> Regards,
> 
> 
> Abe (2024-01-15 15:20)
> 
> 
>> Let me start with I think we're largely on the same page here.
>> 
>> The transition I see happening next is that the consumer traffic largely 
>> moves to IPv6 with no CG-NAT.  That is, if you're at home or on your phone 
>> watching video or doing social media or using whatever app is all the rage 
>> it's going to be over IPv6. 
>> 
>> My point was largely that I believe that at some point the big consumer (not 
>> business) focused companies are going to realize they can use market forces 
>> to encourage the remaining IPv4-only eyeball networks to transition to 
>> support IPv6 connections from their customers.  I don't know if the 
>> timeframe is next year or 20 years from now,  but I do know the tech 
>> companies are very good at looking at the costs of maintaining backwards 
>> compatibility with old tech and figuring out ways to shed those costs when 
>> they no longer make sense.  If they can utilize various forms of pressure to 
>> make this happen quicker, I fully expect them to do so. 
>> 
>> Inside a business network,  or even at home,  it wouldn't surprise me if 
>> we're both long gone before IPv4 is eradicated.   I know there is going to 
>> be a lot of IPv4 in my network for years to come just because of product 
>> lifecycles.   
>> 
>> As far as "CG-NAT-like" technologies go (meaning NAT in a provider's 
>> network), they're unfortunately going to be with us for a long time since 
>> customers seem to want to be able to reach everything regardless of the IPv4 
>> or IPv6 status of the customer or endpoint.   I also expect that most 
>> service providers with business customers are going to be carrying both IPv4 
>> and IPv6 for a long time, not to mention doing a fair bit of translation in 
>> both directions.  
>> 
>> I won't go deeply into the whole IPv4 vs IPv6 discussion for a business 
>> customer's "public address" because the topic is far too nuanced for an 
>> email to cover them accurately.   Suffice it to say that I don't disagree 
>> that business today largely wants IPv4, but some seem to be becoming aware 
>> of what IPv6 can do and are looking to have both options available to them, 
>> at least outside the firewall.
>> 
>> On Sat, Jan 13, 2024, 2:04 AM Brett O'Hara > > wrote:
>>> Ok you've triggered me on your point 2.  I'll address the elephant in the 
>>> room.
>>> 
>>> IPv4 is never ever going away.
>>> 
>>> Right now consumer services are mostly (mobile, wireless, landline, wide 
>>> generalization) are IPv6 capable.  Most consumer applications are ipv6 
>>> capable, Google, Facebook, etc.There is light at the very end of the tunnel 
>>> that suggests that one day we won't have to deploy CGNAT444 for our 
>>> consumers to get to content, we may only have to do NAT64 for them to get 
>>> to the remaining Ipv4 Internet.  We're still working hard on removing our 
>>> reliance on genuine ipv4 ranges to satisfy our customer needs, It's still a 
>>> long way off, but it's coming.
>>> 
>>> Here's the current problem.  Enterprise doesn't need ipv6 or want ipv6.  
>>> You might be able to get away with giving CGNAT to your consumers, but your 
>>> enterprise customer will not accept this. How will they terminate their 
>>> remote users?  How will they do B2B with out inbound NAT?  Yes, there are 
>>> solutions, but if you don't need to, why?  They pay good money, why can't 
>>> they have real ipv4?  All their internal networks are IPv4 rfc1918.  They 
>>> are happy with NAT.  Their application service providers are ipv4 only. 
>>> Looking at the services I access for work things like SAP, SerivceNow, 
>>> Office386, Sharepoint, Okta, Dayforce, Xero, and I'm sure many more, none 
>>> can not be accessed on ipv6 alone..  Their internal network lifecycle is 
>>> 10+ years.  They have no interest in trying new things or making new 
>>> technology work without a solid financial reason and there is none for them 
>>> implementing ipv6.   And guess where all the IP addresses we're 

Re: IPv6 Traffic Re: IPv6? Re: Where to Use 240/4 Re: 202401100645.AYC Re: IPv4 address block

[Owen DeLong via NANOG]

> On Jan 14, 2024, at 19:50, Abraham Y. Chen  wrote:
> 
> Hi, Ryan:
> 
> 1) " ... it accounts for 40% of the traffic at Google.   ":
> 
> Perhaps you were referring to the following?
> 
> https://www.google.com/intl/en/ipv6/statistics.html

> 
> 2)If so, your quotation is correct, except there are some hidden stories 
> below the surface:
> 
> A.When you Google for it with key words "IPv6 Traffic Google", the 
> first hit shows "IPv6 Adoption" that lead to the above. So, strictly 
> speaking, it is not traffic data that you are looking at.

Correct, that graph shows fraction of google unique end points that have IPv6 
capability. It does not reflect traffic at all.

> B.Above the actual graph, you will find statements, such as "  ...  
> the availability of IPv6 connectivity among Google users. " So, legally, 
> the graph is correct on its own right, but may not be exactly what you 
> thought. Reader be aware!

Correct… I do not know of a graph showing traffic as a percentage for google.

> It implies that the graph the IPv6 capability (equipment readiness) of 
> Google users, not necessarily the actual traffic they generate. The two do 
> not equate to each other.

No, it shows actual IPv6 reachable, not equipment capability. Likely there is 
some relatively close degree of correlation between fraction of users and 
fraction of traffic, but you are correct that they are independent numbers. 
It’s entirely possible, I suppose, that that 45% of endpoints  reachable via 
IPv6 represents 10% of Google traffic and doesn’t really use Google very much 
at all. OTOH, it’s equally likely that 45% of end points is actually 
responsible for 90% of Google traffic. I doubt that either of these extremes is 
likely, however.

In many ways, however, the fact that 45% of eyeball endpoints have IPv6 
reachability is much more meaningful than whatever random fraction of traffic 
they happen to represent.

>  
> 3)However, the above did seem to support what was generally said in the 
> public. Until, we found an interesting ongoing (the only one of such resource 
> that is updated about every ten minutes) statistics by AMS-IX (AMSterdam 
> Internet eXchange) :
> 
> https://stats.ams-ix.net/sflow/ipv6.html   
> 
> https://stats.ams-ix.net/sflow/ether_type.html
> a
> The second URL shows that IPv6 accounts for approximately 5.7% of the 
> overall Internet traffic that AMS-IX sees today. If one traces back through 
> the archived data, the earlier numbers were even much lower. In fact, those 
> graphs looked meaningless, because there was hardly any visible trace colored 
> for IPv6. This has been going on for at least the last one decade. So, it 
> could not be an error.

This isn’t a surprise since the vast majority of Google’s (and most other 
content providers) traffic is delivered via private network interconnect and 
not on public peering points.

> 4)We contacted AMS-IX for a possible explanation of the obvious 
> discrepancy. They politely referred us to our own ISPs. This triggered our 
> curiosity. We decided that we needed to find the full world-wide IPv6 traffic 
> data.
> 
> 5)There was an annual world-wide Internet traffic statistics and forecast 
> published by Cisco that stopped after 2017 (see URL below to the last issue). 
> We contacted Cisco in 2020 and got an eMail confirmation.
> 
> 
> https://cloud.report/Resources/Whitepapers/eea79d9b-9fe3-4018-86c6-3d1df813d3b8_white-paper-c11-741490.pdf

If you dig deeper on that, you’ll find that their data is purely estimated 
based on very limited collection.

> 6)However, there has never been any equivalent publication for the IPv6 
> by itself that we could locate.

There is an interesting bit of data from Akamai in this post:
https://www.akamai.com/blog/trends/10-years-since-world-ipv6-launch#:~:text=Akamai%27s%20IPv6%20traffic%20levels%20and%20client%20base=As%20of%20May%202022%2C%20Akamai%27s,years%20ago%20in%20February%202020.

Which reports that 2022 Akamai IPv6 traffic was over 41Tbps, up from just over 
1Gbps in 2012.

While IPv4 has grown in that same 10 years, I doubt that it has grown 
4,100,000% in that same 10 years.

> 7)In search for a possible explanation of the discrepancy between Pts. 1) 
> & 3), we came across the following article. In brief, it reported that the 
> Peering agreements among Internet backbone providers were less settled for 
> IPv6 than IPv4. Thus, higher percentage of IPv6 traffic than that of IPv4 
> should have been directed through the IXs (Internet eXchanges), such as 
> AMS-IX.
> 
> https://www.theregister.com/2018/08/28/ipv6_peering_squabbles/

1. This is largely untrue today. Most IPv6 capable networks that peer on a 
public exchange with another IPv6 capable network set up sessions for v4 and v6 
at the same time.

2. There’s a much more plausible explanation… Most of the big eyeball networks 
and most of the big content providers don’t 

Re: Stealthy Overlay Network Re: 202401100645.AYC Re: IPv4 address block

[Owen DeLong via NANOG]

Apologies, I failed to realize that we were still discussing the absurdity of 240/4 and thought we were talking IPv6. All my comments below apply to v6 progress. 240/4 remains in the who knows?/who cares? Category as far as I’m concerned. OwenOn Jan 12, 2024, at 22:51, Owen DeLong  wrote:Windows, Mac, Linux, bad are all done. Juniper, MikroTik, even Cisco are done.Most consumer routers sold in the last 2-3 years are done. Not sure what “hardware vendor” would be necessary beyond those. There are probably some little used corner cases of routers, os, etc. but for the most part, we are actually there, it’s just deployment at this point. What’s missing to some extent:Applications (some, less every day)Logging/monitoring/log parsing (some, getting better)Technical training (especially service providers)OwenOn Jan 12, 2024, at 10:02, Mike Hammett  wrote:" every networking vendor, hardware vendor, and OS vendor"How far are we from that, in reality? I don't have any intention on using the space, but I would like to put some definition to this boogey man.-Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISPFrom: "Ryan Hamel" To: "Abraham Y. Chen" , "Vasilenko Eduard" Cc: "Abraham Y. Chen" , nanog@nanog.orgSent: Thursday, January 11, 2024 11:04:31 PMSubject: Re: Stealthy Overlay Network Re: 202401100645.AYC Re: IPv4 address block




Abraham,


You may not need permission from the IETF, but you effectively need it from every networking vendor, hardware vendor, and OS vendor. If you do not have buy in from key stakeholders, it's dead-on arrival.



Ryan


From: NANOG  on behalf of Abraham Y. Chen 
Sent: Thursday, January 11, 2024 6:38:52 PM
To: Vasilenko Eduard 
Cc: Chen, Abraham Y. ; nanog@nanog.org 
Subject: Stealthy Overlay Network Re: 202401100645.AYC Re: IPv4 address block
 







Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments.







Hi, Vasilenko:


1)    ... These “multi-national conglo” has enough influence on the IETF to not permit it.":


    As classified by Vint Cerf, 240/4 enabled EzIP is an overlay network that may be deployed stealthily (just like the events reported by the RIPE-LAB). So, EzIP deployment does not need permission from the IETF.


Regards,




Abe (2024-01-11 21:38 EST)









On 2024-01-11 01:17, Vasilenko Eduard wrote:





>
It has been known that multi-national conglomerates have been using it without announcement.
This is an assurance that 240/4 would never be permitted for Public Internet. These “multi-national
 conglo” has enough influence on the IETF to not permit it.
Ed/



From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org]
On Behalf Of Abraham Y. Chen
Sent: Wednesday, January 10, 2024 3:35 PM
To: KARIM MEKKAOUI 

Cc: 
nanog@nanog.org; Chen, Abraham Y. 

Subject: 202401100645.AYC Re: IPv4 address block
Importance: High


 

Hi, Karim:


 


1)    If you have control of your own equipment (I presume that your business includes IAP - Internet Access Provider, since you are asking to buy IPv4 blocks.), you can get
 a large block of reserved IPv4 address for free by
disabling the program codes in your current facility that has been
disabling the use of 240/4 netblock. Please have a look at the below whitepaper. Utilized according to the outlined disciplines, this is a practically unlimited resources. It has
 been known that multi-national conglomerates have been using it without announcement. So, you can do so stealthily according to the proposed mechanism which establishes uniform practices, just as well. 


 


   

https://www.avinta.com/phoenix-1/home/RevampTheInternet.pdf


 


2)    Being an unorthodox solution, if not controversial, please follow up with me offline. Unless, other NANOGers express their interests.


 


 


Regards,


 


 


Abe (2024-01-10 07:34 EST)


 


 


 


On 2024-01-07 22:46, KARIM MEKKAOUI wrote:


Hi Nanog Community
 
Any idea please on the best way to buy IPv4 blocs and what is the price?
 
Thank you
 
KARIM

 

 

 







Virus-free.www.avast.com




 

Re: Stealthy Overlay Network Re: 202401100645.AYC Re: IPv4 address block

[Owen DeLong via NANOG]

Windows, Mac, Linux, bad are all done. Juniper, MikroTik, even Cisco are done.Most consumer routers sold in the last 2-3 years are done. Not sure what “hardware vendor” would be necessary beyond those. There are probably some little used corner cases of routers, os, etc. but for the most part, we are actually there, it’s just deployment at this point. What’s missing to some extent:Applications (some, less every day)Logging/monitoring/log parsing (some, getting better)Technical training (especially service providers)OwenOn Jan 12, 2024, at 10:02, Mike Hammett  wrote:" every networking vendor, hardware vendor, and OS vendor"How far are we from that, in reality? I don't have any intention on using the space, but I would like to put some definition to this boogey man.-Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISPFrom: "Ryan Hamel" To: "Abraham Y. Chen" , "Vasilenko Eduard" Cc: "Abraham Y. Chen" , nanog@nanog.orgSent: Thursday, January 11, 2024 11:04:31 PMSubject: Re: Stealthy Overlay Network Re: 202401100645.AYC Re: IPv4 address block




Abraham,


You may not need permission from the IETF, but you effectively need it from every networking vendor, hardware vendor, and OS vendor. If you do not have buy in from key stakeholders, it's dead-on arrival.



Ryan


From: NANOG  on behalf of Abraham Y. Chen 
Sent: Thursday, January 11, 2024 6:38:52 PM
To: Vasilenko Eduard 
Cc: Chen, Abraham Y. ; nanog@nanog.org 
Subject: Stealthy Overlay Network Re: 202401100645.AYC Re: IPv4 address block
 







Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments.







Hi, Vasilenko:


1)    ... These “multi-national conglo” has enough influence on the IETF to not permit it.":


    As classified by Vint Cerf, 240/4 enabled EzIP is an overlay network that may be deployed stealthily (just like the events reported by the RIPE-LAB). So, EzIP deployment does not need permission from the IETF.


Regards,




Abe (2024-01-11 21:38 EST)









On 2024-01-11 01:17, Vasilenko Eduard wrote:





>
It has been known that multi-national conglomerates have been using it without announcement.
This is an assurance that 240/4 would never be permitted for Public Internet. These “multi-national
 conglo” has enough influence on the IETF to not permit it.
Ed/



From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org]
On Behalf Of Abraham Y. Chen
Sent: Wednesday, January 10, 2024 3:35 PM
To: KARIM MEKKAOUI 

Cc: 
nanog@nanog.org; Chen, Abraham Y. 

Subject: 202401100645.AYC Re: IPv4 address block
Importance: High


 

Hi, Karim:


 


1)    If you have control of your own equipment (I presume that your business includes IAP - Internet Access Provider, since you are asking to buy IPv4 blocks.), you can get
 a large block of reserved IPv4 address for free by
disabling the program codes in your current facility that has been
disabling the use of 240/4 netblock. Please have a look at the below whitepaper. Utilized according to the outlined disciplines, this is a practically unlimited resources. It has
 been known that multi-national conglomerates have been using it without announcement. So, you can do so stealthily according to the proposed mechanism which establishes uniform practices, just as well. 


 


   

https://www.avinta.com/phoenix-1/home/RevampTheInternet.pdf


 


2)    Being an unorthodox solution, if not controversial, please follow up with me offline. Unless, other NANOGers express their interests.


 


 


Regards,


 


 


Abe (2024-01-10 07:34 EST)


 


 


 


On 2024-01-07 22:46, KARIM MEKKAOUI wrote:


Hi Nanog Community
 
Any idea please on the best way to buy IPv4 blocs and what is the price?
 
Thank you
 
KARIM

 

 

 







Virus-free.www.avast.com




 

Re: IPv6? Re: Where to Use 240/4 Re: 202401100645.AYC Re: IPv4 address block

[Owen DeLong via NANOG]

Frankly, I care less. No matter how you use whatever IPv4 space you attempt to cajole into whatever new form of degraded service, the simple fact remains. IPv4 is a degraded technology that only continues to get worse over time. NAT was bad. CGNAT is even worse (and tragically does nothing to eliminate consumer NAT, just layers more disaster on top of the existing mess). The only currently available end to end peer to peer technology, for better or worse, is IPv6. Despite its naysayers, it is a proven technology that has been shouldering a significant fraction of internet traffic for many years now and that fraction continues to grow.You simply can’t make IPv4 adequate and more hackers to extend its life merely expands the amount of pain and suffering we must endure before it is finally retired. OwenOn Jan 12, 2024, at 03:46, Abraham Y. Chen  wrote:

  

  
  
Hi, Ryan:

   
1)   " ...  Save
yourself the time and effort on this and implement IPv6.    ":

   
    What is your selling
point?

  

  
Regards,

  

  
Abe (2024-01-12 06:44)









2024-01-11 12:39, Ryan Hamel wrote:


  
  Abraham,
  
  
  You're arguing semantics instead of the actual
point. Residential
  customers want Internet access, not intranet access.
Again, VRFs are plentiful and so are CG-NAT firewall appliances
or servers to run those VMs. 
  
  
  Save yourself the time and effort on this and
implement IPv6.
  


Ryan
  
  

From:
  NANOG 
  on behalf of Abraham Y. Chen 
  Sent: Thursday, January 11, 2024 9:24:18 AM
  To: Michael Butler 
  Cc: nanog@nanog.org 
  Subject: Where to Use 240/4 Re:
  202401100645.AYC Re: IPv4 address block



  

  
  
  
Caution:
  This is an external email and may be malicious. Please
  take care when clicking links or opening attachments.

  

  



  Hi, Michael:
  

  1)    " ... While
  you may be able to get packets from point A to B in a
  private setting, using them might also be .. a challenge.
  ...   ":
  

      EzIP uses
  240/4 netblock only within the RAN (Regional Area Network)
as "Private" address, not "publicly" routable, according to the
  conventional Internet definition. This is actually the
  same as how 100.64/10 is used within CG-NAT. 
  

  2)    However,
  this might be where the confusion comes from. With the
  geographical area coverage so much bigger, an RAN is
  effectively a public network. To mesh the two for
  consistency, we defined everything related to 240/4 as
  "Semi-Public" to distinguish this new layer of networking
  facility from the current public / private separation.
  That is, the CG-NAT routers will become SPRs (Semi-Public
  Routers) in EzIP's RAN, once the 240/4 is deployed.

  

  Hope this helps,
  

  

  Abe (2024-01-11
  12:21)
  
   
  
  
  
  
  
  On 2024-01-10 10:45, Michael
Butler via NANOG wrote:
  
  On 1/10/24 10:12, Tom Beecher wrote: 
Karim- 
  
  Please be cautious about this advice, and understand the
  full context. 
  
  240/4 is still classified as RESERVED space. While you
  would certainly be able to use it on internal networks if
  your equipment supports it, you cannot use it as
  publicly routable space. There have been many proposals
  over the years to reclassify 240/4, but that has not
  happened, and is unlikely to at any point in the
  foreseeable future. 


While you may be able to get packets from point A to B in a
private setting, using them might also be .. a challenge. 

There's a whole bunch of software out there that makes
certain assumptions about allowable ranges. That is, they've
been compiled with a header that defines .. 

#define IN_BADCLASS(i)    (((in_addr_t)(i) & 0xf000)
== 0xf000) 

Michael 

  
  
  
 

Re: 202401100645.AYC Re: IPv4 address block

[Owen DeLong via NANOG]

At the time this was being considered, ARIN, APNIC, and RIPE NCC were each 
burning approximately 6 /8s per year. 240/4 is 16x/8, so with an RIR burn rate 
of 18 /8s per year (not counting LACNIC and AFRINIC which each accounted for <1 
per year IIRC), it seemed the 240/4 lasting a year was an optimistic count.

Owen


> On Jan 11, 2024, at 13:15, Christopher Hawker  wrote:
> 
> Hi Tom,
> 
> I'm not too sure I understand where the idea came from that 2 x /8 would only 
> last one year. APNIC received their final allocation of the 103/8 prefix from 
> ICANN/IANA on 03 February 2011, and commenced delegating space from the 
> prefix on 18 April 2011. With the right policies in place, it can be well and 
> truly extended.
> 
> Looking at an APNIC Blog article authored by Guangliang Pan on 09 October 
> 2023 (https://blog.apnic.net/2023/10/09/nearing-the-end-of-103-8/), as of the 
> time the article was written there were 121 available /24 prefixes from the 
> 103/8 pool still available. Not a great deal in the grand scheme of things, 
> however, it demonstrates that policy works in preserving what finite 
> resources we have left, and that a 2 x /8 will last a lot longer than one 
> year.
> 
> I could say the same, that 2 x /8 lasting a little more than a year is an 
> extremely remote and highly unlikely assumption. Bear in mind that APNIC 
> policy was changed 1/2 way through to drop 103/8 delegations from a /22 to a 
> /23. Based on this, 65,536 x /23 delegations can be made to new members and 
> as Peter said, if post-exhaustion policy is applied to 240/4 it'll go an 
> extremely long way.
> 
> Regards,
> Christopher Hawker
> 
> 
> 
> On Fri, 12 Jan 2024 at 04:26, Tom Beecher  > wrote:
>> Christopher-
>> 
>>> Reclassifying this space, would add 10+ years onto the free pool for each 
>>> RIR. Looking at the APNIC free pool, I would estimate there is about 1/6th 
>>> of a /8 pool available for delegation, another 1/6th reserved. 
>>> Reclassification would see available pool volumes return to pre-2010 levels.
>> 
>> Citing Nick Hilliard from another reply, this is an incorrect statement. 
>> 
>>> on this point: prior to RIR depletion, the annual global run-rate on /8s
>>> measured by IANA was ~13 per annum. So that suggests that 240/4 would
>>> provide a little more than 1Y of consumption, assuming no demand
>>> back-pressure, which seems an unlikely assumption.
>> 
>>> I share Dave's views, I would like to see 240/4 reclassified as unicast 
>>> space and 2 x /8s delegated to each RIR with the /8s for AFRINIC to be held 
>>> until their issues have been resolved.
>> 
>> This has been discussed at great length at IETF. The consensus on the 
>> question has been consistent for many years now; doing work to free up 
>> 12-ish months of space doesn't make much sense when IPv6 exists, along with 
>> plenty of transition/translation mechanisms. Unless someone is able to 
>> present new arguments that change the current consensus, it's not going to 
>> happen. 
>> 
>> On Thu, Jan 11, 2024 at 5:54 AM Christopher Hawker > > wrote:
>>> There really is no reason for 240/4 to remain "reserved". I share Dave's 
>>> views, I would like to see 240/4 reclassified as unicast space and 2 x /8s 
>>> delegated to each RIR with the /8s for AFRINIC to be held until their 
>>> issues have been resolved.
>>> 
>>> Reclassifying this space, would add 10+ years onto the free pool for each 
>>> RIR. Looking at the APNIC free pool, I would estimate there is about 1/6th 
>>> of a /8 pool available for delegation, another 1/6th reserved. 
>>> Reclassification would see available pool volumes return to pre-2010 levels.
>>> 
>>> https://www.apnic.net/manage-ip/ipv4-exhaustion/
>>> 
>>> In the IETF draft that was co-authored by Dave as part of the IPv4 Unicast 
>>> Extensions Project, a very strong case was presented to convert this space.
>>> 
>>> https://www.ietf.org/archive/id/draft-schoen-intarea-unicast-240-00.html
>>> 
>>> Regards,
>>> Christopher Hawker
>>> 
>>> On Thu, 11 Jan 2024 at 20:40, Dave Taht >> > wrote:
 On Wed, Jan 10, 2024 at 11:06 AM Tom Beecher >>> > wrote:
 >>
 >> There's a whole bunch of software out there that makes certain
 >> assumptions about allowable ranges. That is, they've been compiled with
 >> a header that defines ..
 >
 >
 > Of course correct. It really depends on the vendor / software / versions 
 > in an environment. A lot of vendors removed that years ago, because 
 > frankly a lot of large networks have been using 240/4 as pseudo RFC1918 
 > for years. Others have worked with smaller vendors and open source 
 > projects to do the same.
 >
 > It's consistently a topic in the debates about 240/4 reclassification.
 
 There's debates still? I gave up. After making 240/4 and 0/8 work
 across all of linux and BSD and all the 

Re: 202401100645.AYC Re: IPv4 address block

[Owen DeLong via NANOG]

ARIN has been gutting IPv4 free-pool based policy left and right lately… Other 
RIRs have not been quite as aggressive, but have done some similar things. 
This, if for no other reason, makes it a bad idea to suddenly restore RIR IPv4 
free pools.

Just my $0.02.

I’ve got as little power in the IETF as it’s possible to have, but I admit I do 
share in the consensus view that the effort spent writing up a plan for 240/4 
would be better invested in deploying IPv6.

Owen


> On Jan 11, 2024, at 13:05, Matthew Petach  wrote:
> 
> 
> 
> On Thu, Jan 11, 2024 at 9:29 AM Tom Beecher  > wrote:
>> Christopher-
>> 
>>> Reclassifying this space, would add 10+ years onto the free pool for each 
>>> RIR. Looking at the APNIC free pool, I would estimate there is about 1/6th 
>>> of a /8 pool available for delegation, another 1/6th reserved. 
>>> Reclassification would see available pool volumes return to pre-2010 levels.
>> 
>> Citing Nick Hilliard from another reply, this is an incorrect statement. 
>> 
>>> on this point: prior to RIR depletion, the annual global run-rate on /8s
>>> measured by IANA was ~13 per annum. So that suggests that 240/4 would
>>> provide a little more than 1Y of consumption, assuming no demand
>>> back-pressure, which seems an unlikely assumption.
> 
> 
> Hi Tom,
> 
> I think that's a bit of an unfair categorization--we can't look at 
> pre-exhaustion demand numbers and extrapolate to post-exhaustion allocations, 
> given the difference in allocation policies pre-exhaustion versus 
> post-exhaustion.
> 
> If we limited ISPs to a single /22 of post-exhaustion space, with a minimum 1 
> year waiting period to come back to request an additional /22, 240/4 would 
> last a good long time.
> That aligns with ARIN's current NPRM initial allocation, post-exhaustion:
> 4.2.2. Initial Allocation to ISPs
> All ISP organizations without direct assignments or allocations from ARIN 
> qualify for an initial allocation of up to a /22, subject to ARIN’s minimum 
> allocation size.
> 
> 
> If you already *have* existing IPv4 space, I would propose you be ineligible 
> to apply to ARIN for space from within 240/4; you already have a functioning 
> business with some amount of IPv4 space, and can look at either trying to be 
> more efficient with what you have (more CG-NAT, renumber off public space for 
> internal links, etc.), or participating in the open market for IPv4 space 
> transfers.
> 
> 240/4 can be made to last a very long time, if we apply post-exhaustion 
> rules, rather than allowing pre-exhaustion demand curves to continue forward.
> 
> 
>>> I share Dave's views, I would like to see 240/4 reclassified as unicast 
>>> space and 2 x /8s delegated to each RIR with the /8s for AFRINIC to be held 
>>> until their issues have been resolved.
>> 
>> This has been discussed at great length at IETF. The consensus on the 
>> question has been consistent for many years now; doing work to free up 
>> 12-ish months of space doesn't make much sense when IPv6 exists, along with 
>> plenty of transition/translation mechanisms. Unless someone is able to 
>> present new arguments that change the current consensus, it's not going to 
>> happen. 
> 
> The key difference is that IPv6-only doesn't (currently) work, 
> transition/translation mechanisms require an entity to have at least *some* 
> IPv4 addresses to anchor their transition/translation mechanisms to, and 
> we've created a situation that presents significant barriers to entry for new 
> applicants that existing entities don't face.  At some point in the near 
> future, I suspect governments will begin to look at the current ISP 
> environment as anti-competitive if we don't adjust our stance to ensure a 
> fair and level playing field for new entrants as well as existing incumbent 
> providers.  I think we're going to need to ensure that new applicants are 
> able to get their initial allocation of space for the foreseeable future in 
> order to fend off increasing regulatory pressure.  Adding space from 240/4 to 
> the initial-allocations-only pool would help ensure that.
> 
>  
>> 
>> On Thu, Jan 11, 2024 at 5:54 AM Christopher Hawker > > wrote:
>>> There really is no reason for 240/4 to remain "reserved". I share Dave's 
>>> views, I would like to see 240/4 reclassified as unicast space and 2 x /8s 
>>> delegated to each RIR with the /8s for AFRINIC to be held until their 
>>> issues have been resolved.
>>> 
>>> Reclassifying this space, would add 10+ years onto the free pool for each 
>>> RIR. Looking at the APNIC free pool, I would estimate there is about 1/6th 
>>> of a /8 pool available for delegation, another 1/6th reserved. 
>>> Reclassification would see available pool volumes return to pre-2010 levels.
>>> 
>>> https://www.apnic.net/manage-ip/ipv4-exhaustion/
>>> 
>>> In the IETF draft that was co-authored by Dave as part of the IPv4 Unicast 
>>> Extensions Project, a very strong case was 

Re: Interesting Ali Express web server behavior...

[Owen DeLong via NANOG]

Given micro services and VM architectures these days, it’s not even difficult 
to imagine a company as large as Ali Baba burning through more than 17 milllion 
hosts.

Owen


> On Dec 10, 2023, at 00:08, Christopher Hawker  wrote:
> 
> Starting to digress here for a minute...
> 
> How big would a network need to get, in order to come close to exhausing 
> RFC1918 address space? There are a total of 17,891,328 IP addresses between 
> the 10/8 prefix, 172.16/12 space and 192.168/16 space. If one was to allocate 
> 10 addresses to each host, that means it would require 1,789,132 hosts to 
> exhaust the space.
> 
> - Christopher H.
> 
> On Sun, 10 Dec 2023 at 18:45, Sabri Berisha  <mailto:sa...@cluecentral.net>> wrote:
>> - On Dec 9, 2023, at 9:55 PM, Owen DeLong via NANOG nanog@nanog.org 
>> <mailto:nanog@nanog.org> wrote:
>> 
>> Hi,
>> 
>> > Location: http://33.3.37.57/
>> 
>> > But why would AliExpress be redirecting to DDN space? Is this legitimate? 
>> > Ali
>> > hoping to get away with squatting, or something else?
>> 
>> Not very long ago I worked for a well-known e-commerce platform where we 
>> nearly
>> ran out of RFC1918 space. We seriously considered using what was then
>> un-advertised DOD space to supplement RFC1918 space inside our data centers.
>> 
>> Perhaps AliExpress did get to that level of desperateness?
>> 
>> Thanks,
>> 
>> Sabri

Re: Interesting Ali Express web server behavior...

[Owen DeLong via NANOG]

No, in this case, I was using HE uplink from the Cabinet in FMT2 for testing 
using my AS1734 space 192.159.10.0/24 as source address.

Owen


> On Dec 9, 2023, at 23:51, Mike Lyon  wrote:
> 
> I notice a weird issue like this with Alibaba when i try to use my Comcast 
> connection. Turn my wifi off and now it works flawlessly.
> 
> Are you using your comcast connection?
> 
> -Mike
> 
>> On Dec 9, 2023, at 21:17, Stephane Bortzmeyer  wrote:
>> 
>> On Sat, Dec 09, 2023 at 09:55:31PM -0800,
>> Owen DeLong via NANOG  wrote
>> a message of 1136 lines which said:
>> 
>>> But why would AliExpress be redirecting to DDN space? Is this
>>> legitimate? Ali hoping to get away with squatting, or something
>>> else?
>> 
>> No idea. The IP address does not reply to HTTP requests, anyway. A
>> practical joke?
>> 
>> Note that this redirection takes places only when there is no
>> User-Agent field. If you say 'User-Agent: Mozilla', you get a proper
>> redirection, in my case to https://fr.aliexpress.com/.

Interesting Ali Express web server behavior...

[Owen DeLong via NANOG]

So I’m having trouble connecting to the Ali Express web server this evening and 
decided to investigate a little.

What I found surprised me…

owen@odmbpro3-3 ~ % openssl s_client -connect www.aliexpress.com:443
CONNECTED(0005)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global 
Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
verify return:1
depth=0 C = CN, ST = \E6\B5\99\E6\B1\9F\E7\9C\81, L = 
\E6\9D\AD\E5\B7\9E\E5\B8\82, O = Alibaba Cloud Computing Ltd., CN = 
ae01.alicdn.com
verify return:1
… certificate stuff, blah blah from Akamai, routine…
SSL-Session:
Protocol  : TLSv1.3
Cipher: AEAD-CHACHA20-POLY1305-SHA256
Session-ID: 
Session-ID-ctx: 
Master-Key: 
Start Time: 1702187128
Timeout   : 7200 (sec)
Verify return code: 0 (ok)
---
read R BLOCK
read R BLOCK
GET / HTTP/1.1
Host: www.aliexpress.com

HTTP/1.1 302 Moved Temporarily
Content-Type: text/html
Content-Length: 258
Location: http://33.3.37.57/
Access-Control-Allow-Origin: https://hz.aliexpress.com
Server: Tengine/Aserver
EagleEye-TraceId: 2103253917021871367418570ec8e3
Strict-Transport-Security: max-age=31536000
Timing-Allow-Origin: *
Date: Sun, 10 Dec 2023 05:45:36 GMT
Connection: keep-alive
Set-Cookie: ali_apache_id=33.3.37.57.1702187136742.612980.2; path=/; 
domain=.aliexpress.com; expires=Wed, 30-Nov-2084 01:01:01 GMT
Server-Timing: cdn-cache; desc=MISS
Server-Timing: edge; dur=65
Server-Timing: origin; dur=3
Server-Timing: ak_p; 
desc="1702187128314_400069768_2521981097_6837_5323_25_43_-";dur=1



302 Found

302 Found
The requested resource resides temporarily under a different URI.
Powered by Tengine

… OK, so far so good, though the hard coded IP redirect is a bit odd. 
Especially when you consider this:

NetRange:   33.0.0.0 - 33.255.255.255
CIDR:   33.0.0.0/8
NetName:DISN-IP-LEGACY
NetHandle:  NET-33-0-0-0-1
Parent:  ()
NetType:Direct Allocation
OriginAS:   
Organization:   DoD Network Information Center (DNIC)
RegDate:1991-01-01
Updated:2013-09-11
Ref:https://rdap.arin.net/registry/ip/33.0.0.0


OrgName:DoD Network Information Center
OrgId:  DNIC
Address:3990 E. Broad Street
City:   Columbus
StateProv:  OH
PostalCode: 43218
Country:US
RegDate:
Updated:2011-08-17
Ref:https://rdap.arin.net/registry/entity/DNIC


OrgTechHandle: REGIS10-ARIN
OrgTechName:   Registration
OrgTechPhone:  +1-844-347-2457 
OrgTechEmail:  disa.columbus.ns.mbx.arin-registrati...@mail.mil
OrgTechRef:https://rdap.arin.net/registry/entity/REGIS10-ARIN

OrgAbuseHandle: REGIS10-ARIN
OrgAbuseName:   Registration
OrgAbusePhone:  +1-844-347-2457 
OrgAbuseEmail:  disa.columbus.ns.mbx.arin-registrati...@mail.mil
OrgAbuseRef:https://rdap.arin.net/registry/entity/REGIS10-ARIN

OrgTechHandle: MIL-HSTMST-ARIN
OrgTechName:   Network DoD
OrgTechPhone:  +1-844-347-2457 
OrgTechEmail:  disa.columbus.ns.mbx.hostmaster-dod-...@mail.mil
OrgTechRef:https://rdap.arin.net/registry/entity/MIL-HSTMST-ARIN

Which seems in line with the announcement of that address I’m seeing:

owen@delong-fmt2-mx-01> show route 33.3.37.57 

inet.0: 947480 destinations, 2018685 routes (947480 active, 0 holddown, 0 
hidden)
+ = Active Route, - = Last Active, * = Both

33.0.0.0/8 *[BGP/170] 1w6d 05:39:58, localpref 2000
  AS path: 6939 3356 749 I, validation-state: unverified
>  to 64.71.131.26 via ge-2/0/0.0
[BGP/170] 1w6d 05:35:29, localpref 100, from 192.124.40.252
  AS path: 36236 2914 3356 749 I, validation-state: 
unverified
>  via gr-2/3/0.70

(AS749 is also DISA/DDI)

But why would AliExpress be redirecting to DDN space? Is this legitimate? Ali 
hoping to get away with squatting, or something else?

Owen

Re: Outside plant - prewire customer demarc preference

[Owen DeLong via NANOG]

> On Nov 28, 2023, at 07:27, Brandon Martin  wrote:
> 
> On 11/27/23 18:52, owen--- via NANOG wrote:
>> Why would 1” be significantly harder to get than 3/4”? Both in EMT and PVC, 
>> it’s readily available to the best of my knowledge.
> 
> Most residential electrical contractors are going to use rated ENT since it's 
> what they can easily get at a normal supply house or even home center.  This 
> is the stuff made popular by Carlon in their trademark blue that makes people 
> call it "smurf tube".  It's readily available in 1/2" and 3/4" everywhere 
> from home centers to basically any supply house.  1" is less common - most 
> home centers don't have it, and since it isn't normally needed in residential 
> nor is ENT basically ever used in commercial, neither do a lot of strictly 
> electrical supply houses.

I’ve never used ENT (never even seen that name, TBH). 1” EMT is readily 
available at Home Depot and Lowes out here as well as several reputable supply 
houses.

> You can of course get corrugated communication duct in that size, often with 
> mule tape pre-installed as a bonus, but a lot of electrical supply houses 
> that deal only in "electrical" and not "communications" don't have that and 
> will send folks to a "low voltage communication supplier" or have to special 
> order it.  A lot of electrical contractors, especially residential, would 
> rather not bother with a separate supply run, and having to wait is a pain if 
> it wasn't planned early on and also often means you're paying freight 
> separately.

Agreed, but out here at least, almost all of the electricians don’t bother with 
ENT and most do both commercial and residential work, so they just keep EMT on 
the trucks and usually have up to 2” readily available, with 3” and 4” in the 
not particularly difficult, but too bulky to carry unless needed category. I 
guess this may vary with locality.

> Even then, most of the folks going to a communications supplier are going to 
> reach straight for 1.25" or larger in commercial since you don't have to 
> worry about drilling studs.  As I recall, my local communication cable house 
> didn't even have 1" in stock, but they did have 1.25" in stock, and their 
> price was basically the same as the 1" as a result.

Sure, but smurf tubing is a hole other thing. Amusingly, at least locally, 
smooth interduct is easier to find than corrugated smurf tubing, though both 
are readily available.

> So it's not that it doesn't exist or anything, it's just that, at least as 
> far as I've seen, there's not a ton of demand for it, so local stock is thin.

I guess this varies by locality.

> All that said, I just checked Home Depot, and they are showing some stock on 
> 1" ENT at my local store, so maybe that's changing.  Used to be they only 
> stocked 3/4" and 1/2".  They still only have 25ft hanks of the 1" stuff (you 
> can get 1/2" and 3/4" in 25 or 100-200ft), but at least they have it now.  It 
> is still about 4x the price of 3/4" per unit length, though.

Interesting… ENT is apparently plastic and has interesting snap fittings. Until 
this email, I’ve never even looked into it. Used plenty of the “ENT” boxes, but 
always just called them PVC (since that’s what the ENT stuff is apparently made 
of). EMT is way more common out here than ENT, and even where plastic is used, 
most seem to use straight electrical PVC (grey stuff usually) instead of of the 
ENT brand stuff.

YMMV of course.

Owne

Re: ipv6 address management - documentation

[Owen DeLong via NANOG]

Spreadsheets are terrible for IPAM regardless of address length, but I am 
curious to know why you think IPv6 would be particularly worse than IPv4 in 
such a scenario?

Owen


> On Nov 16, 2023, at 10:02, Aaron Gould  wrote:
> 
> For years I've used an MS Excel spreadsheet to manage my IPv4 addresses.  
> IPv6 is going to be maddening to manage in a spreadsheet.  What does everyone 
> use for their IPv6 address prefix management and documentation?  Are there 
> open source tools/apps for this?
> 
> -- 
> -Aaron

Re: Am I the only one who thinks this is disconcerting?

[Owen DeLong via NANOG]

It can’t be legacy space, there is no such thing in IPv6.

Legacy status only refers to IPv4 blocks that were issued by the predecessors 
to the current registry system and have not yet been placed under RIR contract.

Owen


> On Nov 13, 2023, at 12:57, Matt Corallo  wrote:
> 
> I'd be very curious to see a lawsuit over an IP hijack that isn't interfering 
> with the operation of any of Cogent's services and is restoring service to 
> HE's customers. Doubly so if they prepend aggressively to avoid it being a 
> preferred path (Cogent currently announces a /48 for the C root server, and I 
> assume there's ~nothing else in that block, but dunno).
> 
> IANAL and really have no idea what the basis for that would be? I guess if 
> its legacy space you might argue its property and theft?
> 
> Matt
> 
> On 11/13/23 12:38 PM, Ryan Hamel wrote:
>> Matt,
>> Why would HE hijack Cogent's IP space? That would end in a lawsuit and 
>> potentially even more de-peering between them.
>> Ryan Hamel
>> 
>> *From:* NANOG  on behalf of Matt 
>> Corallo 
>> *Sent:* Monday, November 13, 2023 11:32 AM
>> *To:* Bryan Fields ; nanog@nanog.org 
>> *Subject:* Re: Am I the only one who thinks this is disconcerting?
>> Caution: This is an external email and may be malicious. Please take care 
>> when clicking links or opening attachments.
>> On 11/8/23 2:23 PM, Bryan Fields wrote:
>>> On 11/8/23 2:25 PM, o...@delong.com wrote:
 Seems irresponsible to me that a root-server (or other critical DNS 
 provider) would engage in a
 peering war to the exclusion of workable DNS.
>>> 
>>> I've brought this up before and the root servers are not really an IANA 
>>> function IIRC.  There's not
>>> much governance over them, other than what's on root-servers.org.  I think 
>>> a case could be made that
>>> C is in violation of the polices on that page and RFC 7720 section 3.
>>> 
>>> Basically none of the root servers want to change this and thus it's never 
>>> going to change.  DNS
>>> will fail and select another to talk to, and things will still work.
>> At what point does HE just host a second C root and announce the same IPv6s? 
>> Might irritate Cogent,
>> but its not more "bad" than Cogent failing to uphold the requirements for 
>> running a root server.
>> Matt

Am I the only one who thinks this is disconcerting?

[Owen DeLong via NANOG]

https://dnsviz.net/d/10.159.192.in-addr.arpa/dnssec/


Seems to report a bunch of errors in the DS records for 192.in-addr.arpa held 
in the in-addr.arpa zone.

I figured I’d wait a few days and try again the first few times I encountered 
this, but it’s persisted for more than two weeks now.

Owen

Re: [EXTERNAL] Charter DNS servers returning malware filtered IP addresses

[Owen DeLong via NANOG]

> On Oct 30, 2023, at 07:58, Livingood, Jason  
> wrote:
> 
> On 10/27/23, 19:01, "NANOG on behalf of Owen DeLong wrote:
> 
>> If it’s such a reasonable default, why don’t any of the public resolvers 
>> (e.g. 1.1.1.1, 8.8.8.8, 9.9.9.9, etc.) do so?
>> DNS isn’t the right place to attack this, IMHO.
> 
> Are we sure that the filtering is done in the default view - I would suggest 
> the user check to ensure they don't have a filtering service (e.g. parental 
> controls/malware protection) turned on. In my **personal** opinion, the 
> default view should have DNSSEC validation & no filtering; users can always 
> optionally select additional protection services that might include DNS-based 
> filtering as well as other mechanisms. 
> 
> JL
> 

Looks like 9.9.9.9 is filtered but ONLY for actual verified security threats, 
not spam, etc.
If you want unfiltered, they offer 9.9.9.10.

Cloudflare offers two different filtered services, but 1.1.1.1 remains 
unfiltered.

1.1.1.2 is “No Malware”
1.1.1.3 is “No Malware or Adult Content”

So yes, apparently one (and only one) public resolver now filters by default.

I stand by my statement… It should be an opt-in choice, not a default.

Owen

Re: Charter DNS servers returning malware filtered IP addresses

[Owen DeLong via NANOG]

>> DNS isn’t the right place to attack this, IMHO.
> 
> Why not (apart from a purity argument), and where should it happen instead? 
> As others pointed out, network operators have a vested interest in protecting 
> their customers from becoming victims to malware.


Takedowns of the hostile target sites.

You dismiss the purity argument, but IMHO, there’s merit to the purity argument.

Any such DNS filtration, if provided, should be provided on an opt-in basis, 
not as a default.

I’ve seen plenty of situations where the filters were just plain wrong and if 
the end user didn’t actively choose that filtration, the target site may be 
victimized without anyone knowing where to go to complain.

Owen

Re: [EXTERNAL] Charter DNS servers returning malware filtered IP addresses

[Owen DeLong via NANOG]

> On Oct 27, 2023, at 14:20, John Levine  wrote:
> 
> It appears that Bryan Fields  said:
>> -=-=-=-=-=-
>> -=-=-=-=-=-
>> On 10/27/23 7:49 AM, John Levine wrote:
>>> But for obvious good reasons,
>>> the vast majority of their customers don't
>> 
>> I'd argue that as a service provider deliberately messing with DNS is an 
>> obvious bad thing.  They're there to deliver packets.
> 
> For a network feeding a data center, sure. For a network like
> Charter's which is feeding unsophisticated nontechnical users, they
> need all the messing they can get.
> 
> If you're one of the small minority of retail users that knows enough
> about the technology to pick your own resolver, go ahead.  But it's
> a reasonable default to keep malware out of Grandma's iPad.
> 
> R's,
> John

If it’s such a reasonable default, why don’t any of the public resolvers (e.g. 
1.1.1.1, 8.8.8.8, 9.9.9.9, etc.) do so?

DNS isn’t the right place to attack this, IMHO.

Owen

Re: RPKI unknown for superprefixes of existing ROA ?

[Owen DeLong via NANOG]

In fairness, however, there is a natural tendency for many of those PNIs to be 
built in locations
in common with IXPs and often they start as IXP connections and with growth of 
traffic end up
migrating to PNIs for further expansion.

Owen


> On Oct 24, 2023, at 18:15, Randy Bush  wrote:
> 
>> Believe it or not, Job, there are parts of the internet that exchange
>> traffic and move packets that are not IXPs.
> 
> in fact, measurements had shown that the majority of inter-domain
> traffic is over pnis
> 
> randy

Re: RPKI unknown for superprefixes of existing ROA ?

[Owen DeLong via NANOG]

Yes, but we weren’t talking about an IXP here.

We’re talking about an ISP.

Believe it or not, Job, there are parts of the internet that exchange traffic 
and move packets that are not IXPs.

Owen


> On Oct 22, 2023, at 11:48, Job Snijders via NANOG  wrote:
> 
> On Sun, 22 Oct 2023 at 20:33, Tom Beecher  > wrote:
>>> Basically, I guess, it means that the AS 0 solution shouldn't be used, at 
>>> least not usually.
>> 
>> It's like everything else. Understand what the tools do and what they don't 
>> do, and use them appropriately. 
> 
> 
> A primary risk for an IXP is the existence of a more-specific of the IX 
> peering LAN prefix, a less-specific wouldn’t matter or inflict damage.
> 
> So in the above context an AS 0 ROAs can be useful to improve protection of 
> IXP Peering LANs where the IX operator doesn’t want the fabric to be globally 
> reachable - and one of the IX participants failed to correctly EBGP in/out 
> policies.
> 
> Kind regards,
> 
> Job

Re: RPKI unknown for superprefixes of existing ROA ?

[Owen DeLong via NANOG]

The covering /20 isn’t his to announce… He has a /22. He’s announcing all 4 
/24s, and may not have a legitimate place to announce the covering /22, which 
wouldn’t help in this case anyway.

So I’m not sure why you think that’s a solution.

Owen


> On Oct 22, 2023, at 10:45, Tom Beecher  wrote:
> 
>> Look again, Tom. This is an attack vector using a LESS specific route. The 
>> /22 gets discarded, but a covering /0-/21 would not. 
> 
> Yes. And reliant on the operator doing something exceptionally not smart to 
> begin with.  Relying on an AS0 ROA alone and not actually announcing the 
> covering prefix as well isn't a good thing to do. 
> 
> On Sun, Oct 22, 2023 at 1:39 PM Owen DeLong  <mailto:o...@delong.com>> wrote:
>> Look again, Tom. This is an attack vector using a LESS specific route. The 
>> /22 gets discarded, but a covering /0-/21 would not. 
>> 
>> Owen
>> 
>>> On Oct 22, 2023, at 10:06, Tom Beecher >> <mailto:beec...@beecher.cc>> wrote:
>>> 
>>> 
>>>> And is it your belief that this addresses the described attack vector?
>>>> AFAICT, it does not.
>>> 
>>> Quoting myself : 
>>> 
>>>> WITH the assertion that all routers in the routing domain are RPKI 
>>>> enabled, and discarding RPKI INVALIDs.
>>> 
>>>  In the mixed RPKI / non-RPKI environment of today's internet, no it 
>>> doesn't. This does not mean that RPKI is deficient, or the AS 0 ROA doesn't 
>>> work as intended, as was stated.
>>> 
>>>  
>>> 
>>> On Sun, Oct 22, 2023 at 12:57 PM William Herrin >> <mailto:b...@herrin.us>> wrote:
>>>> On Sun, Oct 22, 2023 at 9:38 AM Tom Beecher >>> <mailto:beec...@beecher.cc>> wrote:
>>>> >> He's saying that someone could come along and advertise 0.0.0.0/1 
>>>> >> <http://0.0.0.0/1> and
>>>> >> 128.0.0.0/1 <http://128.0.0.0/1> and by doing so they'd hijack every 
>>>> >> unrouted address block
>>>> >> regardless of the block's ROA.
>>>> >>
>>>> >> RPKI is unable to address this attack vector.
>>>> >
>>>> >
>>>> > https://www.rfc-editor.org/rfc/rfc6483
>>>> >
>>>> > Section 4
>>>> >>
>>>> >>
>>>> >> A ROA with a subject of AS 0 (AS 0 ROA) is an attestation by the
>>>> >> holder of a prefix that the prefix described in the ROA, and any more
>>>> >> specific prefix, should not be used in a routing context.
>>>> 
>>>> And is it your belief that this addresses the described attack vector?
>>>> AFAICT, it does not.
>>>> 
>>>> Regards,
>>>> Bill Herrin
>>>> 
>>>> 
>>>> -- 
>>>> William Herrin
>>>> b...@herrin.us <mailto:b...@herrin.us>
>>>> https://bill.herrin.us/

Re: RPKI unknown for superprefixes of existing ROA ?

[Owen DeLong via NANOG]

Look again, Tom. This is an attack vector using a LESS specific route. The /22 gets discarded, but a covering /0-/21 would not. OwenOn Oct 22, 2023, at 10:06, Tom Beecher  wrote:And is it your belief that this addresses the described attack vector?AFAICT, it does not.Quoting myself : WITH the assertion that all routers in the routing domain are RPKI enabled, and discarding RPKI INVALIDs. In the mixed RPKI / non-RPKI environment of today's internet, no it doesn't. This does not mean that RPKI is deficient, or the AS 0 ROA doesn't work as intended, as was stated. On Sun, Oct 22, 2023 at 12:57 PM William Herrin  wrote:On Sun, Oct 22, 2023 at 9:38 AM Tom Beecher  wrote:
>> He's saying that someone could come along and advertise 0.0.0.0/1 and
>> 128.0.0.0/1 and by doing so they'd hijack every unrouted address block
>> regardless of the block's ROA.
>>
>> RPKI is unable to address this attack vector.
>
>
> https://www.rfc-editor.org/rfc/rfc6483
>
> Section 4
>>
>>
>> A ROA with a subject of AS 0 (AS 0 ROA) is an attestation by the
>> holder of a prefix that the prefix described in the ROA, and any more
>> specific prefix, should not be used in a routing context.

And is it your belief that this addresses the described attack vector?
AFAICT, it does not.

Regards,
Bill Herrin


-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: RPKI unknown for superprefixes of existing ROA ?

[Owen DeLong via NANOG]

Actually, Job, the 1.2.0/20 would be the longest prefix announced for 1.2.4/24 and 1.2.7/24 in this case. It’s a rather clever end-run. The /20 won’t match the more specific as0 ROAs, so it gets accepted. The /24s either aren’t advertised or they get discarded as invalid. Of course, if RIRs were issuing AS0 ROAs for unissued space, this wouldn’t be a problem, but sadly, several communities have opposed that process. OwenOn Oct 22, 2023, at 08:47, Job Snijders via NANOG  wrote:On Sun, 22 Oct 2023 at 17:42, Amir Herzberg  wrote:Bill, thanks! You explained the issue much better than me. Yes, the problem is that, in my example, the operator was allocated 

1.2.4/22 but the attacker is announcing 

1.2.0/20, which is larger than the allocation, so the operator cannot issue ROA for it (or covering it). Of course, the RIR _could_ do it (but I don't think they do, right?). So this `superprefix hijack' may succeed in spite of all the ROAs that the operator could publish. I'm not saying this is much of a concern, as I never heard of such attacks in the wild, but I guess it _could_ happen in the future.How is “success” measured here?The attacker won’t be drawing traffic towards itself destined for addresses in the /22, because of LPM https://en.wikipedia.org/wiki/Longest_prefix_matchAttackers don’t hijack IP traffic by announcing less-specifics. It don’t work that way.Kind regards,Job

Re: Acceptance of RPKI unknown in ROV

[Owen DeLong via NANOG]

I ask because there was discussion at the ARIN meeting and Kevin Blumburg made 
the suggestion that “in 2024, routes will not be accepted without ROAs”.

I didn’t think this was likely, but as someone with resources for which I 
cannot create ROAs, it is a concern. So far, I haven’t really seen a 
significant benefit to going to the trouble of creating ROAs, but I also don’t 
want to suddenly find myself offline because I didn’t, so I figured it was a 
good idea to get a sense of the community on this.

Thanks to those that replied.

Owen


> On Oct 19, 2023, at 12:17, Job Snijders  wrote:
> 
> On Thu, 19 Oct 2023 at 12:12, Aftab Siddiqui  > wrote:
>> A quick check to my routing table suggests that I have 206700 preferred 
>> routes (v4/v6) to notfound (unknown) destinations. So yeah I don't think 
>> anyone can afford to do this right now.
> 
> 
> I don’t think anyone can afford to ever do this, regardless of the number of 
> unknown destinations!
> 
> Imagine not being able to reach North American destinations for 23 hours 
> because of a cryptographic signing issue at the RIR [0] causing all ROAs to 
> blip out of existence.
> 
> Kind regards,
> 
> Job
> 
> [0] 
> https://www.arin.net/announcements/20200826/

Acceptance of RPKI unknown in ROV

[Owen DeLong via NANOG]

A question for network operators out there that implement ROV…

Is anyone rejecting RPKI unknown routes at this time?

I know that it’s popular to reject RPKI invalid (a ROA exists, but doesn’t 
match the route), but I’m wondering if anyone  is currently or has any plans to 
start rejecting routes which don’t have a matching ROA at all?

Thanks,

Owen

Re: Amir Golestan sentenced to 5 years in prison for IP theft scheme

[Owen DeLong via NANOG]

Sadly, probably not. AUP violations are a civil, not a criminal matter. Fraud,  
OTOH, is a criminal matter.

In the US, at least, people don’t go to prison for civil matters.

Owen


> On Oct 17, 2023, at 19:19, Mel Beckman  wrote:
> 
> So ARIN DB spammers should get at least a year or two, right? :-)
> 
> -mel via cell
> 
>> On Oct 17, 2023, at 7:05 PM, goemon--- via NANOG  wrote:
>> 
>> https://krebsonsecurity.com/2023/10/tech-ceo-sentenced-to-5-years-in-ip-address-scheme/
>> 
>> And a statement from ARIN: 
>> https://www.arin.net/blog/2023/10/16/micfo-golestan-sentencing/

Re: Add communities on direct routes in Juniper

[Owen DeLong via NANOG]

I believe you need to add the communities either on the import policy which 
pulls in the direct route or the export policy to the neighbor(s) you want to 
feed the communities to. 

Owen


> On Oct 15, 2023, at 05:51, Jason R. Rokeach via NANOG  wrote:
> 
> Hi Stanislav,
> I believe this is what you are looking for:
> 
> [edit]
> jcluser@Lothlorien-MX1# show | compare 
> [edit interfaces lo0 unit 0 family inet]
>address 10.0.0.0/32 { ... }
> +   address 5.5.5.5/32;
> [edit protocols bgp]
> -   export IPV4-STATIC;
> +   export [ IPV4-STATIC TAG-DIRECT ];
> [edit policy-options]
> +   policy-statement TAG-DIRECT {
> +   from {
> +   protocol direct;
> +   route-filter 5.5.5.5/32 exact;
> +   }
> +   then {
> +   community set MYCOMMUNITY;
> +   accept;
> +   }
> +   }
> [edit policy-options]
> +   community MYCOMMUNITY members 5:5;
> 
> [edit]
> jcluser@Lothlorien-MX1# commit 
> commit complete
> 
> [edit]
> jcluser@Lothlorien-MX1# run show route advertising-protocol bgp 172.19.0.2 
> detail | find 5.5.5.5 
> * 5.5.5.5/32 (1 entry, 1 announced)
> BGP group RR-LOADBALANCER type External
> Nexthop: Self
> AS path: [65000] I 
> Communities: 5:5
> 
> Regards,
> Jason R. Rokeach
> 
> 
> --- Original Message ---
>> On Sunday, October 15th, 2023 at 8:29 AM, Saku Ytti - saku at ytti.fi 
>>  wrote:
>> 
>> 
>> Unfortunately not yet, as far as I know. Long time ago I gave this to
>> my account team
>> 
>> Title: Direct routes must support tag and or community
>> Platform: Trio, priority MX80, MPC2
>> JunOS: 12.4Rx
>> Command: 'set interfaxe ge-4/2.0 family inet address 10.42.42.1/24
>> tag|community X'
>> JTAC: n/a
>> ER:
>> - Router must be able to add tags communities to direct routes directly, like
>> it does for static routes
>> 
>> Usage Case:
>> Trivial way to signal route information to BGP. Often tag/community is used
>> by service providers to singal 'this is PI/PA prefix, leak it to internet' or
>> 'this is backup route, reduce its MED'. However for some reason it is only
>> supported for static routes, while usage scenario and benefits are exactly 
>> the
>> same for direct routes.
>> 
>> On Sun, 15 Oct 2023 at 15:27, Stanislav Datskevych via NANOG
>> nanog@nanog.org wrote:
>> 
>>> Dear all,
>>> 
>>> Is there a way to add BGP communities on direct (interface) routes in 
>>> Junipers? The task looks to be simple but the solution eludes me.
>>> In Cisco/Arista, for example, I could use "network 192.0.2.0/24 route-map 
>>> ".
>>> 
>>> In Juniper it seems to be impossible. I even tried putting interface-routes 
>>> into rib-group with an import policy.
>>> But it seems the import policy only works on importing routes into 
>>> Secondary routing tables (e.g. inet.50), and not into the Primary one 
>>> (inet.0).
>>> 
>>> I know it's possible to add communities on later stage while announcing 
>>> networks to peers, in [protocols bgp group  export]. But I'd better 
>>> slap the community on the routes right when they're imported into RIB, not 
>>> when they announced to peers.
>>> 
>>> Thanks in advance.
>> 
>> 
>> 
>> --
>> ++ytti
> 

Re: ARIN whois contact abuse from ipv4depot aka Silicon Desert International Inc

[Owen DeLong via NANOG]

> On Oct 12, 2023, at 10:59, Niels Bakker  wrote:
> 
> * Laura Smith [Thu 12 Oct 2023, 19:01 CEST]:
>> I mean, most (all ?) of the registries still can't be bothered to validate 
>> the information the resource holders post to the database.  Last time I 
>> asked, e.g. RIPE about it, they basically said "not my problem guv" , 
>> pointed me to some policy document that said members should provide correct 
>> details and well, that was about it.
>> 
>> So if they don't do that, then what hope is there for them doing something 
>> about the harvesters ?
> 
> RIPE have a policy that states members should submit correct contact details. 
> Having spammers harvest the database discourages people from submitting 
> correct contact details. Ergo, RIPE have a vested interest in ensuring the 
> database doesn't get abused by spammers.

And yet, at least so far, RIPE refuses to take action on such reports, ergo, 
apparently they don’t really care as much as you say they should.


Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Oct 12, 2023, at 01:42, Willy Manga  wrote:
> 
> .
> 
>> On 12/10/2023 10:00, Owen DeLong wrote:
>> [...]
>>>> However, IF YY is paying attention, and YY wants to advertise 
>>>> 2001:db8::/32 as well as allow 2001:db8:8000::/36 and 2001:db8:f000::/36, 
>>>> I would expect AS YY would generate ROAs for
>>>>2001:db8::/32 with ORIGIN-AS=YY MAXPREFIXLEN=36
>>>>2001:db8:0::/33 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>>>2001:db8:8000::/36 with ORIGIN-AS=YY MAXPREFIXLEN=36
>>>>2001:db8:9000::/35 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>>>2001:db8:a000::/34 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>>>2001:db8:c000::/34 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>>>2001:db8:e000::/36 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>>>2001:db8:f000::/36 with ORIGIN-AS=YY MAXPREFIXLEN=36
>>> 
>>> As Dale suggested in another email[1], it's better to just cover ROAs for 
>>> what you are advertising. Why?
>> If that works, perhaps… OTOH, I’m not sure it does. I’m not sure the /32 
>> MAXLEN 32 wouldn’t prevent effectiveness of the /36 ROAs.
>>> 
>>> 1. I can't confirm at this stage that all the implementation allows you to 
>>> leave the maxLength field empty.
>> I can… It’s an Optional Field in the specification.
> 
> For the _specification_ yes. But by "Implementation" I'm referring to 
> whatever either the RIR (those using hosted mode) or your own RPKI 
> Certificate Authority (those using the delegated mode) will allow.

I don’t consider non-compliant implementations as something that needs to or 
even should be accommodated. 

Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Oct 11, 2023, at 19:18, Willy Manga  wrote:
> 
> .
> 
> On 11/10/2023 03:52, Delong.com wrote:
>> [...]
>>> RPKI only asserts that a specific ASN must originate a prefix.  It does 
>>> nothing to validate the authenticity of the origination.
>> Nope… It ALSO asserts (or can assert) an attribute of “Maximum allowed 
>> prefix length”.
>> E.g. if I have a ROA for AS65500 to originate 2001:db8::/32 with a “Maximum 
>> Length” attribute of /36, then any advertisement (even originated by 65500) 
>> that is longer than /36 should be considered invalid.
>>> If I am AS XX, and want to hijack a prefix from AS YY that has RPKI ROAs 
>>> protecting it, and AS YY has allowed more specifics to be announced within 
>>> the prefix range covered by the ROA, I'm in like flynn, because I just need 
>>> to configure my router with AS YY as the origin AS, then insert the 
>>> expected ASN for the neighbor adjacency with my upstreams, and bob's your 
>>> uncle, the more specific prefix passes RPKI validation, and traffic comes 
>>> flying my way.
>> Yes, IF YY has allowed longer prefixes. If YY doesn’t allow longer prefixes 
>> and/or doesn’t supply AS0 ROAs for more specifics that should not be 
>> announced, then YY has indeed aimed a firearm squarely at their lower distal 
>> appendage and fired.
>> However, IF YY is paying attention, and YY wants to advertise 2001:db8::/32 
>> as well as allow 2001:db8:8000::/36 and 2001:db8:f000::/36, I would expect 
>> AS YY would generate ROAs for
>>  2001:db8::/32 with ORIGIN-AS=YY MAXPREFIXLEN=36
>>  2001:db8:0::/33 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>  2001:db8:8000::/36 with ORIGIN-AS=YY MAXPREFIXLEN=36
>>  2001:db8:9000::/35 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>  2001:db8:a000::/34 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>  2001:db8:c000::/34 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>  2001:db8:e000::/36 with ORIGIN-AS=0 (no MAXPREFIXLEN needed)
>>  2001:db8:f000::/36 with ORIGIN-AS=YY MAXPREFIXLEN=36
> 
> As Dale suggested in another email[1], it's better to just cover ROAs for 
> what you are advertising. Why?

If that works, perhaps… OTOH, I’m not sure it does. I’m not sure the /32 MAXLEN 
32 wouldn’t prevent effectiveness of the /36 ROAs.
> 
> 1. I can't confirm at this stage that all the implementation allows you to 
> leave the maxLength field empty.

I can… It’s an Optional Field in the specification.

> 2. If you want to follow that logic, what you are trying to accomplish with 
> AS0 is basically the *complement* of what you are not advertising. I believe 
> that will be much more work and you might miss a lot of specifics. e.g : 
> under your 2001:db8::/32 , do not forget you have 16x/36, 2x/33,4x/34,... You 
> will have to insert statement for every single of them.

Yes, but if I issue a /34 AS0 with no MAXLEN, that _SHOULD_ mark ALL more 
specifics as invalid.

If that doesn’t work, then you’re right, the AS0 ROAs are essentially useless, 
but then one has to wonder what value any RIR issued AS0 ROAs would have as 
well, since they would obviously be less specific.

Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

Ratio of FIB to RIB is only part of the equation.

IPv6 is NOT under the disaggregation pressure that IPv4 is under because there 
is no pressure (other than perhaps scarcity mentality from those that don’t 
properly understand IPv6) to dense-pack IPv6 assignments or undersize IPv6 
allocations.

Look at the difference in prefixes per ASN across the two tables and that tells 
a much grimmer story for IPv4 in terms of RIB growth vs. IPv6.

Owen


> On Oct 4, 2023, at 23:30, Mark Tinka  wrote:
> 
> 
> 
> On 10/5/23 08:24, Geoff Huston wrote:
> 
> 
>> The IPv6 FIB is under the same pressure from more specifics. Its taken 20 
>> years to get there, but the IPv6 FIB is now looking stable at 60% opf the 
>> total FIB size [2]. For me, thats a very surprising outcome in an 
>> essentially unmanaged system. 
> 
> Were you expecting it to be lower than IPv4?
> 
> Mark.

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

I think it needs to be slightly more nuanced than that…

Because IPv4 is driven to dense-packing and tight allocations, I think 
disaggregation of IPv4 will only increase over time.

The hope is that by issuing larger than needed blocks of IPv6, less 
disaggregation becomes necessary over time.

So far, that seems to be largely the case, with more than 50% of ASNs 
represented in the DFZ in IPv6, we see
roughly 191884 unique destinations in IPv6 and 942750 unique destinations in 
IPv4 (admittedly an instantaneous
snapshot a few moments ago from a single DFZ router, YMMV).

Even if we double the IPv6 prefix count or even quadruple it, we’re still 
looking at a much smaller level of disaggregation
in IPv6 than IPv4 in the current state.

Owen


> On Oct 4, 2023, at 22:49, Crist Clark  wrote:
> 
> Been resisting adding to this thread...
> 
> But if the assumption is that networks will always eventually totally 
> deaggregate to the maximum, we're screwed. Routing IPv4 /32s would be 
> nothing. The current practice of accepting /48s could swell to about 2^(48 - 
> 3) = 2^45 = 35184372088832.
> 
> What will prevent unrestricted growth of the IPv6 table if operators push 
> everything out to /48 "to counter hijacks" or other misguided reasons?
> 
> On Wed, Oct 4, 2023 at 8:14 AM Owen DeLong via NANOG  <mailto:nanog@nanog.org>> wrote:
>> If you maximally disaggregate to /24, you end up with about 12M fib entries. 
>> At /25 this doubles and you double it again for every bit you move right. 
>> 
>> At /24, we are on borrowed time without walking right. Also, the CPU in most 
>> routers won’t handle the churn of a 10M prefix RIB. 
>> 
>> Owen
>> 
>> 
>> > On Oct 4, 2023, at 03:15, Mark Tinka  wrote:
>> > 
>> > 
>> > 
>> >> On 10/4/23 12:11, Musa Stephen Honlue wrote:
>> >> 
>> >> Which one is easier,
>> >> 
>> >> 1. Convincing the tens of thousands of network operators and equipment 
>> >> vendors to modify configs and code to accept more specifics than /24, or
>> > 
>> > Equipment vendors can already support 10 million entries in FIB. They just 
>> > ask for a little bit of cash for it.
>> > 
>> > Mark.
>> 
>> 

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Oct 4, 2023, at 03:18, Mark Tinka  wrote:
> 
>  
> 
>> On 10/4/23 09:27, Elmar K. Bins wrote:
>> 
>> 
>>  Justin,
>> 
>> I'm not sure you're not confusing scope here.
>> 
>> Everybody and their sister accept smaller blocks from their customers; we're
>> all talking about the DFZ here, not customer routes that you aggregate.
> 
> Actually, we don't.
> 
> From our customers, the most we are accepting today is a /24 and a /48. This 
> is for transit customers with their own AS and address space.
> 
> Of course, if it's a DIA customer, we can assign longer prefixes, but that is 
> internal address space that belongs to us.
> 
> Mark.

Which if you read his message carefully is pretty much exactly what he said. 

Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

If you maximally disaggregate to /24, you end up with about 12M fib entries. At 
/25 this doubles and you double it again for every bit you move right. 

At /24, we are on borrowed time without walking right. Also, the CPU in most 
routers won’t handle the churn of a 10M prefix RIB. 

Owen


> On Oct 4, 2023, at 03:15, Mark Tinka  wrote:
> 
> 
> 
>> On 10/4/23 12:11, Musa Stephen Honlue wrote:
>> 
>> Which one is easier,
>> 
>> 1. Convincing the tens of thousands of network operators and equipment 
>> vendors to modify configs and code to accept more specifics than /24, or
> 
> Equipment vendors can already support 10 million entries in FIB. They just 
> ask for a little bit of cash for it.
> 
> Mark.

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Owen DeLong via NANOG]

Problem with that theory is the ratio of collateral damage to pain inflicted. 

Filter or deeper cogent and they don’t feel anything themselves. Their 
customers _might_ miss being able to reach your customers (or you), but then it 
is Cogent’s customers that feel the pain the most and Cogent to a much lesser 
degree and only as a second order effect. 

You will definitely miss connecting to some of Cogent’s customers, so you have 
also directly inflicted pain upon your self (and likely your customers). 

Personally, I would support any of my providers teaching Cogent a lesson this 
way, but most customers aren’t so understanding or even aware of the situation 
and they don’t care even if it is explained to them. They expect their packets 
to get delivered. That’s why they pay you.

It would be nice if there were a way to get Cogent fined or to sue Cogent for 
these acts and raise their costs directly without harming their customers, but 
so far nobody has figured out a way to do it. (Perhaps the layer 9 types in the 
list can put their brains to work on this problem). 

Owen


> On Oct 4, 2023, at 04:30, b...@uu3.net wrote:
> 
> This is not the outcome of internet ecosystem, this is outcome
> of commercialization, where money is what is all cared, not good
> product, ethical behavior, etc.
> 
> This is also because good guys do NOT fight back strong enough.
> Cogent start to give you hard time? Start to filter they whole
> prefixes? Maybe depeer them?
> 
> I know this sound extreme, but.. everything else seems to fail..
> 
> 
> -- Original message --
> 
> From: Christopher Morrow 
> To: nanog@nanog.org
> Subject: Re: ARIN email address (was cogent spamming directly from ARIN
>records?)
> Date: Tue, 3 Oct 2023 15:48:11 -0400
> 
> i agree this is a sad outcome of the internet ecosystem.
> 
>> We keep discussing it because we care about keeping the internet
>> running. It's similar to why we keep looking for new security holes in
>> existing software: we don't stop because inevitably we'll find more so
>> it's a lost cause, we keep looking because inevitably we'll find more
>> so the product becomes more secure.
> 
> those are a bit of a false equivalence... but... ok.
> I think: "Oh look, more spam, delete"
> is basically how this sort of problem (email from randos trying to
> sell me ED pills or 10Gs) should be treated.
> I don't know that it's helpful to keep re-litigating that end state :(
> 
> I'm sure telling dave shaeffer: "Hey, your sales droids are being
> rude" is going to end as well as sending him ED pill emails.

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Owen DeLong via NANOG]

I was one of the main people behind their suspension from ARIN whois for 6 
months.

They have not spammed me since.

They’re probably afraid of another cake.

Owen


> On Oct 3, 2023, at 18:18, Mike Lyon  wrote:
> 
> Give it time :)
> 
> -Mike
> 
>> On Oct 3, 2023, at 18:06, Owen DeLong via NANOG  wrote:
>> 
>> But I seem to have finally gotten Cogent trained not to spam this one, so I 
>> think I’ll leave it as is.
>> 
>> YMMV
>> 
>> Owen
>> 
>> 
>>> On Oct 3, 2023, at 08:52, Bryan Fields  wrote:
>>> 
>>>> On 10/2/23 11:28 AM, Mel Beckman wrote:
>>>> I believe they got the contact information from ARIN
>>> 
>>> I'd suggest everyone use an alias unique to ARIN for your POC and/or public 
>>> email.  Makes it super simple to verify where it was sourced from.
>>> 
>>> (and yes I've got the same spam)
>>> -- 
>>> Bryan Fields
>>> 
>>> 727-409-1194 - Voice
>>> http://bryanfields.net
>> 

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Owen DeLong via NANOG]

But I seem to have finally gotten Cogent trained not to spam this one, so I 
think I’ll leave it as is.

YMMV

Owen


> On Oct 3, 2023, at 08:52, Bryan Fields  wrote:
> 
> On 10/2/23 11:28 AM, Mel Beckman wrote:
>> I believe they got the contact information from ARIN
> 
> I'd suggest everyone use an alias unique to ARIN for your POC and/or public 
> email.  Makes it super simple to verify where it was sourced from.
> 
> (and yes I've got the same spam)
> -- 
> Bryan Fields
> 
> 727-409-1194 - Voice
> http://bryanfields.net

Re: MX204 tunnel services BW

[Owen DeLong via NANOG]

You can configure tunnel bandwidth everywhere, but you can’t configure
a given tunnel everywhere, you have to assign it to a particular FPC/PIC/0.

For example, with:
set chassis fps 2 pic 3 tunnel-services bandwidth 10g

You need to create gr-2/3/0 interfaces for tunnels to use that PFE.

You can create multiple tunnel-services bandwidth entries on multiple
PICs, but you can only put a given tunnel on one gr-x/y/0 interface.

Owen


> On Oct 2, 2023, at 23:21, Saku Ytti  wrote:
> 
> On Mon, 2 Oct 2023 at 20:21, Jeff Behrns via NANOG  wrote:
> 
>> Encountered an issue with an MX204 using all 4x100G ports and a logical
>> tunnel to hairpin a VRF.  The tunnel started dropping packets around 8Gbps.
>> I bumped up tunnel-services BW from 10G to 100G which made the problem
>> worse; the tunnel was now limited to around 1.3Gbps.  To my knowledge with
>> Trio PFE you shouldn't have to disable a physical port to allocate bandwidth
>> for tunnel-services.  Any helpful info is appreciated.
> 
> You might have more luck in j-nsp.
> 
> But yes you don't need any physical interface in trio to do tunneling.
> I can't explain your problem, and you probably need JTAC help. I would
> appreciate it if you'd circle back and tell what the problem was.
> 
> How it works is that when PPE decides it needs to tunnel the packet,
> you're going to send the packet back to MQ via SERDES (which will then
> send it again to some PPE, not the same). I think what that bandwidth
> command does is change the stream allocation, you should see it in
> 'show  <#> stream'.
> 
> In theory, because PPE can process packet forever (well, until
> watchdog kills the PPE for thinking it is stuck) you could very
> cheaply do outer+inner at the local PPE, but I think that would mean
> that certain features like QoS would not work on the inner interface,
> so I think all this expensive recirculation and SERDES consumption is
> to satisfy quite limited need, and it should be possible to implement
> some 'performance mode' for tunneling, where these MQ/XM provided
> features are not available, but performance cost in most cases is
> negligible.
> 
> In parallel to opening the JTAC case, you might want to try to
> experiment in which FPC/PIC you set the tunneling bandwidth to. I
> don't understand how the tunneling would work if the MQ/XM is remote,
> like would you then also steal fabric capacity every time you tunnel,
> not just  MQ>LU>MQ>LU SERDES, but MQ>LU>MQ>FAB>MQ>LU. So intuitively I
> would recommend ensuring you have the bandwidth configured at the
> local PFE, if you don't know what the local PFE is, just configure it
> everywhere?
> Also you could consult several counters to see if some stream or
> fabric is congested, and these tunneled packets are being sent over
> congested fabric every time with lower fabric qos.
> 
> I don't understand why the bandwidth command is a thing, and why you
> can configure where it is. To me it seems obvious they should always
> handle tunneling strictly locally, never over fabric, because you
> always end up stealing more capacity if you send it to remote MQ. That
> is, implicitly it should be on for every MQ, and every PPE tunnel via
> local MQ.
> 
> -- 
>  ++ytti

Re: MX204 tunnel services BW

[Owen DeLong via NANOG]

> On Oct 2, 2023, at 20:18, behrnsj...@yahoo.com wrote:
> 
> 
> 
> -Original Message-
> From: Delong.com  
> Sent: Monday, October 2, 2023 5:47 PM
> To: behrnsj...@yahoo.com
> Cc: nanog@nanog.org
> Subject: Re: MX204 tunnel services BW
> 
>> “Tunnel gets whatever bandwidth is left after physical port packets are 
>> processed” and likely some additional overhead for managing the sharing.
> 
>> Could that be what’s happening to you?
> 
> Aggregate throughput for the box was less than 100Gbps while the tunnel was 
> being starved.
> 

Yeah, doesn’t quite work that way…

The tunnel is assigned to one particular PFE.

What was the aggregate throughput on that PFE (which spending on the card may 
well top out at 40Gbps or even 10Gbps, though not likely
on most Trio-based cards, that’s more of the DPC era cards, which did require 
you to sacrifice a port for tunnel bandwidth).

Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

Isn’t that pretty much what Geoff Huston has done with the weekly reports William quoted earlier in this thread?Sure, that’s from a limited set of perspectives, but it probably represents the minimum achievable compression in most circumstances. OwenOn Oct 2, 2023, at 06:41, Joshua Miller  wrote:Seems like we've reached the limits of apriori speculation. At this point I'd like to see data demonstrating that it's at least viable from a statistical perspective. If someone is motivated to demonstrate this, a "backtest" against historical data would be the next step. Later, one could design the study to reveal "transient degradation" (loops, drops, etc.) and their probability, though the duration would be more a function of the implementation. It would be best to "backtest" the status quo as a control because it too has transient degradation, for a more apples-to-apples comparison.I'm not sufficiently motivated (nor knowledgeable in statistics) to take this on. I see this more in the domain of vendors to determine the best approach for their implementation.On Mon, Oct 2, 2023 at 9:21 AM t...@pelican.org  wrote:On Monday, 2 October, 2023 09:39, "William Herrin"  said:

> That depends. When the FIB gets too big, routers don't immediately
> die. Instead, their performance degrades. Just like what happens with
> oversubscription elsewhere in the system.
> 
> With a TCAM-based router, the least specific routes get pushed off the
> TCAM (out of the fast path) up to the main CPU. As a result, the PPS
> (packets per second) degrades really fast.
> 
> With a DRAM+SRAM cache system, the least used routes fall out of the
> cache. They haven't actually been pushed out of the fast path, but the
> fast path gets a little bit slower. The PPS degrades, but not as
> sharply as with a TCAM-based router.

Spit-balling here, is there a possible design for not-Tier-1 providers where routing optimality (which is probably not a word) degrades rather than packet-shifting performance?

If the FIB is full, can we start making controlled and/or smart decisions about what to install, rather than either of the simple overflow conditions?

For starters, as long as you have *somewhere* you can point a default at in the worst case, even if it's far from the *best* route, you make damn sure you always install a default.

Then you could have knobs for what other routes you discard when you run out of space.  Receiving a covering /16?  Maybe you can drop the /24s, even if they have a different next hop - routing will be sub-optimal, but it will work.   (I know, previous discussions around traffic engineering and whether the originating network must / does do that in practice...)

Understand which routes your customers care about / where most of your traffic goes?  Set the "FIB-preference" on those routes as you receive them, to give them the greatest chance of getting installed.

Not a hardware designer, I have little idea as to how feasible this is - I suspect it depends on the rate of churn, complexity of FIB updates, etc.  But it feels like there could be a way to build something other than "shortest -> punt to CPU" or "LRU -> punt to CPU".

Or is everyone who could make use of this already doing the same filtering at the RIB level, and not trying to fit a quart RIB into a pint FIB in the first place?

Thanks,
Tim.

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Oct 2, 2023, at 01:18, Nick Hilliard  wrote:
> 
> William Herrin wrote on 02/10/2023 08:56:
>> All it means is that you have to keep an eye on your FIB
>> size as well, since it's no longer the same as your RIB size.
> 
> the point Jacob is making is is that when using FIB compression, the FIB size 
> depends on both RIB size and RIB complexity.  I.e. what was previously a 
> deterministic 1:1 ratio between RIB and FIB - which is straightforward to 
> handle from an operational point of view - becomes non-deterministic. The 
> difficulty with this is that if you end up with a FIB overflow, your router 
> will no longer route.

It was never 1:1 if you have more than one path for any route. The FIB only 
contains the chosen path(s) to any destination even without fib compression. 

> 
> That said, there are cases where FIB compression makes a lot of sense, e.g. 
> leaf sites, etc. Conversely, it's not a generally appropriate technology for 
> a dense dfz core device.  It's a tool in the toolbox, one of many.

Even at a dense DFZ core device, there are a large number of single-origin 
consecutive /24s in the table which can be fib compressed with no loss. For a 
long time, someone was teaching up and coming operators in Asia that they 
should always announce everything as disaggregated /24s to guard against route 
hijacking. This unfortunate practice persists still, making FIB compression 
quite practical even at core nodes. 

Owen

> 
> Nick

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

First, no, a transient where all route disaggregating disappears from the 
global table is extraordinarily unlikely. 

Second, as I understand it, each update cycle results in rebuilding the fib 
from scratch rather than figuring out how to splice and dice it, so the 
computation required to cope with that single /24 change isn’t exactly as 
described. 

Finally, unless that /24 change ends up changing the exit interface or next-hop 
on the exit interface (which decreases in probability with topological 
distance), it wouldn’t actually change the fib content. 

The process of downloading a new fib to line cards from the RE is very fast. 
Break before make may be tolerable risk. It won’t lose a statistically 
significant number of packets in a single event. If events are occurring 
frequently enough to be an issue, then I think the route flapping is a bigger 
problem than the lost packets. 

YMMV

Owen


> On Oct 1, 2023, at 21:55, Jakob Heitz (jheitz) via NANOG  
> wrote:
> 
> 
> While I did allude to some of the complexity, my main point
> is that FIB compression does not allow you to install a FIB with less memory.
> Because you must be prepared for transients during which the FIB needs to 
> store
> mostly uncompressed anyway.
> All it does is to increase convergence time.
>  
> Kind Regards,
> Jakob
>  
>  
> From: William Herrin 
> Date: Sunday, October 1, 2023 at 6:32 PM
> To: Jakob Heitz (jheitz) 
> Cc: nanog@nanog.org 
> Subject: Re: maximum ipv4 bgp prefix length of /24 ?
> 
> On Sun, Oct 1, 2023 at 5:40 PM Jakob Heitz (jheitz) via NANOG
>  wrote:
> > Among the issues:
> > Suppose the FIB has all the /24 components to make a /20, so it programs a 
> > /20.
> > Then one of the /24's changes nexthop. It now has to undo all that 
> > compression
> 
> Yeah... all this stuff is on the same level of complexity as
> implementing a B-Tree. Standard task on the road to an undergraduate
> computer science degree. Compared to decoding a BGP update message,
> where nearly everything is variable length and you have to nibble away
> at the current field to find the start of the next field, this is a
> cakewalk.
> 
> It doesn't actually get complicated until you want to do more than
> just joining adjacent address blocks.
> 
> Regards,
> Bill Herrin
> 
> 
> 
> 
> -- 
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

Not sure why you think FIB compression is a risk or will be a mess. It’s a 
pretty straightforward task. 

Owen


> On Sep 30, 2023, at 00:03, Mark Tinka  wrote:
> 
>  
> 
>> On 9/30/23 01:36, William Herrin wrote:
>> 
>> 
>>  If I were designing the product, I'd size the SRAM with that in mind.
>> I'd also keep two full copies of the FIB in the outer DRAM so that the
>> PPEs could locklessly access the active one while the standby one gets
>> updated with changes from the RIB. But I'd design the router to
>> gracefully fail if the FIB exceeded what the SRAM could hold.
>> 
>> When a TCAM fills, the shortest prefixes are ejected to the router's
>> main CPU. That fails pretty hard since the shortest prefixes tend to
>> be among the most commonly used. By comparison, an SRAM cache tends to
>> retain the most commonly used prefixes as an inherent part of how
>> caches work, regardless of prefix length. It can operate close to full
>> speed until the actively used routes no longer fit in the cache.
> 
> Well, not sure if you're aware, but starting Junos 21.2, Juniper are 
> implementing FIB Compression on the PTX routers running Express 4 and Junos 
> EVO.
> 
> We have some of these boxes in our network (PTX10001), and I have asked 
> Juniper to provide a knob to allow us to turn it off, as it is currently 
> going to ship as a default-on feature. I'd rather not be part of the 
> potential mess that is going to come with the experimentation of that over 
> the next decade.
> 
> Mark.

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Sep 29, 2023, at 15:14, William Herrin  wrote:
> 
> On Fri, Sep 29, 2023 at 3:11 PM Owen DeLong  wrote:
>> You continue to assume that there is a fast SRAM cache. I’m not sure
>> that is true. I think that all of the FIB RAM on the line cards is fast SRAM
>> and no cache.
> 
> Hi Owen,
> 
> I'm less assuming it and more reading it from this SIGCOMM paper:
> https://people.csail.mit.edu/ghobadi/papers/trio_sigcomm_2022.pdf

Fair enough, but interestingly, I think that the compiled line-card forwarding
table probably always fits in the cache.

Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Sep 29, 2023, at 14:48, William Herrin  wrote:
> 
> On Fri, Sep 29, 2023 at 2:13 PM Tom Beecher  wrote:
>>> My understanding of Juniper's approach to the problem is that instead
>>> of employing TCAMs for next-hop lookup, they use general purpose CPUs
>>> operating on a radix tree, exactly as you would for an all-software
>>> router.
>> 
>> Absolutely are not doing that with "general purpose CPUs".
>> 
>> The LU block on early gen Trios was a dedicated ASIC (LU by itself, then 
>> consolidated slightly) , then later gen Trio put everything on a single 
>> chip, but again dedicated ASIC.
> 
> Hi Tom,
> 
> For clarity, general purpose CPU refers to an architecture not an
> implementation. It's capable of running arbitrary computer software
> and has the typical functions like loading and saving registers, an
> adder, bit shifts, etc. The CPU in a Raspberry Pi is also part of a
> dedicated ASIC that does wifi, packet switching and a bunch of other
> stuff. It's still a CPU. Compare to a TCAM which is purpose built to
> match input bit patterns and is capable of nothing else.
> 
> I suppose the PPEs in the Trio are more like GPUs than CPUs - more
> limited instruction sets but higher parallelism. However, they still
> follow the cache pattern where the frequently used parts of the FIB
> tree are in a fast SRAM cache and the remainder is in slower DRAM
> where it can be loaded into SRAM at the occasional need. The FIB size
> limit before cache thrashing sets in and cuts the PPS is softer than
> the limit with a TCAM but it's still there.

You continue to assume that there is a fast SRAM cache. I’m not sure
that is true. I think that all of the FIB RAM on the line cards is fast SRAM
and no cache.

Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

Several people ate the cake. I received numerous positive comments on it and 
some
of them are about the flavor of the cake.

Owen


> On Sep 29, 2023, at 14:11, Collider  wrote:
> 
> Peering cake... :-)
> 
> i think i was a puppy when that happened and only heard about it way after 
> the fact
> 
> did anyone eat the cake? was it tasty?
> 
> 
> Le 29 septembre 2023 20:55:00 UTC, Owen DeLong via NANOG  a 
> écrit :
>> I have known Mike for many years. I have my disagreements with him and my 
>> criticisms of him.
>> 
>> However, HE decided to stop their free bop tunnel services due to problems 
>> with abuse. A free service
>> which becomes a magnet for problems isn’t long for this world. It’s 
>> unfortunate, but boils down to the
>> usual fact that vandals are the reason the rest of us can’t have nice 
>> things. I have trouble seeing how
>> one can blame Mike for that.
>> 
>> HE has continued to operate their free tunnel service in general and still 
>> provides a very large number
>> of free tunnels. They also provide a number of other services for free and 
>> at very reasonable prices.
>> I don’t see very many major providers giving back to the community to the 
>> extent that HE does.
>> 
>> At this point, if anyone should pay for IPv6 transit between Cogent and HE, 
>> Cogent should be the
>> one paying as they have the (significantly) smaller and less connected IPv6 
>> network. Mike is willing
>> to peer with Cogent for free, just like any other ISP out there. He’s not 
>> asking Cogent for free
>> transit. Cogent is the one with the selective peering policy.
>> 
>> Owen
>> 
>> Full disclosure, yes, I worked for HE for several years and I am a current 
>> HE customer.
>> I am the person behind the (in)famous IPv6 Peering Cake.
>> 
>> 
>> 
>>> On Sep 29, 2023, at 00:44, VOLKAN SALİH  wrote:
>>> 
>>> Many people from big companies/networks are either member of NANOG or 
>>> following/reading NANOG from archives.
>>> 
>>> I was also going to ask if anyone / any company can sponsor (feeless) IPv4 
>>> /24 prefix for my educational research network? (as209395)
>>> 
>>> We do not do or allow SPAM/spoofing and other illegal stuff, we have RPKI 
>>> records and check RPKI of BGP peers.
>>> 
>>> We also consider to have BGP session with HE.net <http://he.net/> and 
>>> CogentCo in the future, so we can re-announce their single-homed prefixes 
>>> to each other, as charity. For the good of everyone on the internet..
>>> 
>>> Mr. M.Leber from He.net <http://he.net/> also stopped feeless BGP tunnel 
>>> service, as he has not seen financial benefit, while still talking about 
>>> community-give-back?! And he still seeks feeless peering from CogentCo, you 
>>> get what you give.whatever goes around comes around
>>> 
>>> Thanks for reading, best regards and wishes
>>> 
>>> 
>>> 
>>> 29.09.2023 09:57 tarihinde Vasilenko Eduard yazdı:
>>>> Well, it depends.
>>>> The question below was evidently related to business.
>>>> IPv6 does not have yet a normal way of multihoming for PA prefixes.
>>>> If IETF (and some OTTs) would win blocking NAT66,
>>>> Then /48 propoisiton is the proposition for PA (to support multihoming).
>>>> Unfortunately, it is at least a 10M global routing table as it has been 
>>>> shown by Brian Carpenter.
>>>> Reminder, The IPv6 scale on all routers is 2x smaller (if people would use 
>>>> DHCP and longer than/64 then the scale would drop 2x additionally).
>>>> Hence, /48 proposition may become 20x worse for scale than proposed 
>>>> initially in this thread.
>>>> Eduard
>>>> From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org] 
>>>> On Behalf Of Owen DeLong via NANOG
>>>> Sent: Friday, September 29, 2023 7:11 AM
>>>> To: VOLKAN SALİH  
>>>> <mailto:volkan.salih...@gmail.com>
>>>> Cc: nanog@nanog.org <mailto:nanog@nanog.org>
>>>> Subject: Re: maximum ipv4 bgp prefix length of /24 ?
>>>>  
>>>> Wouldn’t /48s be a better solution to this need?
>>>>  
>>>> Owen
>>>>  
>>>> 
>>>> 
>>>> On Sep 28, 2023, at 14:25, VOLKAN SALİH >>> <mailto:volkan.salih...@gmail.com>> wrote:
>>>>  
>>>> hello,
>>>> 
>>>> I believe

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

s/DMS/DFZ/ — Not sure how autocrrect did that without me noticing, apologies.


> On Sep 29, 2023, at 14:11, Owen DeLong via NANOG  wrote:
> 
> /32s are perfectly valid in the DMS, but there is currently an IETF limit of 
> ~500M of them (the other 3.5B are not yet released to the RIRs, only 
> 2000::/3).
> 
> Owen
> 
> 
>> On Sep 29, 2023, at 12:54, Collider  wrote:
>> 
>> This thread is utter amateur hour. I too would rather /32s be valid in the 
>> DFZ - but they're not, for good reason (worstcase scenario = circa 4 bln. 
>> routing table entries - no BGP hwaccel can swing that!).
>> 
>> 
>> Le 29 septembre 2023 19:51:29 UTC, Seth Mattinen via NANOG  
>> a écrit :
>>> On 9/29/23 10:24, VOLKAN SALİH wrote:
>>>> 
>>>> you guys become rich this way.. by playing penny pincher.
>>>> 
>>>> I asked global firms like Huawei, not some local company called ADAMS!
>>>> 
>>> 
>>> 
>>> You joined the wrong mailing list then. This is NANOG, which has companies 
>>> of all sizes and private individuals operating networks. This is not a 
>>> "global firms" mailing list.
>>> 
>> 
>> -- 
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> 

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Sep 29, 2023, at 13:43, William Herrin  wrote:
> 
> On Thu, Sep 28, 2023 at 10:29 PM Saku Ytti  wrote:
>> On Fri, 29 Sept 2023 at 08:24, William Herrin  wrote:
>>> Maybe. That's where my comment about CPU cache starvation comes into
>>> play. I haven't delved into the Juniper line cards recently so I could
>>> easily be wrong, but if the number of routes being actively used
>>> pushes past the CPU data cache, the cache miss rate will go way up and
>>> it'll start thrashing main memory. The net result is that the
>>> achievable PPS drops by at least an order of magnitude.
>> 
>> When you say, you've not delved into the Juniper line cards recently,
>> to which specific Juniper linecard your comment applies to?
> 
> Howdy,
> 
> My understanding of Juniper's approach to the problem is that instead
> of employing TCAMs for next-hop lookup, they use general purpose CPUs
> operating on a radix tree, exactly as you would for an all-software
> router. This makes each lookup much slower than a TCAM can achieve.
> However, that doesn't matter much: the lookup delays are much shorter
> than the transmission delays so it's not noticeable to the user. To
> achieve an -aggregate- lookup speed comparable to a TCAM, they
> implement a bunch of these lookup engines as dedicated parallel
> subprocessors rather than using the router's primary compute engine.

In their lower-end hardware, yes. The MX uses ASICs traversing the
tree, if I understood the explanation correctly, but there’s essentially
a copy of the compiled/condensed tree on every line card and many
of the line cards have more than one PFE per line card.

> A TCAM lookup is approximately O(1) while a radix tree lookup is
> approximately O(log n). (Neither description is strictly correct but
> it's close enough to understand the running time.) Log n is pretty
> small so it doesn't take much parallelism for the practical run time
> to catch up to the TCAM.

The only difference, I believe, is that it’s ASICs against RAM rather
than CPUs against CPU Cache, but otherwise, yes, you’ve got the
correct general idea. However, since you brought CPU Cache into
the discussion, that difference seemed worthy of addressing.

Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

/32s are perfectly valid in the DMS, but there is currently an IETF limit of 
~500M of them (the other 3.5B are not yet released to the RIRs, only 2000::/3).

Owen


> On Sep 29, 2023, at 12:54, Collider  wrote:
> 
> This thread is utter amateur hour. I too would rather /32s be valid in the 
> DFZ - but they're not, for good reason (worstcase scenario = circa 4 bln. 
> routing table entries - no BGP hwaccel can swing that!).
> 
> 
> Le 29 septembre 2023 19:51:29 UTC, Seth Mattinen via NANOG  
> a écrit :
>> On 9/29/23 10:24, VOLKAN SALİH wrote:
>>> 
>>> you guys become rich this way.. by playing penny pincher.
>>> 
>>> I asked global firms like Huawei, not some local company called ADAMS!
>>> 
>> 
>> 
>> You joined the wrong mailing list then. This is NANOG, which has companies 
>> of all sizes and private individuals operating networks. This is not a 
>> "global firms" mailing list.
>> 
> 
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

I have known Mike for many years. I have my disagreements with him and my 
criticisms of him.

However, HE decided to stop their free bop tunnel services due to problems with 
abuse. A free service
which becomes a magnet for problems isn’t long for this world. It’s 
unfortunate, but boils down to the
usual fact that vandals are the reason the rest of us can’t have nice things. I 
have trouble seeing how
one can blame Mike for that.

HE has continued to operate their free tunnel service in general and still 
provides a very large number
of free tunnels. They also provide a number of other services for free and at 
very reasonable prices.
I don’t see very many major providers giving back to the community to the 
extent that HE does.

At this point, if anyone should pay for IPv6 transit between Cogent and HE, 
Cogent should be the
one paying as they have the (significantly) smaller and less connected IPv6 
network. Mike is willing
to peer with Cogent for free, just like any other ISP out there. He’s not 
asking Cogent for free
transit. Cogent is the one with the selective peering policy.

Owen

Full disclosure, yes, I worked for HE for several years and I am a current HE 
customer.
I am the person behind the (in)famous IPv6 Peering Cake.



> On Sep 29, 2023, at 00:44, VOLKAN SALİH  wrote:
> 
> Many people from big companies/networks are either member of NANOG or 
> following/reading NANOG from archives.
> 
> I was also going to ask if anyone / any company can sponsor (feeless) IPv4 
> /24 prefix for my educational research network? (as209395)
> 
> We do not do or allow SPAM/spoofing and other illegal stuff, we have RPKI 
> records and check RPKI of BGP peers.
> 
> We also consider to have BGP session with HE.net <http://he.net/> and 
> CogentCo in the future, so we can re-announce their single-homed prefixes to 
> each other, as charity. For the good of everyone on the internet..
> 
> Mr. M.Leber from He.net <http://he.net/> also stopped feeless BGP tunnel 
> service, as he has not seen financial benefit, while still talking about 
> community-give-back?! And he still seeks feeless peering from CogentCo, you 
> get what you give.whatever goes around comes around
> 
> Thanks for reading, best regards and wishes
> 
> 
> 
> 29.09.2023 09:57 tarihinde Vasilenko Eduard yazdı:
>> Well, it depends.
>> The question below was evidently related to business.
>> IPv6 does not have yet a normal way of multihoming for PA prefixes.
>> If IETF (and some OTTs) would win blocking NAT66,
>> Then /48 propoisiton is the proposition for PA (to support multihoming).
>> Unfortunately, it is at least a 10M global routing table as it has been 
>> shown by Brian Carpenter.
>> Reminder, The IPv6 scale on all routers is 2x smaller (if people would use 
>> DHCP and longer than/64 then the scale would drop 2x additionally).
>> Hence, /48 proposition may become 20x worse for scale than proposed 
>> initially in this thread.
>> Eduard
>> From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org] On 
>> Behalf Of Owen DeLong via NANOG
>> Sent: Friday, September 29, 2023 7:11 AM
>> To: VOLKAN SALİH  
>> <mailto:volkan.salih...@gmail.com>
>> Cc: nanog@nanog.org <mailto:nanog@nanog.org>
>> Subject: Re: maximum ipv4 bgp prefix length of /24 ?
>>  
>> Wouldn’t /48s be a better solution to this need?
>>  
>> Owen
>>  
>> 
>> 
>> On Sep 28, 2023, at 14:25, VOLKAN SALİH > <mailto:volkan.salih...@gmail.com>> wrote:
>>  
>> hello,
>> 
>> I believe, ISPs should also allow ipv4 prefixes with length between /25-/27 
>> instead of limiting maximum length to /24..
>> 
>> I also believe that RIRs and LIRs should allocate /27s which has 32 IPv4 
>> address. considering IPv4 world is now mostly NAT'ed, 32 IPv4s are 
>> sufficient for most of the small and medium sized organizations and also 
>> home office workers like youtubers, and professional gamers and webmasters!
>> 
>> It is because BGP research and experiment networks can not get /24 due to 
>> high IPv4 prices, but they have to get an IPv4 prefix to learn BGP in IPv4 
>> world.
>> 
>> What do you think about this?
>> 
>> What could be done here?
>> 
>> Is it unacceptable; considering most big networks that do full-table-routing 
>> also use multi-core routers with lots of RAM? those would probably handle 
>> /27s and while small networks mostly use default routing, it should be 
>> reasonable to allow /25-/27?
>> 
>> Thanks for reading, regards..
>> 

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Sep 28, 2023, at 23:57, Vasilenko Eduard  
> wrote:
> 
> Well, it depends.
> The question below was evidently related to business.
> IPv6 does not have yet a normal way of multihoming for PA prefixes.

The normal way for IETF (which is, IMHO, borked to put it mildly) is to use 
multiple
prefixes and leave source address selection as an exercise for the victim^wend
user.

The obviously better approach is for anyone who cares about such a thing to get
a /48 and an ASN from their friendly neighborhood RIR and move on.

> If IETF (and some OTTs) would win blocking NAT66,

This is an expansion of the problem, not a solution.

> Then /48 propoisiton is the proposition for PA (to support multihoming).

If you’re using a /48, why use PA?

> Unfortunately, it is at least a 10M global routing table as it has been shown 
> by Brian Carpenter.

Not right away and this is an eventuality we will need to face sooner or later 
anyway.

> Reminder, The IPv6 scale on all routers is 2x smaller (if people would use 
> DHCP and longer than/64 then the scale would drop 2x additionally).

Which remains a bad idea for so many other reasons in addition to this one.

> Hence, /48 proposition may become 20x worse for scale than proposed initially 
> in this thread.

I don’t see it that way.

Routing tables are going to continue to grow regardless of what we do in terms 
of end site addressing.
Router vendors will build what is needed. The ability to handle a 10M route 
table is known technology at
this point, and its just a matter of the cost of the line cards.

Owen

> Eduard
> From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org] On 
> Behalf Of Owen DeLong via NANOG
> Sent: Friday, September 29, 2023 7:11 AM
> To: VOLKAN SALİH 
> Cc: nanog@nanog.org
> Subject: Re: maximum ipv4 bgp prefix length of /24 ?
>  
> Wouldn’t /48s be a better solution to this need?
>  
> Owen
>  
> 
> 
> On Sep 28, 2023, at 14:25, VOLKAN SALİH  <mailto:volkan.salih...@gmail.com>> wrote:
>  
> hello,
> 
> I believe, ISPs should also allow ipv4 prefixes with length between /25-/27 
> instead of limiting maximum length to /24..
> 
> I also believe that RIRs and LIRs should allocate /27s which has 32 IPv4 
> address. considering IPv4 world is now mostly NAT'ed, 32 IPv4s are sufficient 
> for most of the small and medium sized organizations and also home office 
> workers like youtubers, and professional gamers and webmasters!
> 
> It is because BGP research and experiment networks can not get /24 due to 
> high IPv4 prices, but they have to get an IPv4 prefix to learn BGP in IPv4 
> world.
> 
> What do you think about this?
> 
> What could be done here?
> 
> Is it unacceptable; considering most big networks that do full-table-routing 
> also use multi-core routers with lots of RAM? those would probably handle 
> /27s and while small networks mostly use default routing, it should be 
> reasonable to allow /25-/27?
> 
> Thanks for reading, regards..
> 

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Sep 28, 2023, at 22:29, Saku Ytti  wrote:
> 
> On Fri, 29 Sept 2023 at 08:24, William Herrin  wrote:
> 
>> Maybe. That's where my comment about CPU cache starvation comes into
>> play. I haven't delved into the Juniper line cards recently so I could
>> easily be wrong, but if the number of routes being actively used
>> pushes past the CPU data cache, the cache miss rate will go way up and
>> it'll start thrashing main memory. The net result is that the
>> achievable PPS drops by at least an order of magnitude.
> 

For that to be an issue, packets would need to be CPU switched which isn’t the 
case on the MX platform. 

Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> 
> In principle, a company could make a business out of announcing a
> large block from a bunch of peering points and then tunneling (vpn)
> parts of it back to customers with sub-/24 assignments. With a broad
> enough selection of peering points, the routing would not be too
> inefficient. And it would divorce the IP addresses from the last-mile
> Internet infrastructure, allowing you to take your addresses with you
> as long as you kept paying the tunnel company.

Actually, such a service would be much easier to stand up as a bunch
of virtual routers running on VPS instances in various cloud providers.
Simple as standing up a VPS running Debian 12 and FRR, then sell
routed blocks to people.

Personally, I think that’s fairly hideous, but someone can probably find a
way to make money doing it.

I know that there are companies charging $ridiculous for “SDN” solutions
that are literally not much more than a tunnel running between two
AWS nodes.

> In practice... there's not enough money in it. If you could ante up
> the cost, you could find a way to qualify for and acquire a full /24.

Given what some of the SDN providers out there are charging, I’m not
so sure that’s true. YMMV.

>> Is it unacceptable; considering most big networks that do full-table-routing 
>> also use multi-core routers with lots of RAM?
> 
> You're thinking of DRAM. But that's not the way it works. Some routers
> use heavily parallel routing engines, each of which need enough dram
> to hold the full forwarding information base and which can suffer from
> CPU cache exhaustion even then. Others use an expensive kind of memory
> called a TCAM that's very fast but both expensive and power hungry, so
> generally not sized for huge numbers of tiny routes.

Trio and Later generations of Juniper MX line cards (which are getting fairly
long in the tooth these days) can handle more than 5M routes in the FIB.
Even the old (now ancient) DPCs can handle ~1.5M routes if you don’t
need a boatload of access lists. (Basically you have to steel FIB memory
from the Access List memory partition, but that’s a simple software
command and a reboot of the line card).

Owen

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

> On Sep 28, 2023, at 21:14, VOLKAN SALİH  wrote:
> 
> IMO, No. ipv4 is not dead yet. we need to raise it, a bit.
> 

Agree to disagree… We need to put the final stake through its heart and move on.

> EINAT solutions are OK
> 

I presume you mean CGNAT? Otherwise, not sure what EINAT is and couldn’t find
a reference with a quick google search.

Again agree to disagree. NAT is bad and more NAT is just worse.
> The future will come very quickly, right now.
> 
One can hope, but it seems to be taking a long time so far.
> We just need to invest in the internet.
> 
Yes, but let’s focus that investment where it makes sense. IPv4 isn’t that.

Owen

> 
> 29.09.2023 07:11 tarihinde Owen DeLong yazdı:
>> Wouldn’t /48s be a better solution to this need?
>> 
>> Owen
>> 
>> 
>>> On Sep 28, 2023, at 14:25, VOLKAN SALİH  
>>> <mailto:volkan.salih...@gmail.com> wrote:
>>> 
>>> hello,
>>> 
>>> I believe, ISPs should also allow ipv4 prefixes with length between /25-/27 
>>> instead of limiting maximum length to /24..
>>> 
>>> I also believe that RIRs and LIRs should allocate /27s which has 32 IPv4 
>>> address. considering IPv4 world is now mostly NAT'ed, 32 IPv4s are 
>>> sufficient for most of the small and medium sized organizations and also 
>>> home office workers like youtubers, and professional gamers and webmasters!
>>> 
>>> It is because BGP research and experiment networks can not get /24 due to 
>>> high IPv4 prices, but they have to get an IPv4 prefix to learn BGP in IPv4 
>>> world.
>>> 
>>> What do you think about this?
>>> 
>>> What could be done here?
>>> 
>>> Is it unacceptable; considering most big networks that do 
>>> full-table-routing also use multi-core routers with lots of RAM? those 
>>> would probably handle /27s and while small networks mostly use default 
>>> routing, it should be reasonable to allow /25-/27?
>>> 
>>> Thanks for reading, regards..
>>> 
>> 

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

CIDR and aggregation in the early 1990s was developed in response to AGS+ 
routers falling over under
the strain of the global size back then. Since then, IPv4 has been a 
progressive loosing proposition and
only gets worse every year. This proposal could certainly accelerate the rate 
at which it continues to get worse.


Owen


> On Sep 28, 2023, at 19:56, Joe Hamelin  wrote:
> 
> Wasn't it about 1997 or so when we ran into deployed Cisco gear (5500s back 
> then) running out of memory for BGP routes?  Been there, done that. -Joe
> 
> On Thu, Sep 28, 2023 at 7:41 PM Jon Lewis  > wrote:
>> On Fri, 29 Sep 2023, VOLKAN SALİH wrote:
>> 
>> > I believe, ISPs should also allow ipv4 prefixes with length between 
>> > /25-/27 instead of limiting maximum length to /24..
>> > 
>> > I also believe that RIRs and LIRs should allocate /27s which has 32 IPv4 
>> > address. considering IPv4 world is now mostly NAT'ed, 32 IPv4s are 
>> > sufficient for most of the
>> > small and medium sized organizations and also home office workers like 
>> > youtubers, and professional gamers and webmasters!
>> > 
>> > It is because BGP research and experiment networks can not get /24 due to 
>> > high IPv4 prices, but they have to get an IPv4 prefix to learn BGP in IPv4 
>> > world.
>> > 
>> > What do you think about this?
>> 
>> Not going to happen any time soon (if at all).
>> 
>> #show ip route summary | i Source|---|bgp
>> Route SourceNumber Of Routes
>> - -
>> bgp   925809
>> 
>> Think about how much network gear is out there that is straining under the 
>> current size of the global table.  Opening the flood gates to many more 
>> prefixes with /25-/27 routes in the global table would mean lots of gear 
>> needs to be upgraded/replaced sooner rather than later.
>> 
>> --
>>   Jon Lewis, MCP :)   |  I route
>>   StackPath, Sr. Neteng   |  therefore you are
>> _ http://www.lewis.org/~jlewis/pgp for PGP public key_
> 
> 
> -- 
> --
> Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474

Re: maximum ipv4 bgp prefix length of /24 ?

[Owen DeLong via NANOG]

Wouldn’t /48s be a better solution to this need?

Owen


> On Sep 28, 2023, at 14:25, VOLKAN SALİH  wrote:
> 
> hello,
> 
> I believe, ISPs should also allow ipv4 prefixes with length between /25-/27 
> instead of limiting maximum length to /24..
> 
> I also believe that RIRs and LIRs should allocate /27s which has 32 IPv4 
> address. considering IPv4 world is now mostly NAT'ed, 32 IPv4s are sufficient 
> for most of the small and medium sized organizations and also home office 
> workers like youtubers, and professional gamers and webmasters!
> 
> It is because BGP research and experiment networks can not get /24 due to 
> high IPv4 prices, but they have to get an IPv4 prefix to learn BGP in IPv4 
> world.
> 
> What do you think about this?
> 
> What could be done here?
> 
> Is it unacceptable; considering most big networks that do full-table-routing 
> also use multi-core routers with lots of RAM? those would probably handle 
> /27s and while small networks mostly use default routing, it should be 
> reasonable to allow /25-/27?
> 
> Thanks for reading, regards..
> 

Re: AFRINIC placed in receivership

[Owen DeLong via NANOG]

> On Sep 15, 2023, at 06:30, Noah  wrote:
> 
> 
> 
> On Fri, 15 Sept 2023, 15:53 John Curran,  > wrote:
>> Noah - 
>> 
>> Indeed, that was a less than ideal situation – but I will note that the 
>> technical advisor was sent away by the Receiver once the Receiver was 
>> apprised of his litigation against AFRINIC.
> 
> 
> John
> 
> It was not a less than ideal situation. Please dont take things lightly here.

Not less than ideal? So was it ideal or more than ideal?

Owen

Re: JunOS config yacc grammar?

[Owen DeLong via NANOG]

FWIW, I find the config archival on-commit to be a very useful feature. I use 
it with SCP. Sadly, neither J nor C support doing this with an SSH private key 
on the router, you have to use a password. J at least encrypts the password if 
you do it right (…”URL” password “plain-text”). Cisco does not encrypt this 
password in the config (perhaps they do in more recent releases. All my C 
hardware is ancient. 

C will do it on “write”. Not quite as good as “on-commit”, but since C lacks 
commit, is what it is. 

YMMV. 

Owen


> On Aug 24, 2023, at 07:11, Christopher Morrow  wrote:
> 
> On Tue, Aug 22, 2023 at 11:39 PM Grant Taylor via NANOG  
> wrote:
>> 
>>> On 8/21/23 7:09 PM, Diogo Montagner wrote:
>>> I would first try to understand what you are trying to achieve. JUNOS is
>>> very flexible on this front and I am wondering why you think yacc is the
>>> right way to achieve what you are trying to do.
>> 
>> Drive by comment:
>> 
>> Perhaps the OP is trying to parse a (pile of) config file(s) downstream
>> of the generation thereof and has no ability to alter their generation.
> 
> this is a common problem (or is common when I look at things, perhaps
> I'm looking wrongly, but...)
> I'd love to have something that parsed all of my device type configs
> and output the results into a
> 'database' that i could then ask questions of like:
>  "Hey, what NTP servers are configured on all devices?"
>  "Hey, which devices have this  configured on 
> them?"
> 
> There are a host of other things I could ask those are but 2 simple
> examples, and YES I can
> grep/sed/awk|sort|uniq|sort-rn my way to success for the 2 examples I
> provided... but really
> that's NOT the way I want to do this, and I do really have a bunch of
> other questions I'd
> like to ask, regularly, to solve rollout-of-new-feature / compliance /
> legal / troubleshooting / etc
> questions.
> 
> In looking around there are examples of some of this, in a way, the
> most common thing
> I end up looking at, and getting sad about, is some java monstrosity
> (who's name escapes me)
> but has shown up in a few nanog presentations over the years... it
> makes me sad because it's
> not super useful  in my world :( 'hard to use' is probably the best
> way to describe it.
> 
> One note about XML and Juniper, the schema changes by OS version, it
> changes quite a bit :(
> You CAN parse through it reasonably well with python lxml.Etree,
> because (I think) python's parse
> is VERY forgiving. If you attempt this path with golang :( you will be
> sad, very sad :( because
> the go->xml world is very 'build a struct of structs that mirrors the
> xml tree' and 'changes at every
> OS version' means now you have a LOT of versions of that :(
> maintenance gets back to saku's
> comment about feature velocity :(
> 
> I do see:
>   https://pypi.org/project/juniper-nxpy/
> 
> which may be useful to you as well Lyndon.
> (I'd also point to tftp as not being the super best option from a
> security and reliability perspective,
> but if that's what you've got that's what you've got... you COULD have
> the switch cronjobs curl/post
> to an https destination with little hard work, and a gain in
> reliabilty/security)
> 
> -chris

Re: IP range for lease

[Owen DeLong via NANOG]

> On Jul 11, 2023, at 09:52, John Curran  wrote:
> 
> 
> 
>> On Jul 11, 2023, at 12:40 PM, Owen DeLong  wrote:
>> 
>> 
>>> On Jul 11, 2023, at 09:04, John Curran  wrote:
>>> 
>>> ...
>>> 
>>> Of course, further policy clarity (whether to make clear that it does apply 
>>> to non-connectivity reassignments or to make clear it does not apply beyond 
>>> downstream customers) would be most welcome; I believe you are already 
>>> aware of the policy proposal submission process if you want to propose 
>>> updating it accordingly.
>> 
>> All of the organizations I know of that are leasing space apply the term 
>> downstream as it pertains to the issuance of the addresses regardless of the 
>> connectivity relationship.
> 
> That may be the case, but since the earliest days of ARIN the term 
> “downstream” has been used by this operator community to refer to customer 
> connectivity, so we’ll maintain current usage until directed otherwise by the 
> community. 
> 
>> I suppose policy clarity here could be useful,
> 
> Indeed.
> 
>> … but I suspect that just like ISPs, the situation will basically boil down 
>> to “those that want to comply will do so in good faith and others will not.”
> 
> That is also up to the community, as there is an obvious tradeoff between 
> enforcement and registry accuracy – if the community wishes more accuracy in 
> the registry, there needs to be clarity in policy regarding what actions ARIN 
> should take with respect to non-compliance. 
> 
> Thanks!
> /John
> 
> John Curran
> President and CEO
> American Registry for Internet Numbers
> 
> 
> 


In fact, John, some further NRPM research reveals the following:

1.  Downstream references almost all apply to address issuance limitations 
and customer utilization limitations.
The only places the term is applied to registration are NRPM 4.2.3..3.2 
(Residential customer privacy),
NRPM 6.5.5.3.1 (Residential Customer Privacy)  and NRPM 6.5.5.4 
(Registration Requested by Recipient)..

2.  Language requiring registration of reallocations and reassignments are 
as follows:

4.2.3.7. Registration
ISPs are required to demonstrate efficient use of IP address space allocations 
by providing appropriate documentation, including but not limited to assignment 
histories, showing their efficient use.
4.2.3.7.1. Reassignment and Reallocation Information

Each IPv4 reassignment or reallocation containing a /29 or more addresses shall 
be registered via SWIP or a directory services system which meets the standards 
set forth in section 3.2.

Reassignment registrations must include each customer name, except where 
specifically exempted by this policy. Reassignment registrations shall only 
include point of contact (POC) information if either: (1) requested by the 
customer; or (2) the reassigned block is intended to be routed and announced 
outside of the provider’s network.

Reallocation registrations must contain the customer’s organization name and 
appropriate point of contact (POC) information.
4.2.3.7.2. Reassignments and Reallocations Visible Within Seven Days
All reassignments and reallocations shall be made visible as required in 
section 4.2.3.7.1 within seven calendar days of reassignment or reallocation.
6.5.5. Registration
ISPs are required to demonstrate efficient use of IP address space allocations 
by providing appropriate documentation, including but not limited to 
reassignment and reallocation histories, showing their efficient use.
6.5.5.1. Reassignment Information
Each static IPv6 reassignment or reallocation containing a /47 or more 
addresses, or subdelegation of any size that will be individually announced, 
shall be registered in the WHOIS directory via SWIP or a distributed service 
which meets the standards set forth in section 3.2. Reassignment and 
reallocation registrations shall include each client’s organizational 
information, except where specifically exempted by this policy.
6.5.5.2. Reassignments and Reallocations Visible Within Seven Days
All reassignments and reallocations shall be made visible as required in 
section 6.5.5.1 within seven calendar days of reassignment or reallocation.
3.  I don’t see anything ambiguous in that text that would exclude 
reassignments or reallocations independent of connectivity from the 
registration requirements.

4.  It is my belief that N$PM4 policies still govern IPv4 space held by 
ARIN subscribers regardless of whether it was obtained by the current 
registrant as a result of NRPM4 or NPRM8. Please let me know if that is in 
error.

Owen

Re: IP range for lease

[Owen DeLong via NANOG]

> On Jul 11, 2023, at 10:02, William Herrin  wrote:
> 
> On Tue, Jul 11, 2023 at 8:47 AM Owen DeLong via NANOG  wrote:
>>> – Leasing of IP address blocks independent of connectivity is not 
>>> explicitly recognized in ARIN number resource policy (i.e. there is no 
>>> policy that specifically allows or prohibits such activity.)
>> 
>> Correct me if I am wrong here, but in general, that which is not explicitly 
>> prohibited is implicitly allowed.
> 
> Hi Owen,
> 
> You're wrong-ish. "Address leasing" is not prohibited per se, it just
> doesn't count as in-use for the utilization requirements.

Yes, but that lack of counting while apparently not making it into the NRPM was 
definitely discussed
extensively with the community and the AC and ARIN staff. This is admittedly 
from memory, but
IIRC, the conclusion was that was the best possible interpretation of existing 
policy as written.

> Consider Amazon AWS. You can have an "elastic IP address" that's not
> attached to a running server. If it stays that way for most of the
> month, they charge you for it explicitly rather than wrap it up in the
> general server charge. In other words, they lease the address without
> any associated connectivity.

Well… Before the lawyers come after me, I’ll agree that $CLOUDPROVIDER
acts as you specify and that $CLOUDPROVIDER’s actions are completely
reasonable and function as you have described. (I’ve been repeatedly advised
to avoid using company names when discussing ARIN policy).

> Is that address in use per ARIN policy? I don't think it is. Has ARIN
> ever asked Amazon to detail the number of elastic IP addresses that
> are not actually in use when it sought more addresses? Probably not.
> Should they have? Only if there's reason to believe that there are a
> large enough number of such addresses to make a difference. Otherwise
> it's purposeless paperwork.

I think this is a very accurate summary of the current situation, yes.

I also suspect that this situation exists in numerous situations where ARIN
remains blissfully unaware of it even when it would matter. (Not necessarily
with any particular or named $CLOUDPROVIDER, but across all the
organizations that ARIN serves, I’d be surprised if none fit this description).

Owen

Re: IP range for lease

[Owen DeLong via NANOG]

> On Jul 11, 2023, at 09:04, John Curran  wrote:
> 
> 
>> On Jul 11, 2023, at 11:47 AM, Owen DeLong  wrote:
>> 
>> Actually, I couldn’t find anything in the NRPM which leads me to believe 
>> that there is any distinction in the documentation requirements for 
>> reassignment/reallocation regardless of associated connectivity. None of the 
>> policies seemed to specify this. As such, I would think that Connectivity 
>> Independent Leasing (CIL) and Connectivity Related Leasing (CRL) would be 
>> subject to exactly the same recording/reporting requirements.
>> 
> 
> Owen – 
> 
> ARIN NRPM Section 4.2.3.  "Reassigning and Reallocating Address Space to 
> Customers" utilizes the term “Downstream” in references to both downstream 
> end-users and downstream ISPs documentation requirements.
> 
> As the community has historically interpreted the phrase “downstream" to 
> refer to connectivity customers (and further that the requirements documented 
> are applied in oder to have accurate operational utilization), ARIN continues 
> to interpret the policy as applicable to reissuance of resources to 
> connectivity customers. 
> 
> Of course, further policy clarity (whether to make clear that it does apply 
> to non-connectivity reassignments or to make clear it does not apply beyond 
> downstream customers) would be most welcome; I believe you are already aware 
> of the policy proposal submission process if you want to propose updating it 
> accordingly.

All of the organizations I know of that are leasing space apply the term 
downstream as it pertains to the issuance of the addresses regardless of the 
connectivity relationship.

I suppose policy clarity here could be useful, but I suspect that just like 
ISPs, the situation will basically boil down to “those that want to comply will 
do so in good faith and others will not.”

Owen

Re: IP range for lease

[Owen DeLong via NANOG]

> On Jul 10, 2023, at 10:22, John Curran  wrote:
> 
>> On Jul 5, 2023, at 10:06 PM, Owen DeLong via NANOG  wrote:
>> ...
>> Opinions regarding leasing vary throughout the industry. In my opinion, 
>> since the shift to provider assigned addresses during the CIDR efforts in 
>> the mid 1990s, the majority of addresses have been leased in one form or 
>> another. 
>> 
>> The only thing novel here is the leasing of addresses independent of 
>> connectivity services. However, once the RIRs and their communities 
>> normalized the sale of addresses through directed transfer policies, I think 
>> this was an inevitable next step in the devolution of IPv4 into a monetized 
>> asset. 
>> 
>> It doesn’t help that the earliest and most prolific adopters of this form of 
>> leasing have been snowshoe spammers. 
>> 
>> However, there are leasing agencies that insist on getting proper 
>> justification from their customers and have strong anti-abuse policies. I 
>> would strongly encourage you to seek out such an organization to partner 
>> with if you choose to lease your addresses as there are a number of pitfalls 
>> you can encounter otherwise. 
> 
> To follow-up on Owen’s points and clarify just a bit (at least to respect to 
> policy in the ARIN region) – 
> 
> – IP address blocks in the ARIN region are issued by ARIN based upon 
> operational need (as per the community-developed policy document in the 
> Number Resource Policy Manual [NRPM - 
> https://www.arin.net/participate/policy/nrpm/] 
> <https://www.arin.net/participate/policy/nrpm/%5D>) 
> 
> – Portions of IP address blocks are routinely “leased” by ISPs to customers, 
> although such leasing has historically been as part of a bundle including 
> connectivity services.
> 
> – Because one needs IP addressed to provide connectivity services, leasing of 
> address space as part of providing connectivity is considered operational 
> need (and as such counts towards utilization of one’s address space) 
> 
> – Leasing of IP address space independent of connectivity doesn’t fulfill 
> operational need, and hence doesn’t count as utilization when you come back 
> to ARIN seeking additional space (or approval of a transfer inwards of an IP 
> address block)

Exceptions apply. For example, I know of situations where providers have 
continued to lease addresses to former customers that wanted to avoid 
renumbering,
yet ARIN has permitted those addresses to be counted as utilized during 
applications for additional space. I don’t know if these exceptions were 
intentional on
ARIN’s part or not, but they have definitely occurred and I’m not convinced 
that ARIN could reject them under existing policy.

> – Leasing of IP address blocks independent of connectivity is not explicitly 
> recognized in ARIN number resource policy (i.e. there is no policy that 
> specifically allows or prohibits such activity.) 

Correct me if I am wrong here, but in general, that which is not explicitly 
prohibited is implicitly allowed.

> – In the ARIN region, we have fairly clear guidelines requiring documentation 
> [via SWIP, RWHOIS, RDAP…] of significant reassignment/reallocations to 
> connectivity customers (as part of documenting IP address block usage), but 
> no clear requirements for reporting of reissuance of space via leasing 
> independent of connectivity.  Furthermore, all address blocks in the ARIN 
> registry are required to have accurate abuse contacts (unless residential in 
> which case accurate contacts must be in the upstream providers block.)

Actually, I couldn’t find anything in the NRPM which leads me to believe that 
there is any distinction in the documentation requirements for 
reassignment/reallocation regardless of associated connectivity. None of the 
policies seemed to specify this. As such, I would think that Connectivity 
Independent Leasing (CIL) and Connectivity Related Leasing (CRL) would be 
subject to exactly the same recording/reporting requirements.

> If folks wish to have the registry operate accordingly to some other 
> policies, please submit a policy proposal 
> <https://www.arin.net/participate/policy/pdp/appendix_b/> (or seek out a 
> member of the ARIN Advisory Council <https://www.arin.net/about/welcome/ac/> 
> which helps shepherd the policy development process and can assist you with 
> preparation of same…) 

I think that you know that if I had a problem with the current status quo, I 
would do exactly that. ;-) I have never hesitated in the past.

Owen

Re: IP range for lease

[Owen DeLong via NANOG]

Karin,Opinions regarding leasing vary throughout the industry. In my opinion, since the shift to provider assigned addresses during the CIDR efforts in the mid 1990s, the majority of addresses have been leased in one form or another. The only thing novel here is the leasing of addresses independent of connectivity services. However, once the RIRs and their communities normalized the sale of addresses through directed transfer policies, I think this was an inevitable next step in the devolution of IPv4 into a monetized asset. It doesn’t help that the earliest and most prolific adopters of this form of leasing have been snowshoe spammers. However, there are leasing agencies that insist on getting proper justification from their customers and have strong anti-abuse policies. I would strongly encourage you to seek out such an organization to partner with if you choose to lease your addresses as there are a number of pitfalls you can encounter otherwise. OwenOn Jul 3, 2023, at 08:25, Noah  wrote:Hi KARIM,Considering the fact that IPs are requested on need-basis by resource holders to number your own networks/systems and that of your clients?Any reason why MEKTEL would want to offer IPs for lease? Cheers,./noahOn Mon, Jul 3, 2023 at 6:16 PM KARIM MEKKAOUI  wrote:







Dear NANOG Community,
 
MEKTEL has a block of IPs that would like to offer in parts or in total for lease and would like to know from your experience your comments on the following:

The right priceThe challenges you facedWas it easy to get back your IPsWas your IPs black listed because of the inappropriate useAny other issue we need to care aboutEtc., etc.
 
Thank you in advance
 
KARIM
MEKTEL INC.
 

Re: Northern Virginia has had enough with data centers

[Owen DeLong via NANOG]

> On Jun 23, 2023, at 18:04, Michael Thomas  wrote:
> 
> 
>> On 6/23/23 4:01 PM, Delong.com via NANOG wrote:
>> The electric grid complaints are about the demand on the grid making the 
>> entire region less stable and proposed construction of new high-voltage 
>> tower corridors for data centers.
>> Yeah, I can kind of understand those, but as long as the grid is properly 
>> planned, it really shouldn’t have a destabilizing effect. In fact, many 
>> datacenter in California do CoGen and end up providing additional grid 
>> stability.
>> 
> Uh, ::cough:: PGE ::cough::
> 
> I so wanted to schadenfreude so bad with Texas and their shitty grid, but 
> then remembered where I live.
> 
> Mike

What’s not to love about a power company that literally qualifies as a 
recidivist felon?

It is my sincere hope to finish my mortgage and then start putting money 
towards electrical independence. (Wind, more solar, batteries, and 
disconnecting Persistent Graft and Extortion). 

Owen

Re: FCC Chair Rosenworcel Proposes to Investigate Impact of Data Caps

[Owen DeLong via NANOG]

> On Jun 16, 2023, at 13:16, Michael Thomas  wrote:
> 
> 
> On 6/16/23 1:09 PM, Mark Tinka wrote:
>> 
>> 
>> On 6/16/23 21:19, Josh Luthman wrote:
>>> Mark,
>>> 
>>> In my world I constantly see people with 0 fixed internet options.  Many of 
>>> these locations do not even have mobile coverage.  Competition is fine in 
>>> town, but for millions of people in the US (and I'm going to assume it's 
>>> worse or comparable in CA/MX) there is no service.
>>> 
>>> As a company primarily delivering to residents, competition is not a focus 
>>> for us and for the urban market it's tough to survive on a ~1/3 take rate.
>> 
>> I should have been clearer... the lack of competition in many markets is not 
>> unique to North America. I'd say all of the world suffers that, since there 
>> is only so much money and resources to go around.
>> 
>> What I was trying to say is that should a town or village have the 
>> opportunity to receive competition, where existing services are capped, 
>> uncapping that via an alternative provider would be low hanging fruit to 
>> gain local marketshare. Of course, the alternative provider would need to 
>> show up first, but that's a whole other thread.
>> 
> Won't Starlink and other LEO configurations be that backstop sooner rather 
> than later? I don't know if they have caps as well, but even if they do they 
> could compete with their caps.
> 
> Mike

Maybe, but ta the rate their prices have been going up and their reviews have 
been going down, it’s not at all promising. I’m in a wait and see mode with 
Starling. Every time I get just about frustrated enough with Comcast to shell 
out, I discover that Starling’s pricing has again moved above my threshold of 
frustration.

Owen