Re: IPv6 woes - RFC

[Michael Thomas]

On 9/13/21 2:52 PM, Baldur Norddahl wrote:



On Mon, Sep 13, 2021 at 8:22 PM Randy Bush > wrote:


real compatibility with ipv4 was disdained.  the transition plan was
dual stack and v4 would go away in a handful of years.  the 93
transition mechanisms were desperate add-ons when v4 did not go away.
and dual stack does not scale, as it requires v4 space proportional to
deployed v6 space.


What I find most peculiar about this whole rant (not just yours but 
the whole thread) is that I may be the only one who found implementing 
IPv6 with dual stack completely trivial and a non issue? There is no 
scale issue nor any of the other rubbish.


I agree on the host side. It didn't even occur to me at the time I was 
looking at it that it would be any sort of issue -- we had all kinds of 
other protocols on our boxes like SMB, Netware, DEC LAT, etc. We would 
have done it if customers told us they wanted it, just like we 
implemented ACL's not realizing why they were especially important. Back 
in the early days all routing was done in software so it wouldn't have 
been hard to squeeze v6 in. All of that changed when the forwarding 
plane got cast in silicon though which made it far, far more difficult 
to get anybody to stick their necks out vs a skunk works software 
project. But before that it would have been completely doable if 
somebody was willing to throw money at it.


Mike

Re: IPv6 woes - RFC

[Joe Maimon]

Baldur Norddahl wrote:



On Mon, Sep 13, 2021 at 8:22 PM Randy Bush > wrote:


real compatibility with ipv4 was disdained.  the transition plan was
dual stack and v4 would go away in a handful of years.  the 93
transition mechanisms were desperate add-ons when v4 did not go away.
and dual stack does not scale, as it requires v4 space proportional to
deployed v6 space.


What I find most peculiar about this whole rant (not just yours but 
the whole thread) is that I may be the only one who found implementing 
IPv6 with dual stack completely trivial and a non issue? There is no 
scale issue nor any of the other rubbish.


Baldur

The essential point is that your dual stack is barely relevant until 
every stack is dual. Which is impossible without CGNAT.


It also turns out that is barely relevant how easy it may have been for you.

Joe

RE: IPv6 woes - RFC

[Tony Wicks]

In resource challenged regions we have been using IPv4+CGN+IPv6 dual stack for 
the last ten or so years. For 20K subs you can use one /24 of ipv4 and a /40 or 
so of ipv6. There have been available RGW’s and sufficient vendor support 
throughout this time. The only issues I have ever really seen have been with 
Sony PSN doing random ipv4 /32 blocks. Apart from that it has been working just 
fine. Over 50% of traffic now flows on the V6 side and in general end customers 
have no clue, it just works.

 

>For all I care we already have a perfect working system with IPv4+CGN+IPv6. 
>The CGN part was the most troublesome, not the IPv6.

Re: IPv6 woes - RFC

[Michael Thomas]

On 9/13/21 11:22 AM, Randy Bush wrote:

< rant >

ipv6 was designed at a time where the internet futurists/idealists had
disdain for operators and vendors, and thought we were evil money
grabbers who had to be brought under control.

the specs as originally RFCed by the ietf is very telling.  for your
amusement, take a look at rfc 2450.  it took five years of war to get
rid of the tla/sla crap.  and look at the /64 religion today[0].

real compatibility with ipv4 was disdained.  the transition plan was
dual stack and v4 would go away in a handful of years.  the 93
transition mechanisms were desperate add-ons when v4 did not go away.
and dual stack does not scale, as it requires v4 space proportional to
deployed v6 space.

we are left to make the mess work for the users, while being excoriated
for not doing it quickly or well enough, and for trying to make ends
meet financially.

This is really easy to say in hindsight. 30 years ago it wasn't even 
vaguely a given that the Internet would even win and the size of the IP 
universe was still tiny. The main problem is that the internet was a 
classic success disaster where you're going as fast as possible and 
falling farther and farther behind. All of the gripes about particulars 
strike me as utterly irrelevant in the global scheme of things. As I 
mentioned, if they did nothing more than bolted on two more address 
bytes it still would have been just as impossible to get vendors and 
providers to care because everybody was heads down trying to deal with 
the success disaster. It's really easy to say that ipv6 suffers from 
second system syndrome -- which it does -- but that doesn't provide any 
concrete strategy for what would have been "better" in both getting 
vendors and providers to care. None of them wanted to do anything other 
than crank out kit that could be sold in the here and now that providers 
were willing to buy. That was certainly my experience at Cisco. As I 
said, the exec I talked to didn't actually want to do anything at all 
but was willing to let a couple of engineers navel gaze if it gave him 
something to talk about were the subject to actually come up with 
customers and a bludgeon against 3COM (iirc) at the time.


None of this is technical. It was which short term hack is going to keep 
the gravy train flowing? I was a developer at the time keeping an eye on 
the drafts as they were coming out. They didn't strike me was overly 
difficult to implement nor did they strike me as particularly 
overwrought. From a host standpoint, i didn't think it would take too 
much effort to get something up and running but I waited until somebody 
started asking for it. That never came. Nothing ever came. Then NAT's 
came and kicked the can down the roads some more. Now we have mega-yacht 
NAT's to kick it down the road even farther. Tell us what else would 
have prevented that?


Mike

Re: [EXTERNAL] Re: Xfi Advances Security (comcast)

[Livingood, Jason via NANOG]

On 9/13/21, 12:02, "Owen DeLong"  wrote:
> Yes, but it’s tragically opt-out instead of opt-in as it should be.

It is not a default for an Internet access service. It comes bundled as one of 
several features in an optional add on service. See 
https://www.xfinity.com/learn/internet-service/modems-and-routers for details. 
This is targeted at the average consumer, particularly those that may want 
parental controls, mesh WiFi, a voice port, and so on - so not really targeted 
at NANOG list subs like us. ;-) That said, I have an XB7 modem at home and 
really like it a lot - especially the new AQM feature that dramatically lowered 
working latency.

> That means that anyone whose site happens to get miscategorized by them gets 
> the added costs of dealing with the user complaints instead of Comcast having 
> to bear the costs of their error.

As my other reply noted, this service uses a bunch of 3rd party services and it 
is those 3rd parties that maintain the lists (a la anti-spam and anti-phishing 
email list vendors). So if an IP/FQDN/URL happens to be on "our" list it is 
very likely getting filtered/blocked in a lot of network places because it is 
on a well-known independent list.

BUT, how do we know that was even the case here? Do we have a traceroute or a 
screen shot of an error or block message? We seem to have concluded it was 
blocked by a content filter but what technical evidence do we have (that can 
help troubleshoot)? I know you are not the OP (it is Chris) - but I'd love to 
know more technical detail and I am in communication off-list with the OP 
(along with my colleague Tony Tauber, who was the first to reach out to Chris 
1:1).

Jason

Re: IPv6 woes - RFC

[Baldur Norddahl]

On Mon, Sep 13, 2021 at 8:22 PM Randy Bush  wrote:

> real compatibility with ipv4 was disdained.  the transition plan was
> dual stack and v4 would go away in a handful of years.  the 93
> transition mechanisms were desperate add-ons when v4 did not go away.
> and dual stack does not scale, as it requires v4 space proportional to
> deployed v6 space.
>

What I find most peculiar about this whole rant (not just yours but the
whole thread) is that I may be the only one who found implementing IPv6
with dual stack completely trivial and a non issue? There is no scale issue
nor any of the other rubbish.

Some say what we have is not a true dual stack setup because we run MPLS
and the IPv6 mostly lives inside L2VPN or L3VPN tunnels. Most of our
network gear has no idea what an IPv6 address is. But neither does that
equipment touch the public IPv4 internet. Nevertheless we configure our
IPv4 and IPv6 both only on our few edge Juniper MX devices and that is it.
I do not believe IPv6 has 100 config lines in my network total.

For all I care we already have a perfect working system with IPv4+CGN+IPv6.
The CGN part was the most troublesome, not the IPv6.

Regards,

Baldur

Re: IPv6 woes - RFC

[Randy Bush]

> it's easy to be critical of design decisions with 25y of hindsight

there was a good number of senior implementors and ops who screamed
loudly at the time.  to no avail.

randy

Re: IPv6 woes - RFC

[Nick Hilliard]

Randy Bush wrote on 13/09/2021 19:22:

the specs as originally RFCed by the ietf is very telling.  for your
amusement, take a look at rfc 2450.  it took five years of war to get
rid of the tla/sla crap.  and look at the /64 religion today[0].


architectural decisions were made because of a mixture of actual and 
perceived problems at the time, but several of the outcomes make little 
sense now or in some cases, actively cause problems.  E.g. using mcast 
for address resolution because large flat l2 networks were the order of 
the day, that "privacy addresses" would give privacy, that client 
self-selected addresses should be the only game in town for 
auto-addressing (it took years to get any form of dhcp through), that 
extension headers were a great idea, that "transition mechanisms" would 
be viable for fundamentally incompatible protocols, etc.  That said, 
it's easy to be critical of design decisions with 25y of hindsight, and 
even easier to understate how difficult it is to dislodge ipv4 which 
took 40 years of evolution to cement itself into its current position.


Nick

Re: IPv6 woes - RFC

[Randy Bush]

< rant >

ipv6 was designed at a time where the internet futurists/idealists had
disdain for operators and vendors, and thought we were evil money
grabbers who had to be brought under control.

the specs as originally RFCed by the ietf is very telling.  for your
amusement, take a look at rfc 2450.  it took five years of war to get
rid of the tla/sla crap.  and look at the /64 religion today[0].

real compatibility with ipv4 was disdained.  the transition plan was
dual stack and v4 would go away in a handful of years.  the 93
transition mechanisms were desperate add-ons when v4 did not go away.
and dual stack does not scale, as it requires v4 space proportional to
deployed v6 space.

we are left to make the mess work for the users, while being excoriated
for not doing it quickly or well enough, and for trying to make ends
meet financially.

randy, trying to deal with the mess since the early '90s

---

[0] https://datatracker.ietf.org/doc/html/draft-bourbaki-6man-classless-ipv6-05

Re: Amazon Prime Video contact

[L Sean Kennedy]

Amazon does have a process to investigate these issues and unfortunately
the information on the Brothers WISP page is dated (that page took over the
nanog.cluepon.net data which was referenced in previous NANOG
presentations).   I strongly recommend that any ISP who has blocked IPs or
customers reporting issues ensure that an Amazon ticket is opened.
Although it is possible to open such a ticket through CS channels, I am
happy to open a ticket for anyone who contacts me off-list (and I am
currently opening a ticket for David).

There are several data providers which allow you to search their
information and also report false positives which is important given that
multiple streaming providers can use the same data, but unfortunately not
all are as transparent.  So the feedback on this provider is helpful and
useful, but I would suggest contacting the streaming providers too.  And I
would like to know of ISPs having issues with Prime Video - there are
multiple data sources used, so if you use a whack-a-mole approach you may
run into a frustrating dead-end.

Thanks,
 Sean


Em seg., 13 de set. de 2021 às 13:36, Elijah 
escreveu:

> Geoguard takes care of Amazon, they were pretty responsive for me.
>
> n...@geoguard.com
>
> Elijah Zeida
> Network Administrator
> AirBridge Broadband
> 208-494-4343x1001
> https://airbridgebroadband.com/
>
> On 9/13/2021 7:41 AM, David Brain wrote:
> > Hi,
> >
> > Is there a contact for Amazon Prime Video - we are seeing a block of
> > IPs that are used for a content filtering solution being flagged as
> > proxies (which they are, but not in that way...) - which is impacting
> > a number of our customers.
> >
> > Thanks.
> >
> > David.
> >
>

Re: Amazon Prime Video contact

[Elijah]

Geoguard takes care of Amazon, they were pretty responsive for me.

n...@geoguard.com

Elijah Zeida
Network Administrator
AirBridge Broadband
208-494-4343x1001
https://airbridgebroadband.com/

On 9/13/2021 7:41 AM, David Brain wrote:

Hi,

Is there a contact for Amazon Prime Video - we are seeing a block of
IPs that are used for a content filtering solution being flagged as
proxies (which they are, but not in that way...) - which is impacting
a number of our customers.

Thanks.

David.

Re: Xfi Advances Security (comcast)

[Owen DeLong via NANOG]

> On Sep 13, 2021, at 07:56 , Livingood, Jason via NANOG  
> wrote:
> 
> On 9/10/21, 10:58, "NANOG on behalf of Chris Boyd" 
>  cb...@gizmopartners.com> wrote:
> 
>> Why is Comcast blocking things? That seems like it’s out of scope for an ISP.
> 
> For Internet access, sure. But ISPs also have value added protection services 
> and this part of an optional content filtering service that is integrated 
> into the leased Comcast gateways. Users can turn on things like parental 
> controls, including time limit and time-of-day boundaries for certain devices 
> (e.g. cut off kid's game console Internet access at midnight on school 
> nights). See 
> https://www.xfinity.com/support/articles/using-xfinity-xfi-advanced-security
> 
> Jason
> 
> 

Yes, but it’s tragically opt-out instead of opt-in as it should be. That means 
that anyone whose site happens to get miscategorized by them gets the added 
costs of dealing with the user complaints instead of Comcast having to bear the 
costs of their error.

It’s a classic example of the toxic polluter business model. Do something 
stupid while making sure that the costs of your errors fall on someone else.

Owen

Re: IPv6 woes - RFC

[Owen DeLong via NANOG]

> On Sep 13, 2021, at 05:17 , Mark Tinka  wrote:
> 
> 
> 
> On 9/13/21 01:00, Michael Thomas wrote:
> 
>> 
>> If vendors actually cared they could make the CGNAT's and other hacks 
>> ridiculously buggy and really expensive to deploy and maintain. I doubt many 
>> vendors were chomping at the bit to support CGNAT and are probably wondering 
>> what fresh hell is next to keep ipv4 limping along.
>> 
> 
> 10's of millions of dollars being spent by African mobile operators, every 
> year, in CG-NAT hardware and licenses, with the vendors.
> 
> They will make sure that code is bug-free :-).

ROFLMAO

100s of millions of dollars are spent every year on major backbone router kit. 
That code has never been bug free.

Your assertion is provably false.

Owen

Amazon Prime Video contact

[David Brain]

Hi,

Is there a contact for Amazon Prime Video - we are seeing a block of
IPs that are used for a content filtering solution being flagged as
proxies (which they are, but not in that way...) - which is impacting
a number of our customers.

Thanks.

David.

-- 
David Brain - MCNC - 919.248.1998

Re: Xfi Advances Security (comcast)

[Livingood, Jason via NANOG]

As Alex said, you can submit a request to review a block at 
https://spa.xfinity.com.
 Note that this service relies substantially on 3rd party list sources – so if 
any IP/FQDN appears on other lists (e.g. webroot and similar) then it may be 
here as well. So you may want to take a look more broadly, especially if you 
rely on any virtual infrastructure.

Thanks
Jason

From: NANOG  on 
behalf of Jason Kuehl 
Date: Friday, September 10, 2021 at 11:10
To: Jim Popovitch 
Cc: NANOG 
Subject: Re: Xfi Advances Security (comcast)

This is an SSL VPN that is being blocked. This is what failure looks like. Curl 
is the same.

Once we disable the Xfi  Advanced Security everyone can connect.

[cid:ii_ktehov470]

On Fri, Sep 10, 2021 at 11:01 AM Jim Popovitch via NANOG 
mailto:nanog@nanog.org>> wrote:
On Fri, 2021-09-10 at 10:31 -0400, Jason Kuehl wrote:
> For whatever reason Comcast Xfinity is blocking my VPN URL.

Not certain that this applies, but Concast Advanced Security (setup in
your Comcast gateway) only allows outbound VPN connections to UDP ports
500, 4500, and 62515 and TCP port 1723.

-Jim P.


--
Sincerely,

Jason W Kuehl
Cell 920-419-8983
jason.w.ku...@gmail.com

Re: Xfi Advances Security (comcast)

[Livingood, Jason via NANOG]

On 9/10/21, 10:58, "NANOG on behalf of Chris Boyd" 
 wrote:

> Why is Comcast blocking things? That seems like it’s out of scope for an ISP.

For Internet access, sure. But ISPs also have value added protection services 
and this part of an optional content filtering service that is integrated into 
the leased Comcast gateways. Users can turn on things like parental controls, 
including time limit and time-of-day boundaries for certain devices (e.g. cut 
off kid's game console Internet access at midnight on school nights). See 
https://www.xfinity.com/support/articles/using-xfinity-xfi-advanced-security

Jason

Re: IPv6 woes - RFC

[Mark Tinka]

On 9/13/21 02:16, Michael Thomas wrote:



But it's hardly uniform across the industry. This is a classic 
reverse-tragedy of the commons.


The problem is it's uniform in the corners that contain scale and the 
money to make a difference at vendor-land.


7 million mom & pop ISP's vs. 10 large mobile operators... yeah, $vendor 
will struggle to make that decision :-).


Mark.

Re: IPv6 woes - RFC

[Mark Tinka]

On 9/13/21 01:00, Michael Thomas wrote:



If vendors actually cared they could make the CGNAT's and other hacks 
ridiculously buggy and really expensive to deploy and maintain. I 
doubt many vendors were chomping at the bit to support CGNAT and are 
probably wondering what fresh hell is next to keep ipv4 limping along.




10's of millions of dollars being spent by African mobile operators, 
every year, in CG-NAT hardware and licenses, with the vendors.


They will make sure that code is bug-free :-).

Mark.

RE: Xfi Advances Security (comcast)

[Brotman, Alex via NANOG]

https://spa.xfinity.com should have a form to request removal.  Note they say 
resolution time can be up to three business days

--
Alex Brotman
Sr. Engineer, Anti-Abuse & Messaging Policy
Comcast

From: NANOG  On Behalf Of 
Jason Kuehl
Sent: Friday, September 10, 2021 10:31 AM
To: NANOG 
Subject: Xfi Advances Security (comcast)

For whatever reason Comcast Xfinity is blocking my VPN URL. I've started the 
process to unblock, and I'm trying to get a hold of their security team to 
resolve this. I've been bounced around all morning.

Does anyone have a contact at Comcast that can whitelist a URL or get me to a 
team that can understand what is going on for the block to happen?

--
Sincerely,

Jason W Kuehl
Cell 920-419-8983
jason.w.ku...@gmail.com

Re: Xfi Advances Security (comcast)

[Matt Goldman via NANOG]

We ran into this same issue for the first time yesterday too.  Xfi Advanced 
Security started blocking our websockets endpoint, 
websocket.carsandbids.com/carsandbids.  Our logs just showed a couple users 
failing to make the connection.  We never would have figured it out except that 
one of our employees was getting blocked and was able to help us debug.

We submitted the url to https://spa.xfinity.com/report to get whitelisted, but 
haven’t heard back yet, the form said estimated three business days.

Matt Goldman
Backend Engineer
matt at carsandbids dot com