Re: QUIC traffic throttled on AT residential

[Dan Wing]

There are choices, such as making connection initiation, connection acceptance, 
and connection termination parsable by network elements on the path so state 
can be established, maintained, and cleared, DoS can be identified, and so on.  
The decision was to hide all that from network elements.

-d


> On Feb 20, 2020, at 7:54 PM, Matthew Kaufman  wrote:
> 
> 
> 
> On Thu, Feb 20, 2020 at 8:10 AM Ca By  > wrote:
> 
> 
> Not indiscriminate. 
> 
> As Google was informed by network operators all along since 2014, ipv4 UDP is 
> a major uptime threat via DDoS to access networks.  
> ...
> 
> Google choose not to be sensitive to that, they were told where the landmines 
> were, they choose to go on anyhow. 
> 
> 
> It isn’t like they had a choice. You can’t build a transport protocol like 
> QUIC on top of TCP (I know, I built one of its ancestors, which also uses UDP 
> underneath). And if you think getting UDP through existing networks is hard, 
> try using a novel IP protocol number.
> 
> Matthew Kaufman
> 

Re: QUIC traffic throttled on AT residential

[Dan Wing]

Yes, other than AT increasing their permitted incoming UDP traffic -- the 
easiest thing AT can do -- AT could ask the vendor of their flow 
restricting device to use bi-directional UDP traffic on same 5-tuple to 
indicate "desire to receive", rather than solely examining incoming UDP traffic 
as they are doing today; this is not easy but also not impossible.

While it is true Youtube/Google could treat AT connections as 'bad' 
and force TCP, but I am sure Youtube/Google is already gathering those 
statistics and already aware that AT is throttling.  For all we know, you and 
the others noticing the issue have fallen into the pit of A/B testers checking 
for their current throttling, and others aren't being throttled.  Perhaps it's 
everyone, the tests are not well described or well announced.  Youtube/Google 
is hoping customers complain to AT so that AT removes or improves the flow 
restrictor, because otherwise AT customers won't be able to get QUIC.  
Similarly, AT could protect their users from AT's own rate limiting by 
blocking QUIC towards major servers that support QUIC but such blocking becomes 
problematic as QUIC rolls out beyond Google to Cloudflare and elsewhere.

This tussle has similarities to IPv6 vs IPv4.

-d


> On Feb 18, 2020, at 4:00 PM, Ca By  wrote:
> 
> 
> 
> On Tue, Feb 18, 2020 at 5:44 PM Daniel Sterling  > wrote:
> I've AT fiber (in RTP, NC) (AS7018) and I notice UDP QUIC traffic
> from google (esp. youtube) becomes very slow after a time.
> 
> This especially occurs with ipv4 connections. I'm not the only one to
> notice; a web search for e.g. "Extremely Poor Youtube TV Performance"
> notes the issue.
> 
> I assume traffic is being throttled on AT's side. And it's not done
> with their CPE; I'm bypassing their NAT box and connecting my laptop
> directly to the ONT.
> 
> A quick google search shows people are aware that QUIC can look like
> DoS traffic -- but how widely known is this problem? It may become
> worse if / when sites transition to HTTP/3
> 
> For now I reject UDP 443 on the client firewall. Applications using
> QUIC client libraries then fallback to TCP. This works well and I have
> no issues with latency or throughput when using TCP.
> 
> May I naively ask if Google staff are working with AT to address this?
> 
> Sounds like you found the answer, ATT just needs to scale up your rejection 
> approach that is proven to work well. 
> 
> Yes, most ddos traffic is ipv4 udp and yes Google was made aware that they 
> would be mixing with bad company in the pool of ipv4 udp traffic ... but they 
> have reasons. 
> 
> I am not a fan of quic or any udp traffic. My suggestion was that Google use 
> an new L4 instead of UDP, but that was too hard for the Googlers. 
> 
> So here we are.
> 
> Just say no to udp. 
> 
> 
> 
> 
> -- Dan

AT postmaster

[Dan Wing]

I help run a community machine (not for my employer) and we've been trying 
unsuccessfully for a month to get off AT's email blacklist; their automated 
systems claim we're off, but mail is still being blocked. Can an AT 
postmaster drop me an email to help work through this?

Thanks.
-d

Re: real-world data about fragmentation

[Dan Wing]

On Apr 2, 2014, at 11:14 AM, Joe Abley jab...@hopcount.ca wrote:

 Hi all,
 
 It's common wisdom that a datagram that needs to be fragmented between 
 endpoints (because it is bigger than the path MTU) will demonstrate less 
 reliable delivery and reassembly than a datagram that doesn't need to be 
 fragmented, because math, firewall, other, take your pick.
 
 Is anybody aware of any wide-scale studies that examine the probability of 
 fragmentation of datagrams of different sizes?
 
 For example, I could reasonable expect an IPv4 packet of 576 bytes not to be 
 fragmented very often (to choose a size not at random). The probability of a 
 10,000 octet IPv4 packet getting fragmented seems likely to be 100%, if we're 
 talking about arbitrary paths across the Internet.
 
 What does the curve look like between 576 bytes and 10,000 bytes?
 
 I might expect exciting curve action around 1500 bytes (because ethernet), 
 1492 (PPPoE), 1480 (GRE), etc. But I'm interested in actual data.
 
 Anybody have any pointers? IPv4 and IPv6 are both interesting.

Seems a good thing for RIPE Atlas probes to measure.  But they are probably not 
generally connected to representative networks (read: poor networks).

-d

RE: CGN fixed/hashed nat question

[Dan Wing]

 -Original Message-
 From: Eric Oosting [mailto:eric.oost...@gmail.com]
 Sent: Monday, January 21, 2013 9:06 AM
 To: NANOG
 Subject: CGN fixed/hashed nat question
 
 Let me start out by saying I'm allergic to CGN, but I got to ask the
 question:
 
 Some of the CGN providers are coming out with fixed nat solutions for
 their IPv6 transition/IPv4 preservation technologies to reduce logging.
 This appears to provide for a static mapping of outside ports/IPs to a
 particular customer such that the service provider doesn't need to log
 literally every session through the box.
 
 At the last nanog, I seem to remember someone stepping up and discussing
 the problems associated with just taking ports 1025 through 1025+X and
 giving it to some customer and had brought up the idea of using a hash
 or salt to map what would appear to be random ports to a customer in
 such a way that you could reverse the port back to the customer later if
 need be.
 For the life of me, I can't find anything on the internets about this
 concept.
 
 I had it in my head it was a lightning talk or something, but reviewing
 the agenda doesn't ring any bells. Anyone know what I'm talking about
 and what it's called?


later, Eric Oosting eric.oost...@gmail.com wrote:

  draft-donley-behave-deterministic-cgn
 
 That's it. Or more specifically, the section of that draft that points
 to https://tools.ietf.org/html/rfc6431#section-2.2

I am also not a fan of CGN or NAT, but I co-chair the IETF's BEHAVE
working group that works on NAT.

draft-donley-behave-deterministic-cgn provides that functionality in
an attempt to help randomize ports (see RFC6056).  However, because
the ports are fixed and there are relatively few ports, an attacker
can determine the ports by causing the victim to open a bunch 
of TCP connections.  This can be done by a bunch of img src tags
in an HTML-encoded email message, among other mechanisms.  If the
hashing causes no logging, it creates a new requirement for a strong
audit trail of the CGN configuration.

The hashing provided by draft-donley-behave-deterministic-cgn is 
not necessary to avoid logging every session.  Rather, the reduction
occurs by generating 1 logging event when creating  mapped
ports.  If using the CGN configuration, then no logging event needs
to be generated.

To date, the BEHAVE working group has not standardized any of
the proposed hashing techniques because several require coordination
between the devices (such as CPE and CGN), or between the log
generator and log consumer, or are functions self-contained within
a device and don't require standards action.

-d

RE: what about the users re: NAT444 or ?

[Dan Wing]

 One can do that with or without NAT. This claim that one cannot
 keep a network running without a service provider connected if you
 don't run NAT is a myth of dubious origin.

If the hosts are running DHCP, and the ISP is running the DHCP
server?  I guess they will fall back (after a while) to link-local
and continue on their merry way.

  can accomplish this pretty easily, because the IPv4 addresses in
  the home can be any IPv4 address whatsoever -- which allows the
  in-home CPE (B4, in Dual Stack-Lite parlance) to assign any address
  it wants with its built-in DHCP server.)
 
 
 There are other ways to accomplish this as well.

-d

  -d
 
  and less technically but relevant I think is to ask about cost? who
  pays?
 
 In some cases, ISPs will provide new CPE to their end users. In other
 cases,
 end-users will be expected to pay to upgrade their own.
 
 Owen
 
 
 
  Christian
 
  On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
 
  On Sep 8, 2011 1:47 AM, Leigh Porter
 leigh.por...@ukbroadband.com
  wrote:
 
 
 
  -Original Message-
  From: Owen DeLong [mailto:o...@delong.com]
  Sent: 08 September 2011 01:22
  To: Leigh Porter
  Cc: Seth Mos; NANOG
  Subject: Re: NAT444 or ?
 
  Considering that offices, schools etc regularly have far more
 than
  10
  users per IP, I think this limit is a little low. I've happily
 had
  around 300 per public IP address on a large WiFi network, granted
  these
  are all different kinds of users, it is just something that
  operational
  experience will have to demonstrate.
 
  Yes, but, you are counting individual users whereas at the NAT444
  level, what's really being counted is end-customer sites not
  individual
  users, so the term
  users is a bit misleading in the context. A given end-customer
  site
  may be from 1 to 50 or more individual users.
 
  Indeed, my users are using LTE dongles mostly so I expect they
 will
  be
  single users. At the moment on the WiMAX network I see around 35
  sessions
  from a WiMAX modem on average rising to about 50 at peak times.
 These
  are a
  combination of individual users and home modems.
 
  We had some older modems that had integrated NAT that was broken
 and
  locked up the modem at 200 sessions. Then some old base station
  software
  died at about 10K sessions. So we monitor these things now..
 
 
 
  I would love to avoid NAT444, I do not see a viable way around
 it
  at
  the moment. Unless the Department of Work and Pensions release
  their /8
  that is ;-)
 
 
  The best mitigation really is to get IPv6 deployed as rapidly and
  widely as possible. The more stuff can go native IPv6, the less
  depends
  on fragile NAT444.
 
  Absolutely. Even things like google maps, if that can be dumped on
  v6,
  it'll save a load of sessions from people. The sooner services such
  as
  Microsoft Update turn on v6 the better as well. I would also like
 the
  CDNs
  to be able to deliver content in v6 (even if the main page is v4)
  which
  again will reduce the traffic that has to traverse any NAT.
 
  Soon, I think content providers (and providers of other services
 on
  the
  'net) will roll v6 because of the performance increase as v6 will
 not
  have
  to traverse all this NAT and be subject to session limits, timeouts
  and
  such.
 
 
  What do you mean by performance increase? If performance equals
  latency, v4
  will win for a long while still. Cgn does not add measurable
 latency.
 
  Cb
  --
  Leigh
 
 
 
 
 __
  This email has been scanned by the MessageLabs Email Security
  System.
  For more information please visit http://www.messagelabs.com/email
 
 
 __
 
 
 

RE: NAT444 or ?

[Dan Wing]

 -Original Message-
 From: Owen DeLong [mailto:o...@delong.com]
 Sent: Tuesday, September 13, 2011 9:43 PM
 To: Dan Wing
 Cc: 'Leigh Porter'; 'David Israel'; nanog@nanog.org
 Subject: Re: NAT444 or ?
 
 
  Good point, but aside from these scaling issues which I expect can
 be
  resolved to a point, the more serious issue, I think, is
 applications
  that just do not work with double NAT. Now, I have not conducted any
  serious research into this, but it seems that draft-donley-nat444-
  impacts does appear to have highlight issues that may have been down
 to
  implementation.
 
  Draft-donley-nat444-impacts conflates bandwidth constraints with CGN
  with in-home NAT.  Until those are separated and then analyzed
 carefully,
  it is harmful to draw conclusions such as NAT444 bad; NAT44 good.
 
 
 Continuing to make this claim does not make it any more true.
 
 Draft-donley took networks and measured their real-world functionality
 without NAT444, then, added NAT444 and repeated the same tests.
 Regardless of the underlying issue(s), the addition of NAT444 to the
 mix resulted in the forms of service degradation enumerated in the
 draft.

I disagree it reached that conclusion.  That may have been its
intent.

 Further, I would not ever say NAT444 bad; NAT44 good. I would say,
 rather, NAT44 bad, NAT444 worse. I think that's a pretty safe and
 non-harmful thing to say.

Yes, your statement is completely accurate.  I agree that IPv4 address 
sharing causes additional problems (which encompasses all forms of 
IPv4 address sharing), and CGN causes additional problems.

  Other simple tricks such as ensuring that your own internal services
  such as DNS are available without traversing NAT also help.
 
  Yep.  But some users want to use other DNS servers for performance
  (e.g., Google's or OpenDNS servers, especially considering they
  could point the user at a 'better' (closer) CDN based on Client
  IP), to avoid ISP DNS hijacking, or for content control (e.g.,
  parental control of DNS hostnames).  That traffic will,
 necessarily,
  traverse the CGN.  To avoid users burning through their UDP port
  allocation for those DNS queries it is useful for the CGN to
  have short timeouts for port 53.
 
 If the user chooses to use a DNS server on the other side of a NAT,
 then,
 they are choosing to inflict whatever damage upon themselves. I'm not
 saying that short UDP/53 timeouts are a bad idea, but, I am saying that
 the more stuff you funnel through an LSN at the carrier, the more stuff
 you will see break. This would lead me to want to avoid funneling
 anything
 through said NAT which I could avoid. Then again, I run my own
 authoritative and recursive nameservers in my home and don't use
 any NAT at all, so, perhaps my perspective is different from others.

Yeah, you are probably of about 1000 or maybe 3000 people in the 
world that do that.  Seems to be a minority.

  Certainly some more work can be done in this area, but I fear that
 the
  only way a real idea as to how much NAT444 really doe break things
 will
  be operational experience.
 
  Yep.  (Same as everything else.)
 
 
 I'm sure that will happen soon enough. I, for one, am not looking
 forward to the experience.

Neither am I.

But if major content providers cannot provide  on their
properties, and if ISPs and CPE vendors do not make IPv6
available and working, and if web browsers don't adopt faster
fallback to IPv4 when IPv6 is borked   We will all be 
behind NATs.

-d

RE: NAT444 or ?

[Dan Wing]

 -Original Message-
 From: Geoff Huston [mailto:g...@apnic.net]
 Sent: Wednesday, September 07, 2011 10:27 PM
 To: Leigh Porter
 Cc: nanog@nanog.org list; Daniel Roesen
 Subject: Re: NAT444 or ?
 
 
 On 08/09/2011, at 2:41 AM, Leigh Porter wrote:
 
 
 
  -Original Message-
  From: Daniel Roesen [mailto:d...@cluenet.de]
  Sent: 07 September 2011 17:38
  To: nanog@nanog.org
  Subject: Re: NAT444 or ?
 
  On Wed, Sep 07, 2011 at 12:16:28PM +0200, Randy Bush wrote:
  I'm going to have to deploy NAT444 with dual-stack real soon now.
 
  you may want to review the presentations from last week's apnic
  meeting
  in busan.  real mesurements.  sufficiently scary that people who
 were
  heavily pushing nat444 for the last two years suddenly started to
 say
  it was not me who pushed nat444, it was him!  as if none of us
 had
  a
  memory.
 
  Hm, I fail to find relevant slides discussing that. Could you please
  point us to those?
 
  I'm looking at http://meetings.apnic.net/32
 
  There is a lot in the IPv6 plenary sessions:
 
  http://meetings.apnic.net/32/program/ipv6
 
  This is what I am looking at right now. Randy makes some good
 comments in those sessions. I have not found anything yet, but I am
 only on session 3, pertaining specifically to issues around NAT444.
 
 It may not be what Randy was referring to above, but as part of that
 program at APNIC32 I reported on the failure rate I am measuring for
 Teredo. I'm not sure its all in the slides I was using, but what I was
 trying to say was that STUN is simply terrible at reliably negotiating
 a NAT.

Teredo does not use STUN; Teredo was implemented before STUN was fully
spec'd and does its own thing.

Teredo tries to determine if the type of NAT it is behind (cone, 
symmetric, etc.)  Determining the type of NAT has been found to 
be difficult, and nearly impossible (*) and removed from the current
RFC for STUN (**).  I suspect most of Teredo's failures are related 
to this procedure not working effectively.  A CGN can't improve that.

(*) http://tools.ietf.org/html/rfc5780#section-1
(**) http://tools.ietf.org/html/rfc5389#section-2

 I was then wondering what pixie dust CGNs were going to use that
 would have any impact on the ~50% connection failure rate I'm observing
 in Teredo.  And if there is no such thing as pixie dust (damn!) I was
 then wondering if NATs are effectively unuseable if you want anything
 fancier than 1:1 TCP connections (like multi-user games, for example).
 After all, a 50% connection failure rate for STUN is hardly encouraging
 news for a CGN deployer if your basic objective is not to annoy your
 customers.

If the CGN has both Endpoint-Independent Filtering (***) behavior
and Endpoint-Independent Mapping () behavior, the CGN won't make
Teredo worse -- Teredo will be as bad as today, caused by the home
user's own pretty NAT.  With that behavior, it also won't make 
applications that use STUN or ICE worse, or applications that use 
STUN-like or ICE-like techniques such as Skype.

(***) endpoint-independent filtering: means it doesn't filter incoming
packets after a mapping is created.  See
http://tools.ietf.org/html/rfc4787#section-5 for canonical definition.

() Endpoint-Independent Mapping:  means when the internal host sends a
packet with the same source port, to any destination, the same public port
is mapped.   See http://tools.ietf.org/html/rfc4787#section-4.1 for
canonical definition

-d

RE: NAT444 or ?

[Dan Wing]

...
 The striking thing I picked up is that NTT considers the CGN equipment
 a big black hole where money goes into. Because it won't solve their
 problem now or in the future and it becomes effectively a piece of
 equipment they need to buy and then scrap soon after.

It would get scrapped when all servers support dual stack.  What year
is that predicted to occur?

 They acknowledge the need, but they'd rather not buy one.
 That and they (the isp) get called for anything which doesn't work.

-d

RE: what about the users re: NAT444 or ?

[Dan Wing]

 -Original Message-
 From: Christian de Larrinaga [mailto:c...@firsthand.net]
 Sent: Thursday, September 08, 2011 8:05 AM
 To: Cameron Byrne
 Cc: NANOG
 Subject: what about the users re: NAT444 or ?
 
 I wonder if the discussion as useful as it is isn't forgetting that the
 edge of Internet has a stake in getting this right too! This is not
 just an ISP problem but one where content providers and services that
 is the users need to get from here to there in good order.
 
 So
 
 What can users do to encourage ISPs to deploy v6 to them?
 What can users do to ease the pain in reaching IPv4 only sites once
 they are on IPv6 tails?
 
 Is there not a bit of CPE needed here? What should the CPE do? and not
 do? should it deprecate NAT/PAT when it receives 1918 allocation from a
 CGN?

Careful with that idea -- people like their in-home network to continue
functioning even when their ISP is down or having an outage.  Consider
a home NAS holding delivering content to the stereo or the television.
It is possible to eliminate reliance on the ISP's network and still
have the in-home network function, but it's more difficult than just
continuing to run NAT44 in the home like today.  (Dual Stack-Lite
can accomplish this pretty easily, because the IPv4 addresses in
the home can be any IPv4 address whatsoever -- which allows the
in-home CPE (B4, in Dual Stack-Lite parlance) to assign any address
it wants with its built-in DHCP server.)

-d

 and less technically but relevant I think is to ask about cost? who
 pays?
 
 
 Christian
 
 On 8 Sep 2011, at 15:02, Cameron Byrne wrote:
 
  On Sep 8, 2011 1:47 AM, Leigh Porter leigh.por...@ukbroadband.com
 wrote:
 
 
 
  -Original Message-
  From: Owen DeLong [mailto:o...@delong.com]
  Sent: 08 September 2011 01:22
  To: Leigh Porter
  Cc: Seth Mos; NANOG
  Subject: Re: NAT444 or ?
 
  Considering that offices, schools etc regularly have far more than
 10
  users per IP, I think this limit is a little low. I've happily had
  around 300 per public IP address on a large WiFi network, granted
 these
  are all different kinds of users, it is just something that
 operational
  experience will have to demonstrate.
 
  Yes, but, you are counting individual users whereas at the NAT444
  level, what's really being counted is end-customer sites not
 individual
  users, so the term
  users is a bit misleading in the context. A given end-customer
 site
  may be from 1 to 50 or more individual users.
 
  Indeed, my users are using LTE dongles mostly so I expect they will
 be
  single users. At the moment on the WiMAX network I see around 35
 sessions
  from a WiMAX modem on average rising to about 50 at peak times. These
 are a
  combination of individual users and home modems.
 
  We had some older modems that had integrated NAT that was broken and
  locked up the modem at 200 sessions. Then some old base station
 software
  died at about 10K sessions. So we monitor these things now..
 
 
 
  I would love to avoid NAT444, I do not see a viable way around it
 at
  the moment. Unless the Department of Work and Pensions release
 their /8
  that is ;-)
 
 
  The best mitigation really is to get IPv6 deployed as rapidly and
  widely as possible. The more stuff can go native IPv6, the less
 depends
  on fragile NAT444.
 
  Absolutely. Even things like google maps, if that can be dumped on
 v6,
  it'll save a load of sessions from people. The sooner services such
 as
  Microsoft Update turn on v6 the better as well. I would also like the
 CDNs
  to be able to deliver content in v6 (even if the main page is v4)
 which
  again will reduce the traffic that has to traverse any NAT.
 
  Soon, I think content providers (and providers of other services on
 the
  'net) will roll v6 because of the performance increase as v6 will not
 have
  to traverse all this NAT and be subject to session limits, timeouts
 and
  such.
 
 
  What do you mean by performance increase? If performance equals
 latency, v4
  will win for a long while still. Cgn does not add measurable latency.
 
  Cb
  --
  Leigh
 
 
 
 __
  This email has been scanned by the MessageLabs Email Security
 System.
  For more information please visit http://www.messagelabs.com/email
 
 __
 

RE: NAT444 or ?

[Dan Wing]

 -Original Message-
 From: Leigh Porter [mailto:leigh.por...@ukbroadband.com]
 Sent: Wednesday, September 07, 2011 1:38 PM
 To: David Israel; nanog@nanog.org
 Subject: RE: NAT444 or ?
 
 
 
  -Original Message-
  From: David Israel [mailto:da...@otd.com]
  Sent: 07 September 2011 21:23
  To: nanog@nanog.org
  Subject: Re: NAT444 or ?
 
  On 9/7/2011 3:24 PM, Seth Mos wrote:
   I think you have the numbers off, he started with 1000 users
 sharing
  the same IP, since you can only do 62k sessions or so and with a
  normal timeout on those sessions you ran into issues quickly.
  
 
  Remember that a TCP session is defined not just by the port, but by
 the
  combination of source address:source port:destination
  address:destination port.  So that's 62k sessions *per destination*
 per
  ip address.   In theory, this particular performance problem should
  only
  arise when the NAT gear insists on a unique port per session (which
 is
  common, but unnecessary) or when a particular destination is
  inordinately popular; the latter problem could be addressed by
  increasing the number of addresses that facebook.com and google.com
  resolve to.
 
 Good point, but aside from these scaling issues which I expect can be
 resolved to a point, the more serious issue, I think, is applications
 that just do not work with double NAT. Now, I have not conducted any
 serious research into this, but it seems that draft-donley-nat444-
 impacts does appear to have highlight issues that may have been down to
 implementation.

Draft-donley-nat444-impacts conflates bandwidth constraints with CGN
with in-home NAT.  Until those are separated and then analyzed carefully,
it is harmful to draw conclusions such as NAT444 bad; NAT44 good.

 Other simple tricks such as ensuring that your own internal services
 such as DNS are available without traversing NAT also help.

Yep.  But some users want to use other DNS servers for performance
(e.g., Google's or OpenDNS servers, especially considering they
could point the user at a 'better' (closer) CDN based on Client
IP), to avoid ISP DNS hijacking, or for content control (e.g.,
parental control of DNS hostnames).  That traffic will, necessarily,
traverse the CGN.  To avoid users burning through their UDP port 
allocation for those DNS queries it is useful for the CGN to 
have short timeouts for port 53.

 Certainly some more work can be done in this area, but I fear that the
 only way a real idea as to how much NAT444 really doe break things will
 be operational experience.

Yep.  (Same as everything else.)

-d


 
 
 --
 Leigh
 
 
 
 
 __
 This email has been scanned by the MessageLabs Email Security System.
 For more information please visit http://www.messagelabs.com/email
 __

RE: NAT444 or ?

[Dan Wing]

 -Original Message-
 From: Simon Perreault [mailto:simon.perrea...@viagenie.ca]
 Sent: Wednesday, September 07, 2011 2:29 PM
 To: nanog@nanog.org
 Subject: Re: NAT444 or ?
 
 David Israel wrote, on 09/07/2011 04:21 PM:
  In theory, this
  particular performance problem should only arise when the NAT gear
 insists on a
  unique port per session (which is common, but unnecessary)
 
 What you're describing is known as endpoint-independent mapping
 behaviour. It
 is good for not breaking applications, not so good for scalability. RFC
 4787 section 4.1 makes it a MUST.

There are two dimensions of that scalability, of course:

Endpoint-independent mapping means better scaling of the NAT itself, 
because it stores less state (slightly less memory for each active 
mapping and slightly less per-packet processing).  This savings
is exchanged for worse IPv4 utilization -- which I agree is not so
good for scalability.

-d

RE: NAT444 or ?

[Dan Wing]

 -Original Message-
 From: jean-francois.tremblay...@videotron.com [mailto:Jean-
 francois.tremblay...@videotron.com]
 Sent: Wednesday, September 07, 2011 10:06 AM
 To: d...@cluenet.de
 Cc: nanog@nanog.org
 Subject: Re: NAT444 or ?
 
 On Wed, Sep 07, 2011 at 12:16:28PM +0200, Randy Bush wrote:
   I'm going to have to deploy NAT444 with dual-stack real soon now.
  you may want to review the presentations from last week's apnic
 meeting
  in busan.  real mesurements.  sufficiently scary that people who were
  heavily pushing nat444 for the last two years suddenly started to say
  it was not me who pushed nat444, it was him!  as if none of us had
 a
  memory.
 
  Hm, I fail to find relevant slides discussing that. Could you please
  point us to those?
 
 I had the same question. I found Miyakawa-san's presentation has some
 dramatic examples of CGN NAT444 effects using Google Maps:
 http://meetings.apnic.net/__data/assets/file/0011/38297/Miyakawa-APNIC-
 KEYNOTE-IPv6-2011-8.pptx.pdf
 
 
 However these are with a very high address-sharing ratio (several
 thousands users per address). Using a sparser density (= 64 users per
 address) is likely to show much less dramatic user impacts.

Try it at home.  With aggressive usage of Microsoft's Terraserver,
mapquest, or google maps, I'm able to burn through 120 or so 
TCP connections.  Move the map around, zoom in/out, enable/disable 
traffic, switch between satellite and map and overlay, repeat those
steps 2-3 times.  Don't be slow and don't wait for everything 
to paint.

Or crash your browser and when it restarts watch how many connections
it makes to re-open all your tabs.

I understand iTunes opens lots of connections, but I haven't looked
at that.

To experiment with limited ports at home, load 3rd party firmware 
onto your NAT -- most of them allow controlling the number 
of mappings (and by default, have higher limits than stock
firmware).  Or consume a bunch of your mappings with a 
script (such as the brain-dead Perl script below) and then 
start your testing.  Some NATs and some servers will kill the 
TCP sessions after inactivity (which is why I describe the 
script as brain-dead).

-d



#!/usr/bin/perl -w
use IO::Socket;

$max = shift(@ARGV);
my $count = 0;
my $host = shift(@ARGV) || www.example.com;
my @remote;

print connecting to $host\n;

while ($count  $max) {
printf (connecting...(%d)\n, $count+1);
$remote[$count] = IO::Socket::INET-new(
Proto = tcp,
PeerAddr = $host,
PeerPort = 80)
or warn got an error;
$count++;
}

print press Return to exit\n;
my $junk = STDIN;

$count = 0;
while ($count  $max) {
close $remote[$count];
$count++;
}

exit;

RE: NAT444 or ?

[Dan Wing]

 -Original Message-
 From: Randy Bush [mailto:ra...@psg.com]
 Sent: Wednesday, September 07, 2011 3:16 AM
 To: Leigh Porter
 Cc: North American Network Operators' Group
 Subject: Re: NAT444 or ?
 
  I'm going to have to deploy NAT444 with dual-stack real soon now.
 
 you may want to review the presentations from last week's apnic meeting
 in busan.  real mesurements.  sufficiently scary that people who were
 heavily pushing nat444 for the last two years suddenly started to say
 it was not me who pushed nat444, it was him!  as if none of us had a
 memory.

Many of the problems are due to IPv4 address sharing, which will be
problems for A+P, CGN, HTTP proxies, and other address sharing 
technologies.  RFC6269 discusses most (or all) of those problems.
There are workarounds to those problems, but most are not 
pretty.  The solution is IPv6.

-d

RE: [arin-ppml] NAT444 rumors (was Re: Looking for an IPv6 naysayer...)

[Dan Wing]

 -Original Message-
 From: Chris Grundemann [mailto:cgrundem...@gmail.com]
 Sent: Monday, February 21, 2011 8:17 PM
 To: Dan Wing
 Cc: Owen DeLong; Benson Schliesser; NANOG list; ARIN-PPML List
 Subject: Re: [arin-ppml] NAT444 rumors (was Re: Looking for an IPv6
 naysayer...)
 
 On Mon, Feb 21, 2011 at 19:08, Dan Wing dw...@cisco.com wrote:
 
  Its title, filename, abstract, and introduction all say the problems
  are specific to NAT444.  Which is untrue.
 
 I just re-read the filename, abstract and introduction, and I disagree
 that any of those say that the problems are specific to NAT444. They
 all do state that these problems are present in NAT444, but not that
 it's the only technology/scenario/configuration where you might find
 them.
 
 More importantly, I am unsure the point of this argument.

My point is that:  NAT breaks things, but NAT444 is /not/ the only 
case where breakage occurs.

 Are you
 trying to say that the items listed as broken in the draft are not
 actually broken? Because in my experience they are. IMHO, the fact
 that they are also broken in other (similar) scenarios is not evidence
 that they are not broken in this one. On the contrary, this scenario
 seems to be evidence to the brokenness in the others (until we get a
 chance to test and document them all - are you volunteering? ;).

Vendor test results don't carry much value.

The authors of draft-donley-nat444-impacts did testing, and
I sincerely hope will publish results that split the impacts of
access bandwidth starvation from home NAT from CGN from NAT444.

-d


 Cheers,
 ~Chris
 
 
  -d
 
 
 
 
 
 
 
 --
 @ChrisGrundemann
 weblog.chrisgrundemann.com
 www.burningwiththebush.com
 www.theIPv6experts.net
 www.coisoc.org

RE: [arin-ppml] NAT444 rumors (was Re: Looking for an IPv6 naysayer...)

[Dan Wing]

 -Original Message-
 From: arin-ppml-boun...@arin.net [mailto:arin-ppml-boun...@arin.net] On
 Behalf Of Chris Grundemann
 Sent: Thursday, February 17, 2011 5:55 PM
 To: Benson Schliesser
 Cc: NANOG list; ARIN-PPML List
 Subject: Re: [arin-ppml] NAT444 rumors (was Re: Looking for an IPv6
 naysayer...)
 
 On Thu, Feb 10, 2011 at 14:17, Benson Schliesser
 bens...@queuefull.net wrote:
 
  If you have more experience (not including rumors) that suggests
 otherwise, I'd very much like to hear about it.  I'm open to the
 possibility that NAT444 breaks stuff - that feels right in my gut - but
 I haven't found any valid evidence of this.
 
 In case you have not already found this:
 http://tools.ietf.org/html/draft-donley-nat444-impacts-01

That document conflates problems of NAT444 with problems of NAT44 
with problems of bandwidth starvation with problems of CGN.

For details, see my comments at
http://www.ietf.org/mail-archive/web/behave/current/msg09027.html
and see Reinaldo Penno's comments at
http://www.ietf.org/mail-archive/web/behave/current/msg09030.html

-d

 Cheers,
 ~Chris
 
 
  Regardless, I think we can agree that IPv6 is the way to avoid NAT-
 related growing pains.  We've known this for a long time.
 
  Cheers,
  -Benson
 
  ___
  PPML
  You are receiving this message because you are subscribed to
  the ARIN Public Policy Mailing List (arin-p...@arin.net).
  Unsubscribe or manage your mailing list subscription at:
  http://lists.arin.net/mailman/listinfo/arin-ppml
  Please contact i...@arin.net if you experience any issues.
 
 
 
 
 
 
 
 --
 @ChrisGrundemann
 weblog.chrisgrundemann.com
 www.burningwiththebush.com
 www.theIPv6experts.net
 www.coisoc.org
 ___
 PPML
 You are receiving this message because you are subscribed to
 the ARIN Public Policy Mailing List (arin-p...@arin.net).
 Unsubscribe or manage your mailing list subscription at:
 http://lists.arin.net/mailman/listinfo/arin-ppml
 Please contact i...@arin.net if you experience any issues.

RE: [arin-ppml] NAT444 rumors (was Re: Looking for an IPv6 naysayer...)

[Dan Wing]

 -Original Message-
 From: Owen DeLong [mailto:o...@delong.com]
 Sent: Monday, February 21, 2011 12:59 PM
 To: Dan Wing
 Cc: 'Chris Grundemann'; 'Benson Schliesser'; 'NANOG list'; 'ARIN-PPML
 List'
 Subject: Re: [arin-ppml] NAT444 rumors (was Re: Looking for an IPv6
 naysayer...)
 
 
 On Feb 21, 2011, at 12:37 PM, Dan Wing wrote:
 
  -Original Message-
  From: arin-ppml-boun...@arin.net [mailto:arin-ppml-boun...@arin.net]
 On
  Behalf Of Chris Grundemann
  Sent: Thursday, February 17, 2011 5:55 PM
  To: Benson Schliesser
  Cc: NANOG list; ARIN-PPML List
  Subject: Re: [arin-ppml] NAT444 rumors (was Re: Looking for an IPv6
  naysayer...)
 
  On Thu, Feb 10, 2011 at 14:17, Benson Schliesser
  bens...@queuefull.net wrote:
 
  If you have more experience (not including rumors) that suggests
  otherwise, I'd very much like to hear about it.  I'm open to the
  possibility that NAT444 breaks stuff - that feels right in my gut -
 but
  I haven't found any valid evidence of this.
 
  In case you have not already found this:
  http://tools.ietf.org/html/draft-donley-nat444-impacts-01
 
  That document conflates problems of NAT444 with problems of NAT44
  with problems of bandwidth starvation with problems of CGN.
 
  For details, see my comments at
  http://www.ietf.org/mail-archive/web/behave/current/msg09027.html
  and see Reinaldo Penno's comments at
  http://www.ietf.org/mail-archive/web/behave/current/msg09030.html
 
  -d
 
 The document describes problems that will exist in NAT444 environments.
 It does not state that these problems would be specific to NAT444, but,
 NAT444 will cause or exacerbate each of the problems described.

To the contrary.

Its title, filename, abstract, and introduction all say the problems
are specific to NAT444.  Which is untrue.

 Yes, the problems may have other underlying root causes, but, they
 will all be present in a NAT444 environment, even if they were not
 present in the same environment prior to deployment of NAT444.
 
 
 Let me put it this way...
 
 IPv4 has a TITANIC lack of numeric addresses and has been
 stretched beyond its limits for some time now.
 
 IPv6 is a life boat.
 
 NAT is a seat cushion used for floatation.
 
 NAT444 (and other NAT-based extensions) are deck chairs.
 
 Attempting to improve them beyond their current states is
 an effort to rearrange the deck chairs.

-d

RE: [arin-ppml] NAT444 rumors (was Re: Looking for an IPv6 naysayer...)

[Dan Wing]

  http://tools.ietf.org/html/draft-donley-nat444-impacts-01
  That document conflates problems of NAT444 with problems of NAT44
  with problems of bandwidth starvation with problems of CGN.
 
 it may require a delicate palate to differentiate the different flavors
 of bleep

Running out of bandwidth for Netflix is pretty distinct from
the flavor of fried gNAT.

-d

RE: Problems with removing NAT from a network

[Dan Wing]

 On 1/6/2011 9:28 PM, Dan Wing wrote:
  -Original Message-
  From: Matthew Kaufman [mailto:matt...@matthew.at]
  Not really. Imagine the case where you're on IPv6 and you can only
  reach
  IPv4 via a NAT64, and there's no progress made on the detection
  problem.
  And your family member is on a Skype-enabled TV plugged into an
  IPv4-only ISP.
 
  Now you can't get a direct media path between you, even though their
  ISP
  is giving them IPv4 and your ISP is *claiming* you can still reach
 the
  IPv4 Internet.
 
  Skype can still make this work by relaying,
  Skype could make it work with direct UDP packets in about 92% of
  cases, per Google's published direct-to-direct statistic at
  http://code.google.com/apis/talk/libjingle/important_concepts.html
 
 If one end is behind a NAT64 and there is no mechanism for discovering
 the NAT64's IPv6 interface prefix and mapping algorithm (and at present
 there is not), there is no way to send IPv6 IP packets from the
 IPv6-only host to IPv4 literal addresses (that is to say, addresses
 learned via a mechanism other than DNS responses synthesized by the
 DNS64 part of the NAT64 solution) on the IPv4 Internet through said
 NAT64.
 
 That's the case we're discussing here.

There are a bunch of ideas for how to accomplish that.  Several of
the ideas don't require any support by the network infrastructure,
draft-korhonen-behave-nat64-learn-analysis.

 It breaks Skype, Adobe's RTMFP, BitTorrent, ICE-based NAT traversal,
 etc. Even the protocol described in the referenced document, Jingle (as
 it essentially uses ICE) fails. The candidate IPv4 addresses for the
 end
 that's on the IPv4 Internet (local and STUN-derived) that are delivered
 over Jingle's XMPP path cannot be used by the host that is on IPv6 +
 NAT64 to reach the IPv4 Internet because it has no IPv4 sockets
 available to it and even if it knew that NAT64 existed (which would
 take
 a modification to the Jingle-based apps) and opened an IPv6 socket it
 wouldn't know what IPv6 address to use to reach the IPv4 host because
 there's no discovery mechanism. If you want we can take this back to
 the BEHAVE list now.

Sure.

-d


 
 Matthew Kaufman

RE: Problems with removing NAT from a network

[Dan Wing]

 -Original Message-
 From: Matthew Kaufman [mailto:matt...@matthew.at]
 Sent: Thursday, January 06, 2011 6:55 PM
 To: Owen DeLong
 Cc: Nanog Operators' Group
 Subject: Re: Problems with removing NAT from a network
 
 On 1/6/2011 5:48 PM, Owen DeLong wrote:
  Doesn't all of this become moot if Skype just develops a dual-stack
 capable client
  and servers?
 
 Not really. Imagine the case where you're on IPv6 and you can only
 reach
 IPv4 via a NAT64, and there's no progress made on the detection
 problem.
 And your family member is on a Skype-enabled TV plugged into an
 IPv4-only ISP.
 
 Now you can't get a direct media path between you, even though their
 ISP
 is giving them IPv4 and your ISP is *claiming* you can still reach the
 IPv4 Internet.
 
 Skype can still make this work by relaying,

Skype could make it work with direct UDP packets in about 92% of
cases, per Google's published direct-to-direct statistic at
http://code.google.com/apis/talk/libjingle/important_concepts.html

-d


 but in order to protect the
 relay machine's bandwidth it will rate-limit the traffic, and so your
 A/V experience will suffer. And that's assuming there's enough
 dual-stacked relays... if there aren't, it won't be possible to find a
 relay that they can reach over IPv4 and you can reach over IPv6 that
 has
 available bandwidth.
 
 Matthew Kaufman