Re: Note - ARIN Consultation on proposed Registration Services Agreement Now Open

2015-09-03 Thread Sandra Murphy
There’s been no comment on this announcement on this list, but there were a 
small handful of NANOGers who energetically discussed this on the arin-consult 
list starting mid-August.

On the last day of the comment period (so you might have missed it), a detailed 
comment was submitted, which focussed on the impact to legacy resource holders 
and those who acquire addresses in the transfer market.

I’m not sure how many people on this list would have been following a 
discussion of the ARIN RSA, but there might be some who would find legacy and 
transfer focussed comments to be interesting.

So in case you missed it:

General discussion:  
http://lists.arin.net/pipermail/arin-consult/2015-August/date.html
Focussed comment:  
http://lists.arin.net/pipermail/arin-consult/2015-August/000662.html

—Sandy


On Jul 29, 2015, at 4:11 PM, John Curran  wrote:

> NANOGers -
> 
> To the extent that you are interested in ARIN's Registration Services 
> Agreement, we have
> just initiated a consultation on a proposed new version of the RSA (and 
> LRSA) as noted in
> the attached announcement.   If you wish to provide any feedback, please 
> do so on the
> mailto:arin-cons...@arin.net>> mailing list 
> between now and 30 August 2015.
> 
> Thanks!
> /John
> 
> John Curran
> President and CEO
> ARIN
> 
> Begin forwarded message:
> 
> From: ARIN mailto:i...@arin.net>>
> Subject: [ARIN-consult] Consultation on Registration Services Agreement Now 
> Open
> Date: July 29, 2015 at 4:04:03 PM EDT
> To: mailto:arin-cons...@arin.net>>
> 
> ARIN is seeking community input on a new version of the Registration
> Services Agreement that combines the existing Registration Services
> Agreement (RSA) and Legacy Registration Services Agreement (LRSA) into a
> single agreement (“RSA Version 12.0/LRSA Version 4.0”).
> 
> Following community consultation and finalization, ARIN will provide
> services to new parties per this latest version of the Registration
> Services Agreement. Existing holders of Internet number resources will
> have the opportunity to upgrade to this new version of the Registration
> Services Agreement or remain with their current Registration Services
> Agreement.
> 
> There are notable and substantive changes with this new version
> incorporating input received from the community on the Registration
> Services Agreements.
> 
> These changes include:
> 
> *  Clarifying that the agreement is only applicable to “Included Number
> Resources” (i.e. the Internet number resources pursuant to the
> agreement, not any other number resources that parties may hold under
> other agreements or no agreement)
> *  Similarly clarifying that the provision wherein a Customer is
> agreeing that “Included Number Resources” are not property is only
> applicable to “Included Number Resources” in the agreement and remains
> silent with regard to any other number resources held by the party
> *  Providing uniform service terms and conditions (other than fees) for all
> customers receiving services from ARIN
> *  Elaborating on the definition of ARIN’s services that are covered by
> the agreement, including resource certification
> *  Providing a more balanced agreement with respect to term language
> previously seen as favorable to ARIN
> 
> The new RSA Version 12.0/LRSA Version 4.0 is available at the following URL:
> 
> https://www.arin.net/resources/agreements/rsa_draft_ver12.pdf
> 
> We strongly encourage you to review the current RSA and LRSA against the
> proposed version of this new agreement and to submit your constructive
> comments to arin-cons...@arin.net.
> 
> Discussion on arin-cons...@arin.net will close on 30 August 2015. ARIN
> seeks clear direction through community input, so your feedback is
> important. If you have any questions, please contact us at i...@arin.net.
> 
> 
> Thank you,
> /John
> 
> John Curran
> President and CEO
> American Registry for Internet Numbers
> 
> ___
> ARIN-Consult
> You are receiving this message because you are subscribed to the ARIN Consult 
> Mailing
> List (arin-cons...@arin.net).
> Unsubscribe or manage your mailing list subscription at:
> http://lists.arin.net/mailman/listinfo/arin-consult Please contact the ARIN 
> Member Services
> Help Desk at i...@arin.net if you experience any issues.
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: Whois.net down?

2015-09-03 Thread Brian Reichert
On Thu, Sep 03, 2015 at 07:57:55PM +, Marcin Cieslak wrote:
> whois.net is some site operated by NTT/Verio, 
> this domain tech contact seems to be:
> 
> Tech Name: Verio Hostmaster
> Tech Organization:
> Tech Street: 8005 S. Chester Street Suite 200
> Tech City: Englewood
> Tech State/Province: CO
> Tech Postal Code: 80112
> Tech Country: US
> Tech Phone: +1.2142908620
> Tech Phone Ext:
> Tech Fax: +1.2147451877
> Tech Fax Ext:
> Tech Email: hostmas...@verio.net
> 
> although you might have better chance via Twitter
> https://twitter.com/WhoisNet :(

Thanks for the suggestion. :)

> ~Marcin

-- 
Brian Reichert  
BSD admin/developer at large


Re: Whois.net down?

2015-09-03 Thread Marcin Cieslak
On Thu, 3 Sep 2015, Brian Reichert wrote:

> On Thu, Sep 03, 2015 at 09:59:02PM +0700, David S. wrote:
> > Hi Brian,
> > 
> > I'm able to access https://whois.net, have you check the nameserver of
> > numachi.com?
> > Is the other domain use same authoritative DNS?
> 
> I can access the web page.  When I use the page for certain domains,
> sometimes, I successfully get results.
> 
> Other domains, such as NUMACHI.COM, I get the error I reported.
> 
> CLI-based versions of whois work just fine for all domains I'm
> concerned about; it's that web-baed version that is selectively
> failing.

whois.net is some site operated by NTT/Verio, 
this domain tech contact seems to be:

Tech Name: Verio Hostmaster
Tech Organization:
Tech Street: 8005 S. Chester Street Suite 200
Tech City: Englewood
Tech State/Province: CO
Tech Postal Code: 80112
Tech Country: US
Tech Phone: +1.2142908620
Tech Phone Ext:
Tech Fax: +1.2147451877
Tech Fax Ext:
Tech Email: hostmas...@verio.net

although you might have better chance via Twitter
https://twitter.com/WhoisNet :(

~Marcin


Re: Whois.net down?

2015-09-03 Thread Brian Reichert
On Thu, Sep 03, 2015 at 09:59:02PM +0700, David S. wrote:
> Hi Brian,
> 
> I'm able to access https://whois.net, have you check the nameserver of
> numachi.com?
> Is the other domain use same authoritative DNS?

I can access the web page.  When I use the page for certain domains,
sometimes, I successfully get results.

Other domains, such as NUMACHI.COM, I get the error I reported.

CLI-based versions of whois work just fine for all domains I'm
concerned about; it's that web-baed version that is selectively
failing.

> Best regards,
> David S.

-- 
Brian Reichert  
BSD admin/developer at large


Re: buffer bloat and packet pacing

2015-09-03 Thread Brett Frankenberger
On Thu, Sep 03, 2015 at 05:48:00PM +0300, Saku Ytti wrote:
> Hey Brett,
>
> > Here's a paper that shows you don't need buffers equal to
> > bandwidth*delay to get near capacity:
> > http://www.cs.bu.edu/~matta/Papers/hstcp-globecom04.pdf
> > (I'm not endorsing it.  Just pointing out it out as a datapoint.)
>
> Quick glance makes me believe the S and D nodes are equal bandwidth,
> but only R1-R2 bandwidth is explicitly stated.S1, D1, Sn, Dn are only
> ever mentioned in the topology. If Sender is same or lower rate than
> Destination, then we really shouldn't need almost any buffering.

Unless Sender is higher than R1-R2.

> Issue should only come when Sender is significantly higher rate than
> Destination and network is not limiting them.

I didn't read it in detail either, but at first glance, it appears to
me that the model is infinite bandwidth and zero latency between S and
R1, and D and R2, with queueing happening in R1.

That's not going to give materially different results than, having S-R1
be 4 times R1-R2, and R2-D being the same as R1-R2.  So it fits well
with the original discussion here of 40G into 10G.

 -- Brett


Whois.net down?

2015-09-03 Thread Brian Reichert
I'm trying to use https://www.whois.net/ to resolve info about
several domains, including my own (NUMACHI.COM).

For several domains (but not all, I get 'Error extracting data. No
data available'.  There's a 'please read' tag, but no text associated
with it.

Anyone know if they're having issues there?

-- 
Brian Reichert  
BSD admin/developer at large


Re: buffer bloat and packet pacing

2015-09-03 Thread Saku Ytti
Hey Brett,

> Here's a paper that shows you don't need buffers equal to
> bandwidth*delay to get near capacity:
> http://www.cs.bu.edu/~matta/Papers/hstcp-globecom04.pdf
> (I'm not endorsing it.  Just pointing out it out as a datapoint.)

Quick glance makes me believe the S and D nodes are equal bandwidth,
but only R1-R2 bandwidth is explicitly stated.S1, D1, Sn, Dn are only
ever mentioned in the topology. If Sender is same or lower rate than
Destination, then we really shouldn't need almost any buffering.
Issue should only come when Sender is significantly higher rate than
Destination and network is not limiting them.

-- 
  ++ytti


Re: buffer bloat and packet pacing

2015-09-03 Thread Brett Frankenberger
On Thu, Sep 03, 2015 at 01:04:34PM +0100, Nick Hilliard wrote:
> On 03/09/2015 11:56, Saku Ytti wrote:
> > 40GE server will flood the window as fast as it can, instead of
> > limiting itself to 10Gbps, optimally it'll send at linerate.
> 
> optimally, but tcp slow start will generally stop this from happening on
> well behaved sending-side stacks so you send up ramping up quickly to path
> rate rather than egress line rate from the sender side.  Also, regardless
> of an individual flow's buffering requirements, the intermediate path will
> be catering with large numbers of flows, so while it's interesting to talk
> about 375mb of intermediate path buffers, this is shared buffer space and
> any attempt on the part of an individual sender to (ab)use the entire path
> buffer will end up causing RED/WRED for everyone else.
> 
> Otherwise, this would be a fascinating talk if people had real world data.

The original analysis is flawed because it assumes latency is constant.
Any analysis has to include the fact that buffering changes latency.

If you start with a 300ms path (by propogation delay, switching latency,
hetc.), and 375MB of buffers on a 10G port, then, when the buffers
fill, you end up with a 600ms path[1].  And a 375MB window is no longer
sufficient to keep the pipe full.

Instead, you need a 750MB buffer.

But now the latency is 900ms.

And so on.  This doesn't converge.  Every byte of filled buffer is
another byte you need in the window if you're going to fill the pipe.

Not accounting for this is part of the reason the original analysis is
flawed.  The end result is that you always run out of window or run out
of buffer (causing packet loss).

Here's a paper that shows you don't need buffers equal to
bandwidth*delay to get near capacity:
http://www.cs.bu.edu/~matta/Papers/hstcp-globecom04.pdf
(I'm not endorsing it.  Just pointing out it out as a datapoint.)

 -- Brett

[1] 0.300 + 375E6 * 8 / 10E9 = 600ms


Re: udp 500 packets when users are web browsing

2015-09-03 Thread Oliver O'Boyle
That would do it. Almost certainly enforced by GPO in that case so at least
it's easy to change if you need to.

On Thu, Sep 3, 2015 at 10:25 AM, Robert Webb  wrote:

> Yes, we are looking at this now.
>
> Thanks for everyone's help. I think we are heading in the right direction
> tracking this down. This just showed up in our monitoring and makes sense
> as we just brought up a new locked down domain.
>
> Robert
>
>
>
> On Thu, 3 Sep 2015 10:19:53 -0400
>  "Oliver O'Boyle"  wrote:
>
>> You can configure Windows to encrypt traffic based on protocol
>> definitions.
>> E.g., Use IPSEC to encrypt all traffic on port 80 between hosts X and
>> hosts
>> Y.
>>
>> It's possible that such a policy is in place locally on the workstations
>> and/or servers and it's also possible that it's being enforced using GPOs.
>>
>> On Thu, Sep 3, 2015 at 9:53 AM, Robert Webb  wrote:
>>
>> There is no VPN in the picture here. These are straight workstations on
>>> the network that the packets are coming from.
>>>
>>> According to a pcaket capture in wireshark, these are isakmp packets
>>> reaching out to host names of web sites that are being browsed. So
>>> destinations are sites like twitter, facebook, amazon, cnn, etc..
>>>
>>> We have further discovered that they seem to be initiated from the
>>> Windows
>>> 7 svchost, but we have not been able to find documentation as to how or
>>> why
>>> this is ocurring.
>>>
>>> Robert
>>>
>>>
>>> On Thu, 3 Sep 2015 13:42:21 +
>>>  "Bjoern A. Zeeb"  wrote:
>>>
>>>
 On 03 Sep 2015, at 13:35 , Robert Webb  wrote:

>
> We are seeing udp 500 packets being dropped at our firewall from user's
> browsing sessions. These are users on a 2008 R2 AD setup with Windows
> 7.
>
> Source and destination ports are udp 500 and the the pattern of drops
> directly correlate to the web browsing activity. We have confirmed this
> with tcpdump of port 500 and a single host and watching the pattern of
> traffic as they browse. This also occurs no matter what browser is
> used.
>
> Can anyone shine some light on what may be using udp 500 when web
> browsing?
>
>
 The VPN using IPsec UDP-Encap connection that supposedly gets through
 NAT?   Have you checked the content with tcpdump?   Do you have
 fragments
 by any chance?


 --
>> :o@>
>>
>
>
>


-- 
:o@>


Re: udp 500 packets when users are web browsing

2015-09-03 Thread Robert Webb

Yes, we are looking at this now.

Thanks for everyone's help. I think we are heading in the right direction 
tracking this down. This just showed up in our monitoring and makes sense as 
we just brought up a new locked down domain.


Robert


On Thu, 3 Sep 2015 10:19:53 -0400
 "Oliver O'Boyle"  wrote:
You can configure Windows to encrypt traffic based on protocol 
definitions.
E.g., Use IPSEC to encrypt all traffic on port 80 between hosts X 
and hosts

Y.

It's possible that such a policy is in place locally on the 
workstations
and/or servers and it's also possible that it's being enforced using 
GPOs.


On Thu, Sep 3, 2015 at 9:53 AM, Robert Webb  
wrote:


There is no VPN in the picture here. These are straight workstations 
on

the network that the packets are coming from.

According to a pcaket capture in wireshark, these are isakmp packets
reaching out to host names of web sites that are being browsed. So
destinations are sites like twitter, facebook, amazon, cnn, etc..

We have further discovered that they seem to be initiated from the 
Windows
7 svchost, but we have not been able to find documentation as to how 
or why

this is ocurring.

Robert


On Thu, 3 Sep 2015 13:42:21 +
 "Bjoern A. Zeeb"  wrote:



On 03 Sep 2015, at 13:35 , Robert Webb  wrote:


We are seeing udp 500 packets being dropped at our firewall from 
user's
browsing sessions. These are users on a 2008 R2 AD setup with 
Windows 7.


Source and destination ports are udp 500 and the the pattern of 
drops
directly correlate to the web browsing activity. We have confirmed 
this
with tcpdump of port 500 and a single host and watching the pattern 
of
traffic as they browse. This also occurs no matter what browser is 
used.


Can anyone shine some light on what may be using udp 500 when web
browsing?



The VPN using IPsec UDP-Encap connection that supposedly gets 
through
NAT?   Have you checked the content with tcpdump?   Do you have 
fragments

by any chance?



--
:o@>





Re: udp 500 packets when users are web browsing

2015-09-03 Thread Oliver O'Boyle
Precisely.

On Thu, Sep 3, 2015 at 10:14 AM, Chuck Anderson  wrote:

> Sounds like Opportunistic Encryption.
>
> https://en.wikipedia.org/wiki/Opportunistic_encryption#Windows_OS
>
> On Thu, Sep 03, 2015 at 09:53:46AM -0400, Robert Webb wrote:
> > There is no VPN in the picture here. These are straight workstations
> > on the network that the packets are coming from.
> >
> > According to a pcaket capture in wireshark, these are isakmp packets
> > reaching out to host names of web sites that are being browsed. So
> > destinations are sites like twitter, facebook, amazon, cnn, etc..
> >
> > We have further discovered that they seem to be initiated from the
> > Windows 7 svchost, but we have not been able to find documentation
> > as to how or why this is ocurring.
> >
> > Robert
> >
> >
> > On Thu, 3 Sep 2015 13:42:21 +
> >  "Bjoern A. Zeeb"  wrote:
> > >
> > >>On 03 Sep 2015, at 13:35 , Robert Webb  wrote:
> > >>
> > >>We are seeing udp 500 packets being dropped at our firewall from
> > >>user's browsing sessions. These are users on a 2008 R2 AD setup
> > >>with Windows 7.
> > >>
> > >>Source and destination ports are udp 500 and the the pattern of
> > >>drops directly correlate to the web browsing activity. We have
> > >>confirmed this with tcpdump of port 500 and a single host and
> > >>watching the pattern of traffic as they browse. This also occurs
> > >>no matter what browser is used.
> > >>
> > >>Can anyone shine some light on what may be using udp 500 when
> > >>web browsing?
> > >
> > >The VPN using IPsec UDP-Encap connection that supposedly gets
> > >through NAT?   Have you checked the content with tcpdump?   Do you
> > >have fragments by any chance?
>



-- 
:o@>


Re: udp 500 packets when users are web browsing

2015-09-03 Thread Oliver O'Boyle
You can configure Windows to encrypt traffic based on protocol definitions.
E.g., Use IPSEC to encrypt all traffic on port 80 between hosts X and hosts
Y.

It's possible that such a policy is in place locally on the workstations
and/or servers and it's also possible that it's being enforced using GPOs.

On Thu, Sep 3, 2015 at 9:53 AM, Robert Webb  wrote:

> There is no VPN in the picture here. These are straight workstations on
> the network that the packets are coming from.
>
> According to a pcaket capture in wireshark, these are isakmp packets
> reaching out to host names of web sites that are being browsed. So
> destinations are sites like twitter, facebook, amazon, cnn, etc..
>
> We have further discovered that they seem to be initiated from the Windows
> 7 svchost, but we have not been able to find documentation as to how or why
> this is ocurring.
>
> Robert
>
>
>
> On Thu, 3 Sep 2015 13:42:21 +
>  "Bjoern A. Zeeb"  wrote:
>
>>
>> On 03 Sep 2015, at 13:35 , Robert Webb  wrote:
>>>
>>> We are seeing udp 500 packets being dropped at our firewall from user's
>>> browsing sessions. These are users on a 2008 R2 AD setup with Windows 7.
>>>
>>> Source and destination ports are udp 500 and the the pattern of drops
>>> directly correlate to the web browsing activity. We have confirmed this
>>> with tcpdump of port 500 and a single host and watching the pattern of
>>> traffic as they browse. This also occurs no matter what browser is used.
>>>
>>> Can anyone shine some light on what may be using udp 500 when web
>>> browsing?
>>>
>>
>> The VPN using IPsec UDP-Encap connection that supposedly gets through
>> NAT?   Have you checked the content with tcpdump?   Do you have fragments
>> by any chance?
>>
>>
>>
>
>


-- 
:o@>


Re: udp 500 packets when users are web browsing

2015-09-03 Thread Chuck Anderson
Sounds like Opportunistic Encryption.

https://en.wikipedia.org/wiki/Opportunistic_encryption#Windows_OS

On Thu, Sep 03, 2015 at 09:53:46AM -0400, Robert Webb wrote:
> There is no VPN in the picture here. These are straight workstations
> on the network that the packets are coming from.
> 
> According to a pcaket capture in wireshark, these are isakmp packets
> reaching out to host names of web sites that are being browsed. So
> destinations are sites like twitter, facebook, amazon, cnn, etc..
> 
> We have further discovered that they seem to be initiated from the
> Windows 7 svchost, but we have not been able to find documentation
> as to how or why this is ocurring.
> 
> Robert
> 
> 
> On Thu, 3 Sep 2015 13:42:21 +
>  "Bjoern A. Zeeb"  wrote:
> >
> >>On 03 Sep 2015, at 13:35 , Robert Webb  wrote:
> >>
> >>We are seeing udp 500 packets being dropped at our firewall from
> >>user's browsing sessions. These are users on a 2008 R2 AD setup
> >>with Windows 7.
> >>
> >>Source and destination ports are udp 500 and the the pattern of
> >>drops directly correlate to the web browsing activity. We have
> >>confirmed this with tcpdump of port 500 and a single host and
> >>watching the pattern of traffic as they browse. This also occurs
> >>no matter what browser is used.
> >>
> >>Can anyone shine some light on what may be using udp 500 when
> >>web browsing?
> >
> >The VPN using IPsec UDP-Encap connection that supposedly gets
> >through NAT?   Have you checked the content with tcpdump?   Do you
> >have fragments by any chance?


Re: udp 500 packets when users are web browsing

2015-09-03 Thread Robert Webb
There is no VPN in the picture here. These are straight workstations on the 
network that the packets are coming from.


According to a pcaket capture in wireshark, these are isakmp packets 
reaching out to host names of web sites that are being browsed. So 
destinations are sites like twitter, facebook, amazon, cnn, etc..


We have further discovered that they seem to be initiated from the Windows 7 
svchost, but we have not been able to find documentation as to how or why 
this is ocurring.


Robert


On Thu, 3 Sep 2015 13:42:21 +
 "Bjoern A. Zeeb"  wrote:



On 03 Sep 2015, at 13:35 , Robert Webb  wrote:

We are seeing udp 500 packets being dropped at our firewall from 
user's browsing sessions. These are users on a 2008 R2 AD setup with 
Windows 7.


Source and destination ports are udp 500 and the the pattern of 
drops directly correlate to the web browsing activity. We have 
confirmed this with tcpdump of port 500 and a single host and 
watching the pattern of traffic as they browse. This also occurs no 
matter what browser is used.


Can anyone shine some light on what may be using udp 500 when web 
browsing?


The VPN using IPsec UDP-Encap connection that supposedly gets 
through NAT?   Have you checked the content with tcpdump?   Do you 
have fragments by any chance?








Re: udp 500 packets when users are web browsing

2015-09-03 Thread Bjoern A. Zeeb

> On 03 Sep 2015, at 13:35 , Robert Webb  wrote:
> 
> We are seeing udp 500 packets being dropped at our firewall from user's 
> browsing sessions. These are users on a 2008 R2 AD setup with Windows 7.
> 
> Source and destination ports are udp 500 and the the pattern of drops 
> directly correlate to the web browsing activity. We have confirmed this with 
> tcpdump of port 500 and a single host and watching the pattern of traffic as 
> they browse. This also occurs no matter what browser is used.
> 
> Can anyone shine some light on what may be using udp 500 when web browsing?

The VPN using IPsec UDP-Encap connection that supposedly gets through NAT?   
Have you checked the content with tcpdump?   Do you have fragments by any 
chance?




udp 500 packets when users are web browsing

2015-09-03 Thread Robert Webb
We are seeing udp 500 packets being dropped at our firewall from user's 
browsing sessions. These are users on a 2008 R2 AD setup with Windows 7.


Source and destination ports are udp 500 and the the pattern of drops 
directly correlate to the web browsing activity. We have confirmed this with 
tcpdump of port 500 and a single host and watching the pattern of traffic as 
they browse. This also occurs no matter what browser is used.


Can anyone shine some light on what may be using udp 500 when web browsing?

Robert



Re: buffer bloat and packet pacing

2015-09-03 Thread Saku Ytti
On 3 September 2015 at 15:04, Nick Hilliard  wrote:

> optimally, but tcp slow start will generally stop this from happening on
> well behaved sending-side stacks so you send up ramping up quickly to path
> rate rather than egress line rate from the sender side.

This assumes network is congested and unable to reach its potential
rate. If it can reach its potential rate, eventually the window will
scale to 375MB and the pathological flooding will occur.
Mostly network is congested, and the pathological case cannot happen,
as the egress cannot ingest the floods, not allowing window to grow to
needed size, which also means the potential rate will not be reached,
and rate will be something less than 10Gbps. Essentially we threw the
baby out with the bath water, kind of like protecting from DoS by
killing the victim.

-- 
  ++ytti


Re: buffer bloat and packet pacing

2015-09-03 Thread Nick Hilliard
On 03/09/2015 11:56, Saku Ytti wrote:
> 40GE server will flood the window as fast as it can, instead of
> limiting itself to 10Gbps, optimally it'll send at linerate.

optimally, but tcp slow start will generally stop this from happening on
well behaved sending-side stacks so you send up ramping up quickly to path
rate rather than egress line rate from the sender side.  Also, regardless
of an individual flow's buffering requirements, the intermediate path will
be catering with large numbers of flows, so while it's interesting to talk
about 375mb of intermediate path buffers, this is shared buffer space and
any attempt on the part of an individual sender to (ab)use the entire path
buffer will end up causing RED/WRED for everyone else.

Otherwise, this would be a fascinating talk if people had real world data.

Nick


Re: buffer bloat and packet pacing

2015-09-03 Thread Saku Ytti
> only serialise them 10GE out, causing majority of that 375MB ending up
> in the sender side switch/router buffers.

s/sender/receiver/

-- 
  ++ytti


buffer bloat and packet pacing

2015-09-03 Thread Saku Ytti
Hey,

In past few years there's been lot of talk about reducing buffer
depths, and many seem to think vendors are throwing memory on the
chips for the fun of it.

If we look at some particularly pathological case. Let's assume sender
is CDN network with 40GE connected server and receiver is 10GE
connected. There is 300ms latency between them.

10Gbps * 300ms = 375MB, is the window size the client wants to be able
to fill its pipe to the 40GE sender.
However TCP does not normally pace packets inside the window, so 40GE
server will flood the window as fast as it can, instead of limiting
itself to 10Gbps, optimally it'll send at linerate. While receiver can
only serialise them 10GE out, causing majority of that 375MB ending up
in the sender side switch/router buffers.
If we can't buffer that, then the receiver cannot receive at 10Gbps,
as window size will shrink. Is this a problem? What rate should you be
able to expect to get and at what latency? Usually contracts to
customers won't have any limitations on bandwidth achievable on given
latency and writing such down might make you appear inferior to your
competitor.

Perhaps this is unrealistic case, however if you run the numbers in
much less pathological cases, you'll still end up having much larger
buffer needs than large number of switch chips out there have.

Some new ones, like JNPR QFX10k and Broadcom Jericho come with much
larger buffers than predecessors, and will be able to deal with what I
hope are most practical cases.

Linux actually these days does have bandwidth estimator for TCP
sessions, but it's not used by default for anything, it's just for
consumption for other layers so they can do something about it. And I
believe in 'tc' you can use these to cause packet pacing inside a
window.
QUIC and MinimaLT, AFAIK, do bandwidth estimation and packet pacing by default.

In perfect world, we'd be done now. Receiver side switch can do with
very small buffers, few packets should suffice. However, if network
itself is congested, the bandwidth estimation keeps sinking, and these
well-behaved streams are losing to the aggressive TCP streams, and
you'll end up having 0bps estimations.
So perhaps the bandwidth estimator should be application aware, and
never report lower estimate than what is practical for given
application, so that it could compete fairly with aggressive streams,
up-to required rate.

Information I'd love to have, is how large window sizes do TCP
sessions peak at, in real network? Some CDN network must be collecting
these stats. I'd love to see rough statistics. <1% go over 100MB? 2%
between 50MB-100MB? ... few large brackets of distribution of window
sizes by some CDN offering content download (GGC, OpenConnect are not
interesting, as they won't send large files).
Also, are some CDN's already implementing packet pacing inside window?
If so, how? Do they have lower limit to it?

Some related URLs:
https://lwn.net/Articles/645115/
https://lwn.net/Articles/564978/
http://www.ietf.org/proceedings/88/slides/slides-88-iccrg-6.pdf
http://www.ietf.org/proceedings/84/slides/slides-84-iccrg-2.pdf
-- 
  ++ytti