RE: Internet Exchanges supporting jumbo frames?

-Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Tim McKee >The factor of 6 was just in reduction of overhead. Granted in the greater scheme of things the overall 4% is relatively insignificant, but there have been many times when doing >multiple 10-100+GB

RE: Internet Exchanges supporting jumbo frames?

Message- From: Jakob Heitz (jheitz) [mailto:jhe...@cisco.com] Sent: Saturday, March 19, 2016 00:34 To: Tim McKee Cc: Dale W. Carder; nanog@nanog.org Subject: Re: Internet Exchanges supporting jumbo frames? You would hardly notice it. Helium is 4 times as heavy as hydrogen, but only marginally

RE: Internet Exchanges supporting jumbo frames?

...@nanog.org] On Behalf Of Jakob Heitz (jheitz) Sent: Friday, March 18, 2016 18:21 To: Dale W. Carder Cc: nanog@nanog.org Subject: RE: Internet Exchanges supporting jumbo frames? Then it's mainly TCP slowstart that you're trying to improve? Thanks, Jakob. > -Original Message- > From:

Re: Internet Exchanges supporting jumbo frames?

On Fri, 18 Mar 2016 21:29:44 -, "Jakob Heitz (jheitz)" said: > A single bit error will drop a whole packet. > Larger packets will cause more loss. Cables will need to be > shorter or bitrates lower to compensate. If that's an actual concern in your production network, you probably have bigger

Re: Internet Exchanges supporting jumbo frames?

There was one draft few years ago https://tools.ietf.org/html/draft-mlevy-ixp-jumboframes-00#section-3.1 On 17/03/2016 20:49, Chris Woodfield wrote: > Have their been any efforts on the IETF side of things to standardize this, > at least for IPv4/v6 packets?

Re: Internet Exchanges supporting jumbo frames?

Put MTU in BGP announcements? Imagine how much fun we could have if you could make routing decisions based on available path MTU... Regards, Baldur

Re: Internet Exchanges supporting jumbo frames?

What's driving the desire for larger packets? A single bit error will drop a whole packet. Larger packets will cause more loss. Cables will need to be shorter or bitrates lower to compensate. Byte overhead of packet headers? Are we seeing degradation of packets per second in forwarding due to

Re: Internet Exchanges supporting jumbo frames?

I think that’s the problem in a nutshell…until every vendor agrees on the size of a “jumbo” packet/frame (and as such, allows that size to be set with a non-numerical configuration flag). As is, every vendor has a default that results in 1500-byte IP MTU, but changing that requires entering a

Re: Internet Exchanges supporting jumbo frames?

t transfers. > > Tim McKee > > -Original Message- > From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Jakob Heitz (jheitz) > Sent: Friday, March 18, 2016 18:21 > To: Dale W. Carder > Cc: nanog@nanog.org > Subject: RE: Internet Exchanges supporting j

Re: Internet Exchanges supporting jumbo frames?

Thus spake Jakob Heitz (jheitz) (jhe...@cisco.com) on Fri, Mar 18, 2016 at 09:29:44PM +: > What's driving the desire for larger packets? In our little corner of the internet, it is to increase the performance of a low number of high-bdp flows which are typically dataset transfers. All of our

RE: Internet Exchanges supporting jumbo frames?

: Re: Internet Exchanges supporting jumbo frames? > > Thus spake Jakob Heitz (jheitz) (jhe...@cisco.com) on Fri, Mar 18, 2016 at > 09:29:44PM +: > > What's driving the desire for larger packets? > > In our little corner of the internet, it is to increase the performa

Re: Internet Exchanges supporting jumbo frames?

In message , Joel Maslak writes: > On Wed, Mar 9, 2016 at 9:27 AM, joel jaeggli wrote: > > > PMTU blackhole detection implemented in all hosts. IPv4 is lost cause in > > > my opinion (although it's strange

Re: Internet Exchanges supporting jumbo frames?

On Thu, 10 Mar 2016 08:23:30 +0200 Tassos Chatzithomaoglou wrote: > Niels Bakker wrote on 10/3/16 02:44: > > * nanog@nanog.org (Kurt Kraut via NANOG) [Thu 10 Mar 2016, 00:59 > > CET]: > >> I'm pretty confident there is no need for a specific MTU consensus > >> and not all

Re: Internet Exchanges supporting jumbo frames?

> On 9 Mar 2016, at 21:17, Mikael Abrahamsson wrote: > > On Wed, 9 Mar 2016, Nick Hilliard wrote: > >> Many IXPs have either looked at or attempted to build jumbo peering lans. >> You can see how well they worked out by looking at the number of successful >> deployments.

Re: Internet Exchanges supporting jumbo frames?

Hi, On 3/10/2016 9:23 AM, Tassos Chatzithomaoglou wrote: > Niels Bakker wrote on 10/3/16 02:44: >> * nanog@nanog.org (Kurt Kraut via NANOG) [Thu 10 Mar 2016, 00:59 CET]: >>> I'm pretty confident there is no need for a specific MTU consensus and not >>> all IXP participants are obligated to raise

Re: Internet Exchanges supporting jumbo frames?

On Thu, 10 Mar 2016, Tassos Chatzithomaoglou wrote: Mikael Abrahamsson wrote on 10/3/16 18:21: However, I stand by my earlier statement that we need to include MTU/MRU in ND messages, so that this can be negotiated on a LAN where not all devices support large MTU. Isn't this already

Re: Internet Exchanges supporting jumbo frames?

Mikael Abrahamsson wrote on 10/3/16 18:21: > > However, I stand by my earlier statement that we need to include MTU/MRU in > ND messages, so that this can be negotiated on a LAN where not all devices > support large MTU. > Isn't this already supported?

Re: Internet Exchanges supporting jumbo frames?

Mikael Abrahamsson wrote: > However, I stand by my earlier statement that we need to include MTU/MRU > in ND messages, so that this can be negotiated on a LAN where not all > devices support large MTU. this would introduce a degree of network complexity that is unnecessary and would be prone to

Re: Internet Exchanges supporting jumbo frames?

On Thu, 10 Mar 2016, Saku Ytti wrote: On 10 March 2016 at 02:44, Niels Bakker

Re: Internet Exchanges supporting jumbo frames?

On Thu, 10 Mar 2016, Niels Bakker wrote: You're wrong here. The IXP switch platform cannot send ICMP Packet Too Big messages. That's why everybody must agree on one MTU. "Someone" should do an inventory of the market to find out how many commonly used platforms limit MRU to less than 9180

Re: Internet Exchanges supporting jumbo frames?

On Wed, Mar 9, 2016 at 9:27 AM, joel jaeggli wrote: > PMTU blackhole detection implemented in all hosts. IPv4 is lost cause in > > my opinion (although it's strange how many hosts that seem to get away > > with 1492 (or is it 1496) MTU because they're using PPPoE). > > if your

Re: Internet Exchanges supporting jumbo frames?

Martin Pels wrote on 10/3/2016 4:15 μμ: > On Thu, 10 Mar 2016 08:23:30 +0200 > Tassos Chatzithomaoglou wrote: > >> Niels Bakker wrote on 10/3/16 02:44: >>> * nanog@nanog.org (Kurt Kraut via NANOG) [Thu 10 Mar 2016, 00:59 >>> CET]: I'm pretty confident there is no need

Re: Internet Exchanges supporting jumbo frames?

d" <n...@foobar.org> To: "Saku Ytti" <s...@ytti.fi> Cc: "NANOG list" <nanog@nanog.org> Sent: Wednesday, March 9, 2016 1:46:54 PM Subject: Re: Internet Exchanges supporting jumbo frames? Saku Ytti wrote: > If customer does not react, put it on quara

Re: Internet Exchanges supporting jumbo frames?

On 10 March 2016 at 02:44, Niels Bakker

Re: Internet Exchanges supporting jumbo frames?

On 10/Mar/16 00:22, Tassos Chatzithomaoglou wrote: > I must be missing something very obvious here, because i cannot think of any > reason why an IXP shouldn't enable the maximum possible MTU on its > infrastructure to be available to its customers. Then it's clearly customers' > decision on

Re: Internet Exchanges supporting jumbo frames?

Niels Bakker wrote on 10/3/16 02:44: > * nanog@nanog.org (Kurt Kraut via NANOG) [Thu 10 Mar 2016, 00:59 CET]: >> I'm pretty confident there is no need for a specific MTU consensus and not >> all IXP participants are obligated to raise their interface MTU if the IXP >> starts allowing jumbo

Re: Internet Exchanges supporting jumbo frames?

* nanog@nanog.org (Kurt Kraut via NANOG) [Thu 10 Mar 2016, 00:59 CET]: I'm pretty confident there is no need for a specific MTU consensus and not all IXP participants are obligated to raise their interface MTU if the IXP starts allowing jumbo frames. You're wrong here. The IXP switch

Re: Internet Exchanges supporting jumbo frames?

Hello folks, First of all, thank you all for this amazing debate. So many important ideas were exposed here and I wish we keep going on this. I've seen many opposition to my proposal but I still remain on the side of jumbo frame adoption for IXP. I'm pretty confident there is no need for a

Re: Internet Exchanges supporting jumbo frames?

I must be missing something very obvious here, because i cannot think of any reason why an IXP shouldn't enable the maximum possible MTU on its infrastructure to be available to its customers. Then it's clearly customers' decision on what MTU to use on their devices, as long as: * It fits

Re: Internet Exchanges supporting jumbo frames?

Saku Ytti wrote: > Member may puke L2 loop to IXP, you must have some channel to deal > with your customers. First, mac filters. Second, if someone l2 loops and it causes problems because of hardware failure on our side, we reserve the right to pull connectivity:

Re: Internet Exchanges supporting jumbo frames?

On 9/Mar/16 16:26, Kurt Kraut via NANOG wrote: > > Could anyone share with me Internet Exchanges you know that allow jumbo > frames (like https://www.gr-ix.gr/specs/ does) and how you notice benefit > from it? NAPAfrica in South Africa support jumbo frames: https://www.napafrica.net/

Re: Internet Exchanges supporting jumbo frames?

On 10 March 2016 at 00:01, Nick Hilliard wrote: > Other people would be fine with 1522 core because that suits both their > needs and equipment limitations. So what do you do? Go with 9100 > because it suits you, or 9000 because that's what lots of other people > use? Or 4470

Re: Internet Exchanges supporting jumbo frames?

Saku Ytti wrote: > I would go for 1500B edge, and 9100B core, but that's just me. Other people would be fine with 1522 core because that suits both their needs and equipment limitations. So what do you do? Go with 9100 because it suits you, or 9000 because that's what lots of other people use?

Re: Internet Exchanges supporting jumbo frames?

On 9 March 2016 at 22:56, Nick Hilliard wrote: > - hardware problems If we build everything on LCD, we'll have Internet where just HTTP/80 works on 576B. You can certainly find platform which has problems doing else. > - lack of interest among ixp participants outside

Re: Internet Exchanges supporting jumbo frames?

On 9 March 2016 at 22:28, Nick Hilliard wrote: > iirc, we had problems with a bunch of ios based platforms. It worked > fine on junos / xr platforms. I share your surprise that this could > even have caused a problem, but it did. This is very poor reason to kill it for

Re: Internet Exchanges supporting jumbo frames?

On Wed, 9 Mar 2016, Nick Hilliard wrote: Many IXPs have either looked at or attempted to build jumbo peering lans. You can see how well they worked out by looking at the number of successful deployments. The reason for this tiny number isn't due to lack of effort on the part of the ixp

Re: Internet Exchanges supporting jumbo frames?

Saku Ytti wrote: > It works and has worked 2 decades in real IXP. If you're referring to Netnod, this started out as a fddi platform with a native max frame size of 4470. Maintaining something which already exists is not nearly as difficult as starting something from scratch and trying to reach

Re: Internet Exchanges supporting jumbo frames?

Mikael Abrahamsson wrote: > On all platforms I've configured and connected to an IXP, they would all > be configured by setting max L2 MTU on the main interface, and then you > configure whatever needed IPv4 and IPv6 L3 MTU on the subinterface. iirc, we had problems with a bunch of ios based

Re: Internet Exchanges supporting jumbo frames?

On Wed, 9 Mar 2016, Nick Hilliard wrote: For example, many types of hardware don't allow you to specify a different MTU for different .1q tags on the same physical interface. What hardware types typically connected to an IXP would that be, where this would be a problem? On all platforms

Re: Internet Exchanges supporting jumbo frames?

On Wed, Mar 9, 2016 at 9:50 AM, Kurt Kraut via NANOG wrote: > Thank you for replying so quickly. I don't see why the consensus for an MTU > must be reached. IPv6 Path MTU Discovery would handle it by itself, > wouldn't it? If one participant supports 9k and another 4k, the

Re: Internet Exchanges supporting jumbo frames?

On 9 March 2016 at 21:46, Nick Hilliard wrote: > I've spent a good deal of time and effort trying to get a jumbo peering > vlan to work and it didn't work for the reasons that I've mentioned, and > others. It works and has worked 2 decades in real IXP. -- ++ytti, boy who

Re: Internet Exchanges supporting jumbo frames?

Saku Ytti wrote: > If customer does not react, put it on quarantine VLAN. This can be > automated too. Wrong MTU => open internal case, contact customers > email, no customer response in N days, quarantine VLAN. ... and then the customer will leave the service down because it the primary peering

Re: Internet Exchanges supporting jumbo frames?

Saku Ytti wrote: > > If customer does not react, put it on quarantine VLAN. This can be > automated too. Wrong MTU => open internal case, contact customers > email, no customer response in N days, quarantine VLAN. > > Even the most outrageous success stories in the world, majority of the > people

Re: Internet Exchanges supporting jumbo frames?

On 9 March 2016 at 20:59, Nick Hilliard wrote: > There is a critical difference between these two situations. In the case of > an arp sponge, the ixp operator has control of both the polling and the > workaround. In the case of mtu management they would only have control of

Re: Internet Exchanges supporting jumbo frames?

On 9 Mar 2016, at 18:29, Saku Ytti wrote: > It's not a novel idea, IXPs already do active polling, even ARP > sponges. In a competitive market, hopefully customers will choose the > IXP operator who knows how to ensure minimal pain for the customers. There is a critical difference

Re: Internet Exchanges supporting jumbo frames?

On 9 March 2016 at 20:25, Nick Hilliard wrote: > any ixp configuration which requires active polling to ensure correct > configuration is doomed to failure. You are completely overestimating > human nature if you believe that the IXP operator can make this work by > harassing

Re: Internet Exchanges supporting jumbo frames?

Saku Ytti wrote: > I'm suggesting IXP has active poller which detects customer MTU misconfigs. any ixp configuration which requires active polling to ensure correct configuration is doomed to failure. You are completely overestimating human nature if you believe that the IXP operator can make

Re: Internet Exchanges supporting jumbo frames?

On 9 March 2016 at 20:14, Nick Hilliard wrote: > you're recommending that routers at IXPs do inflight fragmentation? I'm suggesting IXP has active poller which detects customer MTU misconfigs. -- ++ytti

Re: Internet Exchanges supporting jumbo frames?

Mikael Abrahamsson wrote: > I have many times ping:ed with 1 byte packets on a device that has > "ip mtu 9000" configured on it, so it sends out two fragments, one being > 9000, the other one around 1100 bytes, only to get back a stream of > fragments, none of them larger than 1500 bytes.

Re: Internet Exchanges supporting jumbo frames?

Saku Ytti wrote: > Poller in the IXP has too large MTU, it tries to send ping packets > with max_size+1, if they work, customer has too large MTU. Also it > tries to send max_size, if it does not work, customer has too small > MTU. As icing on top, it tries to send max_size+1 but fragments it to >

Re: Internet Exchanges supporting jumbo frames?

On Wed 2016-Mar-09 15:32:32 +0100, Grzegorz Janoszka wrote: On 09/03/2016 15:26, Kurt Kraut via NANOG wrote: Could anyone share with me Internet Exchanges you know that allow jumbo frames (like https://www.gr-ix.gr/specs/ does) and how you notice benefit from it?

Re: Internet Exchanges supporting jumbo frames?

On 3/9/16 7:58 AM, Mikael Abrahamsson wrote: > On Wed, 9 Mar 2016, Nick Hilliard wrote: > >> used. Some will want 9000, some 9200, others 4470 and some people > > I have a strong opinion for jumboframes=9180bytes (IPv4/IPv6 MTU), > partly because there are two standards referencing this size

Re: Internet Exchanges supporting jumbo frames?

On 9 March 2016 at 16:34, Job Snijders wrote: > > https://www.nanog.org/sites/default/files/wednesday.general.steenbergen.antijumbo.pdf IXP can verify if MTU is too large or too small with active poller. Poller in the IXP has too large MTU, it tries to send ping packets

Re: Internet Exchanges supporting jumbo frames?

On Wed, 9 Mar 2016, David Bass wrote: Could you do the same with a 1501 byte packet? I have many times ping:ed with 1 byte packets on a device that has "ip mtu 9000" configured on it, so it sends out two fragments, one being 9000, the other one around 1100 bytes, only to get back a

Re: Internet Exchanges supporting jumbo frames?

Could you do the same with a 1501 byte packet? > On Mar 9, 2016, at 10:51 AM, Nick Hilliard wrote: > > Kurt Kraut wrote: >> Thank you for replying so quickly. I don't see why the consensus for an >> MTU must be reached. IPv6 Path MTU Discovery would handle it by itself, >>

Re: Internet Exchanges supporting jumbo frames?

On Wed, 9 Mar 2016, Nick Hilliard wrote: interface MTU configured to be 9000 bytes, the packet will be blackholed, not rejected with a PTB. That is only true if the router/host sets MRU=MTU. That is definitely not always the case.

Re: Internet Exchanges supporting jumbo frames?

On Wed, 9 Mar 2016, Nick Hilliard wrote: used. Some will want 9000, some 9200, others 4470 and some people I have a strong opinion for jumboframes=9180bytes (IPv4/IPv6 MTU), partly because there are two standards referencing this size (RFC 1209 and 1626), and also because all major core

Re: Internet Exchanges supporting jumbo frames?

Kurt Kraut wrote: > Thank you for replying so quickly. I don't see why the consensus for an > MTU must be reached. IPv6 Path MTU Discovery would handle it by itself, > wouldn't it? If one participant supports 9k and another 4k, the traffic > between them would be at 4k with no manual intervention.

Re: Internet Exchanges supporting jumbo frames?

>> Midwest Internet Exchange >> http://www.midwest-ix.com >> >> >> - Original Message - >> >> From: "Kurt Kraut via NANOG" <nanog@nanog.org> >> To: "Nick Hilliard" <n...@foobar.org> >> Cc: "NANOG list&quo

Re: Internet Exchanges supporting jumbo frames?

.com > > > - Original Message - > > From: "Kurt Kraut via NANOG" <nanog@nanog.org> > To: "Nick Hilliard" <n...@foobar.org> > Cc: "NANOG list" <nanog@nanog.org> > Sent: Wednesday, March 9, 2016 8:50:23 AM > Subject: Re:

Re: Internet Exchanges supporting jumbo frames?

uot; <n...@foobar.org> Cc: "NANOG list" <nanog@nanog.org> Sent: Wednesday, March 9, 2016 8:50:23 AM Subject: Re: Internet Exchanges supporting jumbo frames? 2016-03-09 11:45 GMT-03:00 Nick Hilliard <n...@foobar.org>: > this has been tried before at many

Re: Internet Exchanges supporting jumbo frames?

2016-03-09 11:45 GMT-03:00 Nick Hilliard : > this has been tried before at many ixps. No matter how good an idea it > sounds like, most organisations are welded hard to the idea of a 1500 > byte mtu. Even for those who use larger MTUs on their networks, you're > likely to find

Re: Internet Exchanges supporting jumbo frames?

Kurt Kraut via NANOG wrote: > I'm trying to convince my local Internet Exchange location (and it is not > small, exceed 1 terabit per second on a daily basis) to adopt jumbo frames. this has been tried before at many ixps. No matter how good an idea it sounds like, most organisations are welded

Re: Internet Exchanges supporting jumbo frames?

Hi Kurt, On Wed, Mar 09, 2016 at 11:26:35AM -0300, Kurt Kraut via NANOG wrote: > I'm trying to convince my local Internet Exchange location (and it is not > small, exceed 1 terabit per second on a daily basis) to adopt jumbo frames. > For IPv6 is is hassle free, Path MTU Discovery arranges the

Re: Internet Exchanges supporting jumbo frames?

On 09/03/2016 15:26, Kurt Kraut via NANOG wrote: Could anyone share with me Internet Exchanges you know that allow jumbo frames (like https://www.gr-ix.gr/specs/ does) and how you notice benefit from it? Netnod does it in separate vlan's. -- Grzegorz Janoszka

Internet Exchanges supporting jumbo frames?

Hi, I'm trying to convince my local Internet Exchange location (and it is not small, exceed 1 terabit per second on a daily basis) to adopt jumbo frames. For IPv6 is is hassle free, Path MTU Discovery arranges the max MTU per connection/destination. For IPv4, it requires more planning. For