On Mon, 2007-04-16 at 08:21 -0400, Jared Mauch wrote: > There was a presentation years ago at IETF where Cisco showed > the performance increases of enabling path mtu discovery and how this > took those bgp message sizes grew from the cisco default to match your > link mtu.
I find it interesting that you mention Cisco as an example, since their TCP stack for BGP sessions is badly broken in many versions of IOS. The output of `show ip bgp neighbor <ip>` displays the TCP MSS being used for the BGP session. As an example, on recent 6500 IOS (12.2(18)SXF6 in many of my environments) you will find, with BGP MD5 enabled, the TCP stack is reducing the MSS by an extra 20 bytes as a hack to work around the additional space consumed by the TCP MD5 option. This means an iBGP session with MD5 has a TCP MSS of 516, which I think is illegal! In older 6500 IOS, for example 12.2(17r)S2 that I have tested for this, the above TCP MSS adjustment does not happen; and the TCP stack (which I believe is a seperate TCP stack for the BGP process) can actually send packets which exceed the MTU of an egress interface, for example on a VLAN configured for a 1500 byte MTU, it can actually send > 1500 byte IP packets, which will be dropped by neighboring ethernet switches and counted as giant frames in their error counters. Neither behavior is correct; the MSS should not be adjusted, and the TCP stack should decide how much payload to send in the segment based on the MSS and the size of the TCP header, including options. > I'm advocating a consistent internal MTU for your network, be that > 1500, 4470, or something larger. If the underlying transport does not > support it and you are dealing with broken host stacks from your vendors > then you should discontinue using their equipment until they > repair these critical defects. I agree with you that a bigger MTU can be beneficial (there is a parallel thread on the nanog mailing list) but that doesn't mean it's a safe idea to take advantage of it with a bigger MSS for iBGP sessions. I also agree that a consistent MTU for core links is a good idea, but many technologies exist (such as RSVP MTU signaling, path MTU detection, IP fragmenting, transparent layer-3 devices that adjust TCP MSS, etc.) because it is not always practical. If I recall correctly, the Cisco 3550 series are willing to switch (layer-2) jumbo frames on GE ports, but are not willing to route (layer-3) jumbo IP packets on VLAN SVIs, to provide another Cisco example. It is not very practical to throw away all the 3550s you might be using because they won't work with the MTU you have decided must be uniformly supported across your whole network. > Scaling your bgp update messages to something larger than 500 bytes > can have a significant win in route convergence as we're all carrying > voip and other similar sensitive traffic on our networks (even if we don't > know what all that sensitive traffic is). If Cisco was half as good at software / control-plane as they are at hardware / data plane, I doubt they would give a presentation on why raising the MSS on your TCP sessions (to a value which is not guaranteed to be safe on all mediums capable of forwarding IP packets) could be a "significant win." To look at it from another perspective, the TCP MD5 fix that got applied in the 6500 IOS software sometime between 12.2(17r)S2 and 12.2(18)SXF6 was really done because eBGP sessions were breaking; iBGP sessions are assumed to be over 576 byte MTU path by default and it's extremely rare that it's really that small, so iBGP sessions weren't breaking. Yet the "fix" Cisco did reduces the TCP MSS for both eBGP and iBGP sessions. Do you think they had a conversation about how fixing the bug in this manner, instead of correcting the broken behavior of their TCP stack, would have an adverse impact on convergence time? Maybe they did, but it obviously did not change their decision about how to "fix" it. In any event, when changing the TCP MSS of iBGP sessions, it's important to be aware of the implications of that decision. The MTU you use to pick your MSS should be the minimum MTU on any link that might be used to transport your BGP session. -- Jeff S Wheeler <[EMAIL PROTECTED]> +1-212-981-0607 Sr Network Operator / Innovative Network Concepts _______________________________________________ juniper-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/juniper-nsp
