Re: [Int-area] draft-chroboczek-intarea-v4-via-v6 - "IPv4 routes with an IPv6 next hop"
Thanks for writing this down! Wonder what it would take for a host to do router discovery using a different AF. Achieving the equivalent of "ip route add 0.0.0.0/0 via inet6 fe80::1 dev eth0” O. > On 22 Jan 2024, at 15:39, Warren Kumari wrote: > > Hi there all, > > I discovered that I'd somehow misnamed a draft that Juliusz Chroboczek , Toke > Høiland-Jørgensen, and myself had written — somehow I'd managed to name it > draft-chroboczek-int-v4-via-v6, instead of draft-chroboczek-intarea-v4-via-v6. > > Anyway, it is targeted at intarea, and so I renamed and submitted it, so that > it will now actually show up in the IntArea list of documents… > > The document proposes "v4-via-v6" routing, a technique that uses IPv6 > next-hop addresses for routing IPv4 packets, thus making it possible to route > IPv4 packets across a network where routers have not been assigned IPv4 > addresses. > > This isn't yet another "let's rewrite part of the header and override some > bits", nor some new protocol / tunneling thing. It simply notes that routers > only need to determine the outgoing interface (and usually MAC address) for a > packet, and so it's perfectly acceptable for the next-hop for e.g > 192.0.2.0/24 to be e.g 2001:db8::2342. The router don't care… > > While this may be initially surprising to many people, it's actually nothing > "special", nor really groundbreaking - it's just how IP routing works. > However, because it is surprising, it is not getting widely used — and that > means that many interfaces need IPv4 addresses where they otherwise would > not. > > In fact, this functionality is already supported in (at least!): > Arista EOS (since EOS-4.30.1) > The Babel protocol > Linux (since kernel version 5.2) > Mikrotik RouterOS (since before 7.11beta2) > and the BGP protocol (see RFC8950 - "Advertising IPv4 Network Layer > Reachability Information (NLRI) with an IPv6 Next Hop"). > > So, if this already works, why are we writing a document?! > > A few reasons, including: > 1: This behavior / capability is surprising to many people - this means that > people are "forced" to use IPv4 addresses where they otherwise would not. > > 2: There should be an easy way to reference this type of behaviour/deployment > - the genesis of this document that Babel supports this (RFC9229 - "IPv4 > Routes with an IPv6 Next Hop in the Babel Routing Protocol"), but had to > describe the behavior because there was nothing to point at. > > 2: A large number of implementations don't currently support it (or, at > least, the tooling / CLI / UI doesn't allow configurations like the above). > > 3: There are some unsettled questions around the ICMP behavior — e.g: if a > router has to send an ICMP packet too big, and it doesn't have an IPv4 > address, what should it do? > > We'd really appreciate review and feedback — again, this isn't documenting a > major "change", but more noting this the design of command lines, tooling, > etc should allow it. > > W > > ___ > Int-area mailing list > Int-area@ietf.org > https://www.ietf.org/mailman/listinfo/int-area ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Reverse Traceroute - running code
Rolf, As a historical tidbit you may want to include in your draft. IPv6 supported reverse traceroute out of the box. Using RH0 and sending a packet to yourself via the destination. That code is probably still in stacks. O. > On 25 Nov 2022, at 13:56, Rolf Winter wrote: > > Dear Intarea WG, > > in the not so distant future, we will make an update to our internet draft on > a reverse traceroute mechanism, that you can find here: > > https://datatracker.ietf.org/doc/html/draft-heiwin-intarea-reverse-traceroute-00 > > Feedback welcome! > > We now also have publicly available running code (both server and client > implementations), that you can find here: > > https://github.com/HSAnet/reverse-traceroute > > We would appreciate, if people could offer to host a reverse traceroute > server on the public internet, since we are both evaluating the current > implementation and planning our next improvements. If anybody is interested, > please reach out to me. Also, if people would like to run the client, there > is a command line switch that will send us the result of a reverse traceroute > run. We would appreciate, if you could run our client against one or more > public instances of reverse traceroute, that would be great and use the > switch to send us the measurement. So far there is only one public endpoint, > but the list in the ENDPOINTS file in the above mentioned github repository > will be updated, as more public endpoints come online. > > In addition to the implementation, we presented our work at DENOG14 (the > annual conference of the german network operators group). The feedback from > the operational community was very good and there is interest in the work. If > you would like to hear about reverse traceroute and the things that drove our > design, the talk was recorded and is available here: > > https://youtu.be/Y7NtqLEtgjU > > As always, please read the document, send us feedback and if you can host a > reverse traceroute instance, please reach out. > > Thanks a lot, > > Rolf > ___ > Int-area mailing list > Int-area@ietf.org > https://www.ietf.org/mailman/listinfo/int-area ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] New Version Notification for draft-herbert-ipv4-hbh-destopt-00.txt
>> Taking "the IPsec approach" would be creating a new extension header >> and code point that is unique to IPv4-- I don't see how that's any >> better than just using an existing EH defined for IPv6. > > I'm not sure I agree with that. > > When we added IPsec to IPv4, a system that didn't implement IPsec could not > step past the IPsec header when parsing. It had no idea the length of the > ipSEC header. If we add an extension header to IPv6 (for example, when we > added the security header to IPv6), we could continue to parse the IPv6 > header even if we didn't implement a given header beyond parsing. So they are > not the same thing. In the general case you cannot parse over an unknown extension header in IPv6 either. AH and ESP have already been backported from IPv6 to IPv4. I see no principal difference in doing the same with the remaining two (or three) containers options either. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile
Fred, > Ron, it is just a drop in the bucket compared with the amount of discussion > since > "Fragmentation Considered Harmful (1987)". But, I think we now clearly see a > case where fragmentation is *required*. Absolutely. As tunnels produce a new link-layer, that can (should) be a function of that link-layer. Network layer fragmentation is not needed for that. (For the purpose of making the point and to set future direction, ignoring existing IP tunnel mechanisms). Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile
>> 1) It introduces something new and undescribed in paragraph 2. >> "unless they also include mechanisms to detect that IP fragmentation >> isn't working >> reliably." >> That seems like hand-waving to me. Suggest deleting. > > Fragmentation success or failure is directly testable. Any feedback > mechanism will work and specific ones are mentioned elsewhere (PLPMTUD). > > This differs from ICMP black-holing in path MTU detection. Can you please point me to where in the PLPMTUD document testing for IP fragmentation is described? >>> >>> Any feedback mechanism will detect when fragmentation - or anything else - >>> prevents delivery. >> >> Any pointer to such a mechanism in any IETF protocol? >> Would be interesting to get transport/application perspective on this. >> But unless there is a reference I would claim this is hand-waving. > > As I said, PLPMTUD. It doesn’t have to detect if there’s a layer of > fragmentation it can’t detect; it figures out for itself that a given MTU > doesn’t succeed and sends a smaller MTU. Sorry, I still don't understand how PLPMTUD detects if fragmentation works reliably or not. The paragraph we are disccusing is: "Protocols and applications that rely on IP fragmentation will work less reliably on the Internet unless they also include mechanisms to detect that IP fragmentation isn't working reliably." My view is that unless we can point to a reference or specification for how to do this, then it amounts of hand waving. And possibly sending application developers out on a wild goose chase. And given that applications would have to work in the cases where fragmentation was found to not work reliably... >>> If you fragment and put the result inside GRE or UDP, the impact of the >>> fragment (interfering with flow-based multi path routing, nat traversal, >>> etc) is overcome. >> >> Sure, but that only applies to tunnels that go end to end. > > It applies to tunnels that traverse wherever fragmentation is both needed and > doesn’t work other ways. > > Speaking of deployed mechanisms, it’s also how IPsec tunnels over UDP already > work and are widely deployed. It does not work. Host A -- ipsec tunnel -- SGW -- Host B 1) If host A sends fragments inside of the tunnel to host B, the fragments are equally fragile between the security gateway and host B. 2) If the PMTUD between Host A and security gateway is 1280, then the tunnel would have to support link-layer fragmentation anyway, because of the minimum MTU requirement. There are lots of considerations here. Which is why I do not think adding the new sentence that Bob suggested is a good idea. "The risks of IP fragmentation can also be mitigated through the use of encapsulation, e.g., by transmitting IP fragments as payloads." >> ...This document should not recommend IP in UDP in IP encapsulation to >> achieve end to end IP fragmentation for new applications. > > Why not exactly not? Firstly, a fundamental architectural change of how we would recommend application transport to work is not appropriate for this document. Secondly, the goal of this document is not to "rescue IP fragmentation" at any cost. In your above scheme it is arguable if it is even network layer fragmentation. Tunnels as link-layers can define their own link-layer specific fragmentation mechanism. That is a lot more efficient than carrying a superfluous extra header with you. But I don't think any of this can be in the scope of the document. > Frag and reassembly outside of an app can be, in some cases, more efficient - > even with additional layers of headers. Yes, in _some_ cases. Try to introduce 1% of packet drop and that number of cases likely drop to 0. The large scale deployments of this kind of thing is for example TSO/GSO. As a way to bypass bottlenecks in packet processing in the kernel. But there segmentation is done at the transport layer, which will not suffer from the issues of a single packet drop in a 40 fragment chain. (64K packet / 1500). >> If this paragraph has to be there it would be more accepting to have it in >> the "Legacy protocols" parapgraph above. > > It’s unnecessarily limiting to mention it only there. The whole purpose of this document is to limit use of network layer fragmentation. Section 6.1 with the two sentences deleted: 6.1. For Application and Protocol Developers Developers SHOULD NOT develop new protocols or applications that rely on IP fragmentation. When a new protocol or application is deployed in an environment that does not fully support IP fragmentation, it SHOULD operate correctly, either in its default configuration or in a specified alternative configuration. While there may be controlled environments where IP fragmentation works reliably, this is a deployment issue and can not be known to someone developing a new protocol or application. It is not recommended that new
Re: [Int-area] Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile
> And of course encapsulation can also exacerbate the problem > by increasing packet size. > > All this means is that the fragmentation layer needs to take into account the > size of the outer encapsulation layers that will be added and make sure its > fragments do not exceed 1280 bytes *after* encapsulation. So, e.g., if the > encapsulation layer adds an IPv6 header and a UDP header the fragmentation > layer should not produce fragments larger than 1280 - 40 - 8 = 1232. If the > fragmentation layer does not know the size of the outer encapsulations to > be added, it can overestimate and set a safe smaller value (e.g., 1024). Yes, absolutely. But I don't think we are talking about IP fragmentation any more. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile
Joe, edited to focus on the two added "recommendation sentences". 1) It introduces something new and undescribed in paragraph 2. "unless they also include mechanisms to detect that IP fragmentation isn't working reliably." That seems like hand-waving to me. Suggest deleting. >>> >>> Fragmentation success or failure is directly testable. Any feedback >>> mechanism will work and specific ones are mentioned elsewhere (PLPMTUD). >>> >>> This differs from ICMP black-holing in path MTU detection. >> >> Can you please point me to where in the PLPMTUD document testing for IP >> fragmentation is described? > > Any feedback mechanism will detect when fragmentation - or anything else - > prevents delivery. Any pointer to such a mechanism in any IETF protocol? Would be interesting to get transport/application perspective on this. But unless there is a reference I would claim this is hand-waving. [...] > If you fragment and put the result inside GRE or UDP, the impact of the > fragment (interfering with flow-based multi path routing, nat traversal, etc) > is overcome. Sure, but that only applies to tunnels that go end to end. And any development of new tunnel mechanisms don't need to depend on IP fragmentation. This is essentially link-layer (the tunnel provides a new link layer) fragmentation and reassembly. It would anyway have to do that to deal with IPv4 DF=1 and IPv6. This document should not recommend IP in UDP in IP encapsulation to achieve end to end IP fragmentation for new applications. If this paragraph has to be there it would be more accepting to have it in the "Legacy protocols" parapgraph above. Legacy protocols that depend upon IP fragmentation SHOULD be updated to break that dependency. However, in some cases, there may be no viable alternative to IP fragmentation (e.g., IPSEC tunnel mode, IP- in-IP encapsulation). Applications and protocols cannot necessarily know or control whether they use lower layers or network paths that rely on such fragmentation. In these cases, the protocol will continue to rely on IP fragmentation but should only be used in environments where IP fragmentation is known to be supported. The risks of IP fragmentation can also be mitigated through the use of encapsulation, e.g., by transmitting IP fragments as payloads. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile
Joe, > Comments below. > >> On Sep 5, 2019, at 11:33 PM, Ole Troan wrote: >> >> Bob, et al, >> >> I have two issues with this text. >> >> 1) It introduces something new and undescribed in paragraph 2. >> "unless they also include mechanisms to detect that IP fragmentation isn't >> working >> reliably." >> That seems like hand-waving to me. Suggest deleting. > > Fragmentation success or failure is directly testable. Any feedback mechanism > will work and specific ones are mentioned elsewhere (PLPMTUD). > > This differs from ICMP black-holing in path MTU detection. Can you please point me to where in the PLPMTUD document testing for IP fragmentation is described? I thought PLPMTUD found the largest unfragmented size of a datagram. >> 2) Paragraph 4: >> "The risks of IP fragmentation can also be mitigated >> through the use of encapsulation, e.g., by transmitting IP fragments >> as payloads." >> >> This seems like proposing new unspecified solutions with it's own set >> of considerations. > > Specific widely-deployed methods are mentioned elsewhere in the doc, > including GRE and UDP. Sorry, I couldn't find those either. Inner fragmentation, firstly is only applicable to IPv4. And only applicable to tunnels. And both those specs go to great length in avoiding fragmentation. >> IP fragmentation is a general solution to all hosts, >> encapsulation is certainly not in every host, > > Actually, it is - unless you’re claiming hosts don’t support UDP. Sorry, I don't understand what you mean. Are you saying that a new UDP applications should support the following stack: IPv6 + UDP + IPv6 + FH + UDP + Applcation data So to be able to hide IP fragments from the network? While still having to do the full PLPMTUD to function correctly? >> and has different >> properties with regards to NAT traversal etc. > > If by “different” you mean “much more likely to succeed”, agreed. I need to see numbers of that. But regardless I don't see the relevance to this document. >> vAlso if encapsulation >> was the answer, other segmentation / reassembly that were tunnel >> specific could be developed. > > It is and is widely used - IPsec tunnels over UDP, e.g. That's a encapsulated solution to start with. >> Regardless this also amounts of hand-waving >> and doesn't seem to offer any advice that can be heeded now. >> And of course encapsulation can also exacerbate the problem >> by increasing packet size. > > Yes, it makes the fragments smaller, which may be additional > effort/performance impact, but it completely hides its impact on successful > forwarding. You may be making a point. I'm afraid I don't get it. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile
Bob, et al, I have two issues with this text. 1) It introduces something new and undescribed in paragraph 2. "unless they also include mechanisms to detect that IP fragmentation isn't working reliably." That seems like hand-waving to me. Suggest deleting. 2) Paragraph 4: "The risks of IP fragmentation can also be mitigated through the use of encapsulation, e.g., by transmitting IP fragments as payloads." This seems like proposing new unspecified solutions with it's own set of considerations. IP fragmentation is a general solution to all hosts, encapsulation is certainly not in every host, and has different properties with regards to NAT traversal etc. Also if encapsulation was the answer, other segmentation / reassembly that were tunnel specific could be developed. Regardless this also amounts of hand-waving and doesn't seem to offer any advice that can be heeded now. And of course encapsulation can also exacerbate the problem by increasing packet size. Suggest deletion. New text: 6.1. For Application and Protocol Developers Developers SHOULD NOT develop new protocols or applications that rely on IP fragmentation. When a new protocol or application is deployed in an environment that does not fully support IP fragmentation, it SHOULD operate correctly, either in its default configuration or in a specified alternative configuration. While there may be controlled environments where IP fragmentation works reliably, this is a deployment issue and can not be known to someone developing a new protocol or application. It is not recommended that new protocols or applications be developed that rely on IP fragmentation. Protocols and applications that rely on IP fragmentation will work less reliably on the Internet. Legacy protocols that depend upon IP fragmentation SHOULD be updated to break that dependency. However, in some cases, there may be no viable alternative to IP fragmentation (e.g., IPSEC tunnel mode, IP- in-IP encapsulation). Applications and protocols cannot necessarily know or control whether they use lower layers or network paths that rely on such fragmentation. In these cases, the protocol will continue to rely on IP fragmentation but should only be used in environments where IP fragmentation is known to be supported. Protocols may be able to avoid IP fragmentation by using a sufficiently small MTU (e.g. The protocol minimum link MTU), disabling IP fragmentation, and ensuring that the transport protocol in use adapts its segment size to the MTU. Other protocols may deploy a sufficiently reliable PMTU discovery mechanism (e.g.,PLMPTUD). UDP applications SHOULD abide by the recommendations stated in Section 3.2 of [RFC8085]. Cheers, Ole > On 6 Sep 2019, at 06:05, Bob Hinden wrote: > > Hi, > > Joe and I talked off list. The result is below. Changes were to add a > sentence in the forth and fifth paragraphs. > > Please review. > > Bob > > -- > > 6.1. For Application and Protocol Developers > > Developers SHOULD NOT develop new protocols or applications that rely > on IP fragmentation. When a new protocol or application is deployed > in an environment that does not fully support IP fragmentation, it > SHOULD operate correctly, either in its default configuration or in a > specified alternative configuration. > > While there may be controlled environments where IP fragmentation > works reliably, this is a deployment issue and can not be known to > someone developing a new protocol or application. It is not > recommended that new protocols or applications be developed that rely > on IP fragmentation. Protocols and applications that rely on IP > fragmentation will work less reliably on the Internet unless they > also include mechanisms to detect that IP fragmentation isn't working > reliably. > > Legacy protocols that depend upon IP fragmentation SHOULD be updated > to break that dependency. However, in some cases, there may be no > viable alternative to IP fragmentation (e.g., IPSEC tunnel mode, IP- > in-IP encapsulation). Applications and protocols cannot necessarily > know or control whether they use lower layers or network paths that > rely on such fragmentation. In these cases, the protocol will > continue to rely on IP fragmentation but should only be used in > environments where IP fragmentation is known to be supported. > > Protocols may be able to avoid IP fragmentation by using a > sufficiently small MTU (e.g. The protocol minimum link MTU), > disabling IP fragmentation, and ensuring that the transport protocol > in use adapts its segment size to the MTU. Other protocols may > deploy a sufficiently reliable PMTU discovery mechanism > (e.g.,PLMPTUD). The risks of IP fragmentation can also be mitigated > through the use of encapsulation, e.g., by transmitting IP fragments > as payloads. >
Re: [Int-area] Alissa Cooper's No Objection on draft-ietf-intarea-frag-fragile-16: (with COMMENT)
Fred, >> Is that not the exact definition of outer fragmentation? > > No. I am talking about outer header (OH) followed by tunnel header (TH) > followed > by inner packet (IP). Recipe: > > 1) wrap the IP in a TH to create a tunnel packet (TP) > 2) fragment the TP > 3) encapsulate each tunnel fragment in an independent OH > 4) send each outer packet (OP). These will look like ordinary > unfragmented IP packets, but will contain a tunnel fragment Right you might as well define your own tunnel fragmentation then. And it will have a lot less overhead. Doing "segmentation" at what would be layer 4 (or 3.5) isn't what we are trying to discourage here. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Alissa Cooper's No Objection on draft-ietf-intarea-frag-fragile-16: (with COMMENT)
Fred, I removed the IESG from this list, as we seem to have drifted into a more general fragmentation discussion as opposed to discussing the exact changes to this draft. Why is that more useful than what is in 3.5? If it’s not making a recommendation, why call this out in the introduction. There are lot >> of other things it doesn’t make recommendations about that aren’t in the Introduction either. >>> >>> Because it sets a more appropriate tone and lets the reader know from the >>> onset that >>> fragmentation and encapsulation go hand in hand. And tunnel fragmentation >>> avoids the >>> issues raised by others in this thread. >> >> While inner fragmentation ensures the fragment will reach the tunnel tail >> end, a tunnel endpoint will typically not reassemble that >> fragment, so will generate fragments after the tunnel hop. >> Inner fragmentation is only available on IPv4. > > Not true. For IPv6 packets, simply insert a GUE header or an RFC2473 header > and > fragment on that. The fragments will be reassembled by the tunnel tail end, > then > passed to the next hop as a whole IPv6 packet. The fragmentation footprint is > therefore the same as the tunnel footprint. Is that not the exact definition of outer fragmentation? >> Outer fragmentation will look like any other fragmented packet, > > I am not talking about outer fragmentation. Ehm? >> albeit that the tunnel tail now has to reassemble. At speeds typically >> much higher than a typical end host. > > Using iperf3, I can show fragmentation and reassembly at near line-rate on > 10Gbps > Ethernet gear. That seems pretty good to me. Which shows that implementers > have taken IP fragmentation seriously and put in the hard work necessary to > optimize the performance. On what hardware? 10G is not at all very much, and given fragmentation you have large packets anyway. You need to compare the forwarding performance unfragmented versus fragmented. Then impact of out of order fragments. Fragment chains that don't reassemble, etc etc. Proving that it performs (or not as I would claim) does require quite a bit of work. >> Tunnels within a controlled domain may use fragmentation, although it still >> will have problems. >> Which is why you see most tunnel specifications for controlled domains, >> state that the network MTU must be "well managed". > > We should be able to tunnel within any domain, be it controlled or over the > open Internet. > Inner fragmentation (with nested encapsulation if necessary) accomplishes > that. Sure, you can just slap a UDP header in between. It still be outer fragmentation, but you have hidden it from the intermediate network. Then you might as well invent your own shim tunnel fragment header, and we could deprecate the IPv6 one. (slight smiley on that idea:) Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Alissa Cooper's No Objection on draft-ietf-intarea-frag-fragile-16: (with COMMENT)
Fred, >> Why is that more useful than what is in 3.5? If it’s not making a >> recommendation, why call this out in the introduction. There are lot of >> other things it doesn’t make recommendations about that aren’t in the >> Introduction either. > > Because it sets a more appropriate tone and lets the reader know from the > onset that > fragmentation and encapsulation go hand in hand. And tunnel fragmentation > avoids the > issues raised by others in this thread. While inner fragmentation ensures the fragment will reach the tunnel tail end, a tunnel endpoint will typically not reassemble that fragment, so will generate fragments after the tunnel hop. Inner fragmentation is only available on IPv4. Outer fragmentation will look like any other fragmented packet, albeit that the tunnel tail now has to reassemble. At speeds typically much higher than a typical end host. Tunnels within a controlled domain may use fragmentation, although it still will have problems. Which is why you see most tunnel specifications for controlled domains, state that the network MTU must be "well managed". In summary, I don't think the text can say very much more than what it already does. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] I-D Action: draft-ietf-intarea-frag-fragile-06.txt
Tom, > It does seem that Cisco configuration manuals and marketing materials > are the best source of information on virtual reassembly. > > https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipaddr_frre/configuration/xe-3s/frre-xe-3s-book/virt-frag-reassembly.html > is a little more interesting in that it provides a few more details. > In particular the requirement that all fragments must traverse the > same intermediate device is mentioned: > > "The reassembly process requires all fragments within an IP datagram. > If fragments within an IP datagram are sent to different devices due > to load balancing VFR may fail and fragments may be dropped.” I can’t tell if you are actually interested in how it works, or if you are trying to make a point. Assuming the former, here is an implementation: https://github.com/FDio/vpp/blob/master/src/plugins/map/ip4_map.c#L517 if first fragment in chain found = lookup 4 tuple in reassenbly cache if found forward buffered packets else create session state entry in reassembly cache forward packet else found = lookup 4 tuple in reassembly cache if found forward packet else buffer packet Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] draft-ietf-intarea-frag-fragile-03
> On 30 Nov 2018, at 18:33, Joe Touch wrote: > > > >> On Nov 30, 2018, at 9:22 AM, Ole Troan wrote: >> >> >> >>> On 30 Nov 2018, at 16:49, Joe Touch wrote: >>> >>> 1) the lower down the fragmentation occurs, the less overhead is needed >>> (i.e., when performance is an issue, it’s even more important to fragment >>> as low as possible) >> >> That sounds like an unfounded myth. >> I would think it highly dependent on implementation. > > Reality: > > - every layer down you do it avoids a layer of header in-between *at every > fragment* > ie., IP fragments have only ONE UDP header and ONE application header, but > app-fragments have multiples of both. > > Do the math. Every ipv6 fragment has an additional 8 byte header. But the network might not be the bottleneck here, and a few more bytes might not matter. As I said it depends. When it comes to performance making blanket statements is rarely a good idea. Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] draft-ietf-intarea-frag-fragile-03
> On 30 Nov 2018, at 16:49, Joe Touch wrote: > > 1) the lower down the fragmentation occurs, the less overhead is needed > (i.e., when performance is an issue, it’s even more important to fragment as > low as possible) That sounds like an unfounded myth. I would think it highly dependent on implementation. Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
> >> Of course you can pass non TCP/UDP through limited domains, but you cannot >> expect that to work across the Internet anymore. > > You mean the IPv4nternet, I hope ;-). I hope so too. ;-) Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
Joe, >> For IPv4 part of the transport headers are now part of the network layer. >> Forwarding is done one address and port. >> I.e it's now part of 'normal' routing/forwarding. That's absolutely not the >> case. In IPv6 packets are delivered to hosts based on longest match on >> destination addresses, while for IPv4 it's a match IPv4 DA and DP. > For ECMP, this is never an issue. ECMP can easily use a default for fragments > and move on with life. I am not sure why you bring in ECMP. (It is actually a problem for ECMP, since fragmented and non-fragmented packets are prone to take different routes, but that seems to be far off topic). > For end-system load balancing, similar defaults can work, but it's also > reasonable to expect that a device that acts on behalf of an end system that > way should do the work of an end system to reassemble. > > For DPI, reassembly - and sometimes restoration of the TCP data stream across > segments - is necessary anyway. > > For NAT, virtual reassembly is necessary. If the packet doesn’t isn’t TCP or UDP and the ports are available any IPv4 packet will have a significantly higher drop probability. I’m fine with you not accepting that. > > None of these are specific to IPv4; even in IPv6, flow IDs are not a magic > solution -- they need similar fragmentation support. > Of course they are. One is IETF mandated and required to be able to forward packets, the other is not. Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
Tom, > I don't believe this can be true. Not all protocols even have port > numbers (e.g. ICMP, ESP) and yet we still expect them to be > deliverable. Maybe your referring to ECMP, which does route based on > flow (e.g. port information)? But, ECMP is not required to make IP > work either. Forwarding solely based on destination IPv4 address for > unicast packets needs to work. I can’t see how you can expect anything but TCP and UDP and (some) ICMP to be deliverable. These just do not pass through stuff like A+P, CGNs. Of course you can pass non TCP/UDP through limited domains, but you cannot expect that to work across the Internet anymore. Cheers, Ole > >> I.e it’s now part of ‘normal’ routing/forwarding. That’s absolutely not the >> case. In IPv6 packets are delivered to hosts based on longest match on >> destination addresses, while for IPv4 it’s a match IPv4 DA and DP. > > ECMP is also done for IPv6. I don't see how routing is fundamentally > different in IPv6 except that the flow label could be a good > substitute for ports in ECMP. > > Tom ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
Joe, >>> Again, this does nothing for middleboxes that can handle IPv6 fragments. >>> Flow labels don’t fix DPI or NAT devices. >> >> It wasn’t meant to. If you read my post, I clearly stated the IPv4 Internet. >> >> There is much less justification to “encourage deprecation” of IPv6 >> fragments. >> I think that if so has to be a much more archirectural view of the layering, >> and that network layer fragments problematic for the end host, because it >> can’t do any validation checks on them before reassembly. > > Overlap checks can be done. > > What other kind of check do you think would be meaningful or helpful on > fragments alone? Exactly, and that’s the problem with fragmentation at the network layer. The end host must do reassembly before deciding to throw the packet or not. >> From an IETF perspective middleboxes do not exist in IPv6 (sic). > > And *I’m* the one in denial?? ;-) >> And if we really want to punt fragmentation to the transport layer, I think >> the argument should not be made using middleboxes for justification. > > You might consider that middleboxes are not IP-only devices; they depend on > and modify transport layer headers and even content sometimes. > >> >> Do you see why I insist on making a distinction between IPv4 and IPv6? > > Except for the flow ID for IPv6 - which helps only when used properly AND for > load distribution - no. For IPv4 part of the transport headers are now part of the network layer. Forwarding is done one address and port. I.e it’s now part of ‘normal’ routing/forwarding. That’s absolutely not the case. In IPv6 packets are delivered to hosts based on longest match on destination addresses, while for IPv4 it’s a match IPv4 DA and DP. So no, we can’t treat the protocols the same. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
>> RFC7596 specifies destination unreachable, host unreachable for this. > You have to be careful; that would cause the origin to cease sending all > traffic to that IP address - which works for that RFC (which is L3 oriented), > when in fact it is only the lack of the L4 info that makes it unreachable. > > IMO, if there is a message here, it's reassembly timeout. Anything else runs > the risk of killing off non-fragmented packets to the same destination. > If this signal stopped the host from sending IPv4 packets all the better. ;-) Seriously though, I think the IETF should be a lot more agreesive in documenting the consequences to the IPv4 Internet architecture of the IPv4 run-out. This draft is part of it, but unfortunately you are pushing to smooth things over and make the draft say how IPv4 should have worked in an optimal pre-runout world, instead of accepting how IPv4 works now. We as in the IETF should also make very clear that the IPv4 Internet: - requires client-server initiation - only supports TCP, UDP and some ICMP - with limited number of ports available, and sessions will time out on you. - IP fragmentation has a higher drop probability than other TCP, UDP traffic And to use the end to end Internet that’s only available on IPv6. Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
> See my suggested text - I.e., if you can’t handle the fragments they way you > WANT, then you need to forward them. To where? When the forwarding decision is based on L4 information, as it is inthe IPv4 Internet, the best we could do is to send an ICMP destination unreachable back. > > IMO, if you don’t know what to do with fragments you can’t process AT ALL, > then your middlebox service is either improperly designed or improperly > deployed. We have accepted the fact that the only way to make the IPv4 Internet continue operate is to do so improperly. The earth is also round. Joe ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
> On 12 Nov 2018, at 11:11, Joe Touch wrote: > > > > On Nov 12, 2018, at 1:36 AM, Ole Troan wrote: > >>> And none of these update RFC1812. >>> >>> These things can be rolled out in any numbers you like - they are creating >>> the problem that only they can clean up. Killing capability in the rest of >>> the Internet to make these mechanisms work isn’t a viable solution and >>> never has been. >> >> And if that makes the IPv4 Internet die quicker, I’ll take that as a feature. > > Sure, as long as your goal is to kill IPv6 too. > > Joe And that’s probably as good an end to this thread as any… Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
> And none of these update RFC1812. > > These things can be rolled out in any numbers you like - they are creating > the problem that only they can clean up. Killing capability in the rest of > the Internet to make these mechanisms work isn’t a viable solution and never > has been. And if that makes the IPv4 Internet die quicker, I’ll take that as a feature. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
Joe, > On 9 Nov 2018, at 00:55, Joe Touch wrote: > > Experimental. For a reason. No. The protocol instances are standard documents. Rfc7596, 7597, 7599. We have had this discussion before. We are not going to agree, and there might not be any purpose disturbing the wg with this. But just as you know these mechanisms are rolled out in increasing numbers (of course, since there is no choice until universal deployment of IPv6) and there are millions of users behind them already. Ole > >> On Nov 8, 2018, at 9:13 AM, Ole Troan wrote: >> >> >> >>> On 9 Nov 2018, at 00:02, Joe Touch wrote: >>> >>> Please indicate the basis of your belief. Mine is RFC1812. >> >> CGNs, RFC6346 et al. >> Your belief system is unfortunately outdated. >> I don’t like it anymore than you do. >> >> And no, you cannot pretend these don’t exist by declaring them extensions of >> a host. Not even under same administrative control. >> >> Ole >> >>> >>>> On Nov 8, 2018, at 8:43 AM, Ole Troan wrote: >>>> >>>> >>>> >>>>> On 8 Nov 2018, at 23:15, Joe Touch wrote: >>>>> >>>>> >>>>> It always can forward based on IP info alone. It is only the desire to >>>>> alter packets in transit or forward based on transport info that gets in >>>>> the way. >>>> >>>> Joe, I have told you many times over that this is wrong for IPv4. >>>> Repeating it does not make it less wrong. >>>> >>>> Ole >>> >> > ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
> On 9 Nov 2018, at 00:02, Joe Touch wrote: > > Please indicate the basis of your belief. Mine is RFC1812. CGNs, RFC6346 et al. Your belief system is unfortunately outdated. I don’t like it anymore than you do. And no, you cannot pretend these don’t exist by declaring them extensions of a host. Not even under same administrative control. Ole > >> On Nov 8, 2018, at 8:43 AM, Ole Troan wrote: >> >> >> >>> On 8 Nov 2018, at 23:15, Joe Touch wrote: >>> >>> >>> It always can forward based on IP info alone. It is only the desire to >>> alter packets in transit or forward based on transport info that gets in >>> the way. >> >> Joe, I have told you many times over that this is wrong for IPv4. Repeating >> it does not make it less wrong. >> >> Ole > ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
> On 8 Nov 2018, at 23:15, Joe Touch wrote: > > > It always can forward based on IP info alone. It is only the desire to alter > packets in transit or forward based on transport info that gets in the way. Joe, I have told you many times over that this is wrong for IPv4. Repeating it does not make it less wrong. Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Stateless devices and IP fragmentation
>> We need compliance checks to fix this, or to find another way to put more >> pain where the problem lies. > > I would like to hear from the vendors of the stateless devices whether > they think this is feasible solution. Implementations tend to live within the laws of physics, so demanding something isn’t going to make it so. In our open source virtual reassembly implementation we go to great lengths to try to make it work. But fragments will have a significantly higher drop probability. Virtual reassembly opens up a big attack vector, and a lot of work is required to be able to handle these attacks. At least in our implementation we do not have enough buffers to be anywhere near RFC compliant. IPv4 has a 15s reassembly timer, IPv6 a 60s reassembly timer. If I remember correctly we have default 100ms reassembly timer, with a maximum fragment chain of 3. And a maximum outstanding reassemblies of 1000. Which is not very much given we can process hundreds of millions of packets per second. I have looked at some packet traces with fragments. From the ones I saw: - a significant amount of fragmented packets were attackes UDP port 80 etc. - didn’t reassemble at all - were out of order I would be very interested if anyone had more data on this. Both to tune our virtual reassembly algorithm and to see how big the problem is. Btw: This is for IPv4 reassembly. Where we as a community have decided that we must make forwarding decisions on transport layer ports. So please don’t bother with the “ah your middlebox shouldn’t be there in the first place”. Code: https://git.fd.io/vpp/tree/src/plugins/map/ip4_map.c#n511 Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
[Int-area] Side Meeting: 6MAN/TSVWG Path MTU Discovery Discussion
We have scheduled a side meeting to discuss IPv6 Path MTU Discovery to continue the discussion at today’s 6man session. It will be: Fri, November 9, 10am – 12pm Where: Thai Chitlada 2 room The goal is to discuss the current and new Path MTU Discovery approaches. Please let us know if you plan to attend so we can make sure the room is large enough. Bob & Ole & Gorry ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] [dhcwg] [v6ops] Intro to draft-patterson-intarea-ipoe-health-00
Richard, > As a side-note, is it really that challenging or slow to get a new > DHCP option assigned? Perhaps I'm showing my naivety here. Getting the DHCP option itself isn’t hard. But then you need to get it deployed in DHCP servers, you need to get it deployed in DHCP clients. And you need to add linkage between the health checker and DHCP. Harder from implementors perspectiv and harder from deployers perspective. Does the extra work add value? It is certainly going to add significant amount of time. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] [dhcwg] [v6ops] Intro to draft-patterson-intarea-ipoe-health-00
>> I'm not sure DHCP configuration is really needed. It seems like overkill. > > Other reviewers/commenters previously felt that this was a useful > addition. This would allow Network Operators to trigger or influence > the connectivity check on CE routers that they are not in TR.069/etc. > control of. Requiring the DHCP option makes a big difference in deployability. Without it, I can implement this feature today. With it, I have to wait until the DHCP option is standardised. And we would presumably get into a big debate about what CE routers should do in the cases where they don’t get a DHCP option. Should they try this mechanism anyway? We have not requied explicit configuration in the other cases where we have recommended this mechanism. Ref. RFC5969. While the echo mechanism requires some special provisioning on the local system (ensure that ingress filtering isn’t blocking packets with yourself as source) I am not aware of anything on the PE that woukd block this. If there is consensus on that, I think it’s perfectly fine to require this mechanism on by default in CE routers. Although we might add some specifics to deal with a case where DHCP was successful, state in PE was correct, but health check still failed. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, > On 27 Aug 2018, at 10:27, Joe Touch wrote: > > > >> On Aug 26, 2018, at 11:55 PM, Ole Troan wrote: >> >> Joe, >> >>>> >>>>> On 26 Aug 2018, at 23:12, Joe Touch wrote: >>>>> >>>>> As I’ve mentioned, there are rules under which a NAT is a valid Internet >>>>> device, but it is simply not just a router. >>>> >>>> If there really was, can you point to where those rules are? Describing >>>> the behavior of the host stack and applications? >>> >>> The principles are described and explained here: >>> >>> Touch, J: Middlebox Models Compatible with the Internet. USC/ISI >>> (ISI-TR-711), 2016. ( >>> >> >> I don’t want to dismiss this completely, but it hand waves over how >> applications are supposed to work in this new Internet architecture. >> You can define your way out of breaking end-to-end, but that doesn’t mean >> you can ignore all the issues of NAT traversal. > > You’ve missed the point - if the middleboxes describe behave as required, > apps do not need to change. They work as they would in an Internet without > those boxes. Quite likely. Do you have a document describing how my SIP application works? Ore are you saying PCP, ICE, TURN etc is part of your architecture? Cheers Ole___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, >>> >>> On 26 Aug 2018, at 23:12, Joe Touch wrote: >>> >>> As I’ve mentioned, there are rules under which a NAT is a valid Internet >>> device, but it is simply not just a router. >> >> If there really was, can you point to where those rules are? Describing the >> behavior of the host stack and applications? > > The principles are described and explained here: > > Touch, J: Middlebox Models Compatible with the Internet. USC/ISI > (ISI-TR-711), 2016. ( > I don’t want to dismiss this completely, but it hand waves over how applications are supposed to work in this new Internet architecture. You can define your way out of breaking end-to-end, but that doesn’t mean you can ignore all the issues of NAT traversal. Cheers Ole___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
> On 26 Aug 2018, at 23:12, Joe Touch wrote: > > As I’ve mentioned, there are rules under which a NAT is a valid Internet > device, but it is simply not just a router. If there really was, can you point to where those rules are? Describing the behavior of the host stack and applications? Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] I-D Action: draft-ietf-intarea-frag-fragile-00.txt
> On 16 Aug 2018, at 20:10, Joe Touch wrote: > > > >> On Aug 16, 2018, at 10:02 AM, Ole Troan wrote: >> >> These are not NATs. They are specifically designed to be stateless > > I think we also can agree that this, while intended, is the point of failure. Sure, pick your poison. If you think a CGN cluster is more end to end friendly. It’s fascinating to watch IPv4 evolve through its last phase. :-) Cheers Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] I-D Action: draft-ietf-intarea-frag-fragile-00.txt
Joe, > On 16 Aug 2018, at 15:58, Joe Touch wrote: > > > >> On Aug 16, 2018, at 5:47 AM, Ole Troan wrote: >> >> Joe, >> >>>> IPv4 fragments do have a higher drop probability than other packets. Just >>>> from the fact that multiple end-users are sharing a 16 bit identifier >>>> space. >>> >>> It’s really the fact that NATs that process fragments don’t reassemble >>> before translating and/or don’t rate limit fragments they generate as >>> already required by 791 (as explained in 6884). >> >> That’s incorrect. >> See https://tools.ietf.org/html/rfc7597#section-8.3.3 > > You should re-read that RFC. It correctly points out that this is a flaw in > current devices. > > There is a solution - reassemble before NATing, and when issuing the new > packets, issue then with IDs generated at the NAT. These are not NATs. They are specifically designed to be stateless. Sure you can argue that the A+P solutions break the Internet, our answer to that oh well, move to IPv6. > The correct behavior is already indicated in RFC 6864, Sec 5.3.1 >>> A NAT that is broken isn’t helping users share addresses. It’s just broken. >> >> I wish it was that simple. > > It’s not simple, but saying that “fragmentation is broken” does not make it > more simple either. True. Regardless, I fear we aren’t going to agree on this, but at least I think we have understood each other’s points. Cheers Ole___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] I-D Action: draft-ietf-intarea-frag-fragile-00.txt
Joe, >> IPv4 fragments do have a higher drop probability than other packets. Just >> from the fact that multiple end-users are sharing a 16 bit identifier space. > > It’s really the fact that NATs that process fragments don’t reassemble before > translating and/or don’t rate limit fragments they generate as already > required by 791 (as explained in 6884). That’s incorrect. See https://tools.ietf.org/html/rfc7597#section-8.3.3 > A NAT that is broken isn’t helping users share addresses. It’s just broken. I wish it was that simple. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] I-D Action: draft-ietf-intarea-frag-fragile-00.txt
>> Whether that automatically solves the problem depends on whether >> PMTUD works, which is unpredictable. Therefore, if you want a protocol >> *design* that is universal, it can't depend on PMTUD. Disabling >> fragmentation on its own is not enough. (That isn't news.) >> >> The only universal solution therefore seems to be a fully specified >> PLPMTUD solution, either built into the protocol design, or available >> as a normative reference. I think that would be a better >> recommendation than simply saying "don't rely on fragmentation." > > Summary of my views: > > Networks MUST support PMTUD. > Networks MUST support IP level fragmented packets. That is not realistic for the IPv4 Internet. IPv4 fragments do have a higher drop probability than other packets. Just from the fact that multiple end-users are sharing a 16 bit identifier space. > Protocols/applications SHOULD have PMTUD blackhole detection. > Protocols/applications SHOULD do PLPMTUD. > Protocols/applications SHOULD avoid IP level fragmentation. > >> IMHO RFC4821 isn't a sufficient normative reference for this. Possibly >> draft-ietf-tsvwg-datagram-plpmtud will do it for UDP-based protocols. But >> IMHO we need a solution for each transport protocol, with widely available >> libraries. > > Agreed. The libraries part is extremely important. Programmers implementing > applications should not have to worry about this, it should be taken care of > by the libraries they use. For them it should just work. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, >>> It also looks like (at first glance at least) these devices work only when >>> there isn't multipath between the back and front side. >> >> The A+P routers are stateless and do support multipath. Including traffic >> does not need to be symmetric. >> That’s the main selling point for A+P, that you don’t need per flow state in >> the core of the network. > > The +P part doesn’t seem like it’s compatible with fragmentation, though - > yet it doesn’t update RFC791 to deprecate it throughout the Internet. It’s not incompatible with fragmentation. Just that there are some pitfalls. As explained in rfc7597. Fragmented packets do have a higher drop probability, and a higher probability of mis-reassembly on the end system with A+P. Quite similar in behavior to a CGN. > > The only conclusion is that A+P should never be deployed in the presence of > fragmentation - not that it should drop fragments, nor that we should > consider deprecating fragmentation to address that flaw. The problem A+P solves is to allow IPv4 to continue to grow outside it’s boundaries. The solution keeps session state close to the edge, so much more architecturally clean than CGN. If you can solve this problem in a better way please go ahead. Cheers Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, > I am not ignoring them; I'm claiming that they all have the same inherent > deployment and implementation limitations. > > Just because operators/vendors "want" to do otherwise does not make it > possible. There was IETF consensus behind those documents (A+P). >>> >>> You mean the *experimental* RFCs that describe an approach that doesn't >>> update RFC791? (i.e., RFC6364?) Or something else? >> >> The protocol specifications of A+P are all standards track. >> RFC7596, RFC7597, RFC7599. >> > Thanks for the refs. Note that none of those update RFCs 791 or 1122, so if > frag breaks them, then it's their error. I wouldn’t be surprised if there were disagreements about that interpretation of “updates”. > It also looks like (at first glance at least) these devices work only when > there isn't multipath between the back and front side. The A+P routers are stateless and do support multipath. Including traffic does not need to be symmetric. That’s the main selling point for A+P, that you don’t need per flow state in the core of the network. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, >>> I am not ignoring them; I'm claiming that they all have the same inherent >>> deployment and implementation limitations. >>> >>> Just because operators/vendors "want" to do otherwise does not make it >>> possible. >> >> There was IETF consensus behind those documents (A+P). > > You mean the *experimental* RFCs that describe an approach that doesn't > update RFC791? (i.e., RFC6364?) Or something else? The protocol specifications of A+P are all standards track. RFC7596, RFC7597, RFC7599. >> In the _new_ IPv4 Internet architecture the IPv4 header looks like this: >> >> 0 1 2 3 >> 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 >>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >>|.0x45 |Type of Service| Total Length | >>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >>| Identification|Flags| Fragment Offset| >>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >>| Time to Live | 6|17 | Header Checksum | >>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >>| Source Address | >>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >>|Destination Address| >>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >>| Source Port | Destination Port | >>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >> >> If the the ascii art comes through. >> > A+P didn't update 791. There is no *new* IP header. > > The diagram above is a combination of IP - without options, notably - and > only two specific transports. It isn't an IPsec'd packet, a tunneled packet, > or any other transport. The Internet is not merely TCP and UDP over IP with > no options. For the public IPv4 Internet it is. (Sure there is some support for ICMP as well). >> In contrast to NAT the address and port fields are not rewritten. Only used >> for forwarding. > > And there may be limits to where that kind of approach can be deployed. The > jury is still out - this is experimental. There’s plenty of room for architectural purity in IPv6. Unfortunately there isn’t that luxury in IPv4 any more. Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, > I am not ignoring them; I’m claiming that they all have the same inherent > deployment and implementation limitations. > > Just because operators/vendors “want” to do otherwise does not make it > possible. There was IETF consensus behind those documents (A+P). In the _new_ IPv4 Internet architecture the IPv4 header looks like this: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |.0x45 |Type of Service| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification|Flags| Fragment Offset| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live | 6|17 | Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Destination Address| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ If the the ascii art comes through. In contrast to NAT the address and port fields are not rewritten. Only used for forwarding. The source port and destination port fields are shared between L3 and L4. With an out-of-band provisioning protocol informing the end system which ports are available for applications. Ole___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
But only if you continue to ignore that there are other IPv4 sharing mechanisms than NAT. Ole > On 1 Aug 2018, at 16:11, Joe Touch wrote: > > We all understand that many current NAT devices and their deployments are not > compatible with IP fragmentation (v4 or v6). > > That leaves us with two options: >1. change IP, but that leaves us with problems for which we have no > solution (encrypted payloads, other DPI devices that look further in, etc.) >2. change NATs and how they’re deployed (to require reassembly or its > equivalent before processing, to not be deployed except where they can act as > the host they proxy for) > > Both cost money and will have an impact. > > #2 involves changing less devices AND has the benefit that we know it will > work. > > I see no good reason to continue to try #1 in the meantime. > > Joe ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Tom, > How is this story going to be different for IPv6? How do we ensure that > non-conformant implementation for IPv4 isn't just carried over so that > fragmentation, alternative protocols, and extension headers are viable on the > IPv6 Internet? I don’t think the IPv4 implementations are non-conformant. (With regards to the implications of A+P on the IPv4 architecture). For IPv6 one would fear that the same pressures that has led to IPv4 ossification applies. Well, what can we do? Apart from crypto, ensure that popular applications use the features, so they cannot be shut down? Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, >> The need for fragmentation cannot be completely >> eliminated and we do need it to work. Devices that do things to >> prevent correct operation still need to be fixed, and it would be >> productive for the draft to include statements on how some of the >> sub-problems problems can be fixed (like using flow label for ECMP >> instead of ports). > > On that we agree. That’s my key concern - how to do this in a way that > doesn’t effectively kill off IP fragmentation for the rest of us. For IPv4 it’s an unfortunate side-effect of the fact that we are out of IPv4 addresses. As we continue to increase the ratio of users per IPv4 address, IPv4 fragmentation, or any protocol that isn’t TCP or UDP are not going to work well in the public IPv4 Internet. IPv4 Internet Architecture is evolving as a consequence of address run-out. I think we’ve pretty much explored the solution space for IPv4 sharing mechanisms, so I think you just have to accept this new and unimproved (sic) IPv4 Internet architecture. Cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, > My model describes the rules under which translation devices can operate > correctly and predictably in the Internet model. > > There are only a few alternatives for devices not explained by either model: > 1- the Internet and my model are incomplete > in that case, you’re welcome to provide one for the new device > 2- the Internet and/or my model are incorrect > in that case, you’re welcome to explain why > 3- the device should be considered incorrect and itself corrected > > Un-doing fragmentation at IP is an attempt to jump to a solution for #1 > without explaining WHY, other than “we need to do this to fix the Internet to > support these new devices”. > > I don’t think we should break known models to adapt to devices whose behavior > might never be correctly accommodated. > >> Take A+P (RFC6346), and it's instantiations through e.g. MAP-E (RFC7597). >> That's essentially normal longest match forwarding on addresses and ports. > > So? Any device that sources packets with addresses it owns IS an endpoint on > the Internet. Nothing changes based on how it translates those devices to the > private side. Could you please read those documents and explain how A+P fits in your model? Note an A+P router does not translate, it forwards based on address and port. And as a normal router those addresses (and ports) are not identifying interfaces on the router, but on some end-system further away. Best regards, Ole signature.asc Description: Message signed with OpenPGP ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, >> However much money you throw at it, you can't reassemble fragments >> travelling on different paths, nor can you trivially make network layer >> reassembly not be an attack vector on those boxes. > > Agreed, but here’s the other point: > > Any device that inspects L4 content can do so ONLY as a proxy for the > destination endpoint. > > I.e., I know vendors WANT to sell devices they say can be deployed anywhere > in the network, and operators believe that, but it’s wrong. > > Basically, if you’re not at a place in the network where you represent that > endpoint, you have no business acting as that endpoint - “full stop”. I understand you want it to fit in your model, but it doesn't. Take A+P (RFC6346), and it's instantiations through e.g. MAP-E (RFC7597). That's essentially normal longest match forwarding on addresses and ports. With regards to your point about reassembly at higher layers, crypto is the answer to that. Ole signature.asc Description: Message signed with OpenPGP ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Tom, > On 28 Jul 2018, at 20:48, Tom Herbert wrote: > > On Sat, Jul 28, 2018 at 11:24 AM, Ole Troan wrote: >>> Here’s the thing about fragmentation: >>> >>> 1. all links have a maximum packet size >>> 2. all tunneling/encapsulation/layering increases payload size >>> >>> 1+2 implies there is always the need for fragmentation at some layer: >> >> 1 implies that. >> There is enough head room designed in 1 to accommodate 2. >> > Ole, > > I'm not sure I follow what you're saying here. Ethernet MTU, the most > common value, is 1500 bytes. There's no reference to headroom for > that. If you're referring to the idea of artificially lowering MTUs to > account for potential overhead introduced in encapsulation that can be > done. However to avoid fragmentation _entirely_ one would need to > determine the maximum possible overhead ever added in encapsulation(s) > (plural in case of nested encapsulations). In a sprawling and dynamic > network that has different sub-domains and simultaneously uses > different encapsulation protocols, determining that specific magic > number might be infeasible. There is also the problem that some 0.01% > corner case of encapsulation might need extra large 100s of bytes of > overhead. Lowering the MTU for everyone just to avoid fragmentation > for that case is a poor tradeoff-- it's better to fragment for that > case. I am talking of the headroom of 1500-1280 designed into IPv6. For restricted domains you can increase the headroom by increasing the link MTU. And I am not saying there aren’t corner cases where fragmentation couldn’t be useful for tunnels. Cheers Ole > >>> >>> 3. fragmentation always splits info across packets >>> >>> And there’s something important about layering: >>> >>> 4. layering intends to isolate the behavior of one layer from another, >>> such that >>> it will always be impossible for an upper layer to know exactly what >>> is going on below, >>> i.e., to determine that limiting size across an entire path of >>> possibly virtual tunnels >>> >>> The next two are where we get into trouble: >>> >>> 5. network devices increasingly WANT to inspect contents beyond the >>> layer at which they are intended to operate >> >> not that network devices have an intent in themselves, but yes, it seems >> like network operators want to inspect content or are forced into it because >> of the necessity of IPv4 address sharing. >> >>> 6. inspecting contents ultimately means reassembly, at some level >> >> _some_ content inspection would require that, but I don't think you can make >> that the general rule. >> e.g. a NAT or an L4 ACL only needs access to the L4 header. >> >>> Which brings us to the punchline: >>> >>> 7. but network device vendors want to save money, so they don’t want >>> to reassemble at any layer >> >> We'd all wish it to be that simple. Can you substantiate that claim? >> You can easily make the speculation that customers don't want to pay what it >> costs to be able to do reassembly at terabit speeds... >> Or accept that it's technically hard. >> >> The implementations of e.g. NATs, IPv4 address sharing implementations I'm >> aware of do flavours of network layer reassembly. >> However much money you throw at it, you can't reassemble fragments >> travelling on different paths, nor can you trivially make network layer >> reassembly not be an attack vector on those boxes. >> >>> >>> So I agree, IP fragmentation has its flaws - but those flaws are created >>> not only because it leaves out the transport port numbers, but also because >>> DPI and NAT devices don’t reassemble. And they don’t because it’s cheaper >>> to sell devices that say they run at 1 Gbps (e.g.) that don’t bother to >>> reassemble. >> >> I don't agree with your conclusion. >> NATs extend the network layer to include the L4 ports. NAT implementations >> of course do reassemble. >> >>> I.e., it will never matter what layering we add to fix this - GRE, GUE, >>> Aero, etc. - ultimately, we’re doomed to need fragmentation support down to >>> IP exactly because: >>> >>> a. #1-4 mean we need frag/reassembly at any tunnel ingress >>> b. vendors want to sell #5 at a price that is too low for them to >>> support #6 (i.e., point #7) >> >>> >>> So pushing this to another layer
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
> Here’s the thing about fragmentation: > > 1. all links have a maximum packet size > 2. all tunneling/encapsulation/layering increases payload size > > 1+2 implies there is always the need for fragmentation at some layer: 1 implies that. There is enough head room designed in 1 to accommodate 2. > > 3. fragmentation always splits info across packets > > And there’s something important about layering: > > 4. layering intends to isolate the behavior of one layer from another, > such that > it will always be impossible for an upper layer to know exactly what is > going on below, > i.e., to determine that limiting size across an entire path of possibly > virtual tunnels > > The next two are where we get into trouble: > > 5. network devices increasingly WANT to inspect contents beyond the > layer at which they are intended to operate not that network devices have an intent in themselves, but yes, it seems like network operators want to inspect content or are forced into it because of the necessity of IPv4 address sharing. > 6. inspecting contents ultimately means reassembly, at some level _some_ content inspection would require that, but I don't think you can make that the general rule. e.g. a NAT or an L4 ACL only needs access to the L4 header. > Which brings us to the punchline: > > 7. but network device vendors want to save money, so they don’t want to > reassemble at any layer We'd all wish it to be that simple. Can you substantiate that claim? You can easily make the speculation that customers don't want to pay what it costs to be able to do reassembly at terabit speeds... Or accept that it's technically hard. The implementations of e.g. NATs, IPv4 address sharing implementations I'm aware of do flavours of network layer reassembly. However much money you throw at it, you can't reassemble fragments travelling on different paths, nor can you trivially make network layer reassembly not be an attack vector on those boxes. > > So I agree, IP fragmentation has its flaws - but those flaws are created not > only because it leaves out the transport port numbers, but also because DPI > and NAT devices don’t reassemble. And they don’t because it’s cheaper to sell > devices that say they run at 1 Gbps (e.g.) that don’t bother to reassemble. I don't agree with your conclusion. NATs extend the network layer to include the L4 ports. NAT implementations of course do reassemble. > I.e., it will never matter what layering we add to fix this - GRE, GUE, Aero, > etc. - ultimately, we’re doomed to need fragmentation support down to IP > exactly because: > > a. #1-4 mean we need frag/reassembly at any tunnel ingress > b. vendors want to sell #5 at a price that is too low for them to > support #6 (i.e., point #7) > > So pushing this to another layer will never solve it. What will solve it will > only be a compliance requirement for #6 - which could be done right now, and > has to be done for ANY solution to work. For IPv4 address sharing specifically removing network layer fragmentation would be a solution. > NOTE: even rewriting EVERY application won’t fix this, nor will deploying a > new layer at any level. For some type of content inspection that would require reassembling the whole application context. But that's quite different from IPv4 address sharing, which we have unfortunately made an integral part of the Internet architecture. > And yes, I do intend to add this to draft-ietf-tunnels, so it can be referred > to elsewhere. Ole signature.asc Description: Message signed with OpenPGP ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
> On 27 Jul 2018, at 22:51, Fernando Gont wrote: > >> On 07/27/2018 10:28 PM, Ole Troan wrote: >> >> >>> On 27 Jul 2018, at 22:12, Brian E Carpenter >>> wrote: >>> >>> Fragmentation, (PL)PMTUD, extension headers, and innovative >>> L4 protocols are very possibly not viable on the open Internet. >>> At least, we can't assume that they will work on arbitrary paths. >>> Sad but apparently true. >> >> Hasn’t this been discussed ad infinitum in the ossification work? >> If you want to generalize, nothing is guaranteed to work across an arbitrary >> path in the Internet. >> >> So what? This is part of a tussle and it would be making a self fulfilling >> prophecy for us to take all policy based filtering or other brokenness into >> consideration when designing protocols. > > I see your point. However, how do you engineer something that "works" if > you ignore all brokenness etc.? > > We do normally engineer protocols taking this things into account. e.g. > > * PLPMTUD is a response to ICMP filtering > * ECN had a backup mechanism that would switch to non-ECN for cases > where e.g. firewalls were complaining about previously-unspecified bits > * Quic is most likely implemented over UDP to be able to survive NATs > and firewalls > > > Yes, and one hand is not nice to have to account for all types of > brokenness and filtering. OTOH, it would make any sense to enigneer a > protocol that only works on paper. To bring this back to the draft in question. Each type of “brokenness” needs to be considered on its own merits. E.g brokenness as in network operator blocks all traffic from 1.0.0.0 is different from a poorly designed protocol. While fragmentation has issues because of network policy, the combination of PMTUD and fragmentation also exposes weaknesses in the design, which is something we should fix. Cheers Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
> On 27 Jul 2018, at 22:12, Brian E Carpenter > wrote: > > Fragmentation, (PL)PMTUD, extension headers, and innovative > L4 protocols are very possibly not viable on the open Internet. > At least, we can't assume that they will work on arbitrary paths. > Sad but apparently true. Hasn’t this been discussed ad infinitum in the ossification work? If you want to generalize, nothing is guaranteed to work across an arbitrary path in the Internet. So what? This is part of a tussle and it would be making a self fulfilling prophecy for us to take all policy based filtering or other brokenness into consideration when designing protocols. Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
> On 27 Jul 2018, at 19:20, Tom Herbert wrote: > > Right, but I still think that we should be more clear about the root > origin of problems and blunt in requesting that non-conformant > implementations get fixed. Barring bugs, implementations work the way they do because customers have required them to have features that are used to parse deep into packets. IPv6 extension headers where to some extent designed to make it hard to parse in hardware. Not that, that helped. Yes, we agree on the root cause. But since these are legitimate features (from the perspective of whomever is operating the middlebox) this isn’t something that can be fixed by decree or by claiming it’s a bug. The tool we have in fighting ossification, is crypto. Cheers Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] WG Adoption Call: IP Fragmentation Considered Fragile
Joe, > I still think it would be useful for this doc to describe how tunnels > interact with fragmentation (per draft-ietf-intarea-tunnels), which seems to > be something I’ve noted several times before. There's nothing intrinsic in tunnels that require network layer fragmentation. The current IP in IP / GRE etc tunnel implementations do, but in principle we could design tunnel fragmentation/reassembly above the network layer. > I’m also still not thrilled with the title I’d be happier with “IP > fragmentation still not supported per requirements”, and I’d have to see > where this goes with final recommendations. If we were to conclude that IP fragmentation is done at the wrong layer... Cheers, Ole signature.asc Description: Message signed with OpenPGP ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] draft-bonica-intarea-frag-fragile-01
Joe, > Agreed but note that draft tunnels will update that RFC in some important > ways. With other concerns than those raised in e.g. 4459 and 7597? Unfortunately there are cases where there are no other choice than to do fragmentation/reassembly on tunnel endpoints, but still the recommendation holds. It is so problematic, that it is strongly recommended to engineer the network to avoid that happening. Cheers, Ole signature.asc Description: Message signed with OpenPGP ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] [v6ops] Limiting the size of the IPv6 header chain (draft-ietf-6man-oversized-header-chain)
Joe, an IPv6 router compliant with RFC2460 does not inspect the header chain. That cannot be true; there are headers after IPv6 but before fragmentation that are hop-by-hop. with the exception of the HBH header, correct. I got tired of writing that each time I was repeating myself. the HBH is an issue to itself. expect those packets to be severely rate limited. cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] [v6ops] Limiting the size of the IPv6 header chain (draft-ietf-6man-oversized-header-chain)
Fernando, However, anything that says if the chain is X, then drop is broken, period. At some point, if you want to play IPv6 router, you need to earn the title. an IPv6 router compliant with RFC2460 does not inspect the header chain. I'm not aware of any router that drop packets with extension headers. I'm aware of network operators applying filtering policy that results in packets with extension headers being dropped. IIRC, someone reported that Cisco ASA drop v6 packets with extension headers by default. I think you'll find that the Cisco ASA isn't marketed as a router either, it is a firewall. just to be clear I'm not against the IETF documenting e.g. in a BCP, what the longest expected header chain should be. Well, that seems more std track than bcp to me. I don't think we should encode in standard a number that is based on the ability of current hardware. nor do I think we have any standard documents describing the behaviour of filtering routers / firewalls / middleboxes in this respect. cheers, Ole ___ Int-area mailing list Int-area@ietf.org https://www.ietf.org/mailman/listinfo/int-area
Re: [Int-area] Review of draft-narten-ipv6-3177bis-48boundary-05
Thanks, Tony. Let me comment on one point in your review. On Aug 20, 2010, at 11:47 AM, Tony Li wrote: 5) The draft misses the opportunity to call for work in v6 renumbering. The fact of the matter is that sooner or later, sites will need to renumber. Even given adequate address space, there are other compelling events (e.g., corporate acquisitions) that drive renumbering. There's much work to do here. If we make the assumption that renumbering WILL be easy (and make it come to pass), then it's reasonable to argue that renumbering into a larger prefix is easy and thus we can be more conservative in initial site addressing. When I sat down to write what is now RFC 4192, I was really scratching my head. Given that an IPv6 (or for that matter, an IPv4) interface can take two or more prefixes, it seemed to me that there was an obvious procedure for renumbering a network: 1) start with a working network that you don't like the address plan of 2) design a new address plan using a different set of numbers 3) configure the network equipment to use the new plan in addition to the old 4) test the new plan, fixing whatever needs to be fixed --- you now have two working networks running on the same infrastructure --- but you are only using one, the old one 5) tell the hosts and their applications to use the new address plan 6) verify that the hosts and applications in fact all work using the new plan --- you now have two working networks running on the same infrastructure --- and you are actively using both of them 7) stop advertising services using the old address plan --- you now have two working networks running on the same infrastructure --- but you are only using one, the new one 8) do what you like with the old plan Several of those points obviously imply waiting periods - the fact that you removed a resource record in step 7 doesn't mean you're ready for step 8, for example. I then went to the operational community, inside and outside Cisco, and said OK, I already know I'm insane. What I need to understand is WHY I'm insane. I got an education, and much of what I learned wound up explicitly called out in the document. The thing that makes renumbering hard has nothing to do with the procedures for renumbering. It has to do with places where people type in numeric IP addresses, whether in router configurations like interface addresses and route maps or in applications that Just Know that the address of some system is 192.0.2.1 or 2001:db8::12. Web pages that refer to other servers by address instead of by name, SIP referrals, FTP (which tops it all by having a different passive mode command and behavior for IPv6 than it has for IPv4), and so on. To be really honest, I have concluded that every time we further idiot-proof the world, the world makes better idiots. I'm all for improving our ability to renumber, but I'm not sure that's something the IETF can solve technically. Vendors can help, by providing configuration options that associate names with numbers in one common location and then enables the administrator to use the names in configuration files. But even those have issues. Consider this one: You have a router configured: ! ipv6 unicast-routing ipv6 general-prefix EXAMPLE 2001:0DB8:0:0::/48 ! interface foo 1 ipv6 address EXAMPLE 0:0:0:0::/64 eui-64 ipv6 enable ! interface foo 2 ipv6 address EXAMPLE 0:0:0:1::/64 eui-64 ipv6 enable ! interface foo 3 ipv6 address EXAMPLE 0:0:0:2::/64 eui-64 ipv6 enable ! Now, someone decides to renumber the network, and replaces the general-prefix using ipv6 general-prefix EXAMPLE 2001:0DB8:1:0::/48 What happens? The network stops working for a period of time, at least through that router; depending on the placement of the router, the outage may prevent access to other routers that happen to be beyond it, and certainly disrupts the operations of hosts on the networks it is attached to. Why? Because the existing routing depended on the old prefix, and in replacing the configuration outright it disrupted the existing routing before the new prefix was stable in the network. What should they have done? They should have configured ! ipv6 general-prefix EXAMPLE2 2001:0DB8:1:0::/48 ! interface foo 1 ipv6 address EXAMPLE2 0:0:0:0::/64 eui-64 ! interface foo 2 ipv6 address EXAMPLE2 0:0:0:1::/64 eui-64 ! interface foo 3 ipv6 address EXAMPLE2 0:0:0:2::/64 eui-64 ! waited and tested, and at some later time when the new prefix and old prefixes provably both worked for everyone concerned, applied no ipv6 general-prefix EXAMPLE 2001:0DB8:0:0::/48 ! interface foo 1 no ipv6 address EXAMPLE 0:0:0:0::/64 eui-64 ! interface foo 2 no ipv6 address EXAMPLE 0:0:0:1::/64 eui-64 ! interface
