Hi Iljitsch, Thanks for your response and offer of the correspondence leading to Fred's proposal - I won't read that just now.
Hopefully some other folks can scrutinise Fred's proposal and mine: http://tools.ietf.org/html/draft-templin-inetmtu-06 http://www.firstpr.com.au/ip/ivip/pmtud-frag/ ITR-ETR schemes are fresh, and definitely need some kind of PMTUD system, so it should be fine to build in special functions at both ends. Thanks for your account of the fragmentation reassembly problems of IPv4. > I don't have any references, but in short, the issue is that you > have a 16 bit ID space with a reassembly timeout of something > like a few minutes. This means you can only send 65536 packets > during that "few minute" window or you'll incorrectly reassemble > fragments from different packets if you lose a fragment. This is > especially problematic if the fragmented packets belong to a > tunnel because in that case the IP source/dest addresses are > always the same. In a fresh system such as an ITR-ETR scheme, perhaps a workaround for this would be to set the maximum reassembly time at the ETR to something very much shorter, such as 1, 2 or 3 seconds? That could be a pain where the ETR is an additional function in a server or router with a pre-existing TCP/IP stack. Then it would probably not be possible to shorten the time just for the ITR to ETR packets. >>> It also costs you lots of CPU and could even allow for CPU >>> exhaustion attacks. > >> Yes, but I think it is better than dropping longish packets >> just because we assume some too low PMTU of 1280 or whatever, >> when in fact, within a second or two, the ITR will probably be >> able to establish that the real PMTU is 1500 or somewhat less. > > Who said anything about preemptively dropping packets? Just send > 1500-byte packets + an outer header with DF set and you'll get a > "too big". After that, you know the path MTU and you can in turn > send too bigs to the source of the original packets. OK - here's the Devil's Advocate view: Firstly, the ITR can't be sure it will get a Packet Too Big (PTB) message - that could be dropped. Secondly, there could be another lower MTU limit beyond whatever sent the first PTB message. Your algorithm, as I understand it, does not involve the ITR fragmenting any packets. All the ITR can do when it gets the PTB message is send a PTB back to the sending host. This really slows things down, since the sending host has to create another shorter packet and the whole process repeats itself. With my IPTM approach, as long as the ITR doesn't yet have a reliable estimate of the PMTU to the ETR (which it takes a few seconds to establish with an explicit probe protocol, not using traffic packets, and not relying on PTB messages), it fragments any packet which is longer (after encapsulation) than the default PMTU, which I suggest be something like 1280 bytes. This only persists for a few seconds while the probing takes place. After that, the ITR has a good idea of the PMTU and then sends a PTB to the sending host when it next gets a packet which would exceed this, once encapsulated. In my scheme, when the sending host does get an PTB message, it gets it nearly instantly from the (typically) nearby ITR, rather than from the ITR in your scheme with a long delay due to the long path the encapsulated traffic packet took from the ITR to some router in the tunnel and the delay of the PTB returning to the ITR. Maybe you don't think much of this in Europe or the USA, but from Australia these delays are significant. The round trip to your Netherlands based mailserver 83.149.65.1 is 162ms from my web server in San Francisco, and 343ms from a well-connected (ISP, not via DSL) server in Australia. So my approach shouldn't involve significant delay in packets, whereas yours does significantly delay the first "long" packet. If there is a second shorter MTU limit, the packet which hits that will be delayed similarly as well. These packets would be lost if the PTB message didn't get back to the ITR. Also, in Ivip, traffic packets from the ITR to the ETR sent with the sending host as the outer source address, not the ITR's address. (So any PTB message would go to the sending host, which would ignore it, since the destination address is of the ETR, not of the destination host.) In Ivip, an ITR couldn't use traffic packets as probes for PMTU to the ETR - at least in a system which relied on PTB messages. Using traffic packets as probes would require some special signalling to the ETR, in another packet or in a more complex header of the tunneled packet, to tell the ETR to send an acknowledgement to the ITR. I think this complexifies the encapsulation system - so I use a completely separate probe protocol, between the ITR and the ETR, which has nothing to do with traffic packets. I would rather add probe packets to the total system load than have the lost effort, delayed delivery and additional complications of using traffic packets as probes. With synthetic probes, the ITR can push its luck as far as it likes with larger packets until one size doesn't generate acknowledgements. This is a very robust way of determining PMTU, without relying on PTB packets - though of course the ITR would use any which it received. If you can only use traffic packets to probe the PMTU, you are restricted to the lengths which result from their length and the encapsulation overhead. My approach is: While PMTU to the ETR is being established: Fragment traffic packets which (after encapsulation) are longer than the default PMTU - even if the DF flag is set, since this is only within the tunnel. This gets the data through, in most cases. Reassembly is in the ETR, so the destination host gets one packet. Once this has occurred, probe the PMTU to the ETR with a specific protocol and artificial probe packets - not with traffic packets. Then the ITR can send a reliable PTB message to the source, in the likely event that it gets another packet which is too long to fit in the tunnel, once encapsulated. The result is that all packets are delivered, except the first one for which the ITR sends back a PTB message. Then the sending host has been reliably informed of the entire PMTU situation and adjusts subsequent packets accordingly. There is no recursing of your algorithm, with delayed delivery in each instance, as would be required if there was first a 1450 MTU limit and then further along the tunnel a 1400 limit (unrealistic examples). If the ETR simply forwards packets to the destination host, and there are PMTU limits between the ETR and the destination host, these are dealt with by the sending host, not the ETR or ITR. If the ETR tunnels to the destination host, then that tunnel mechanism needs to handle the MTU problems - such as by using Sprite. The task of my scheme is purely related to the tunnel between one ITR and one ETR. The same ETR could have multiple destination hosts, each with their own MTU limits. > Yes, this allows for PMTUD black holes, but those are subject to > the "so don't do that and the problem goes away" doctrine. ISPs > generally get this, unlike enterprise people and ignorant > consumers who can't live without their firewalls. I think an ITR-ETR should minimising the degradation of communication as much as possible. I don't think we should let the ITR-ETR scheme frequently drop packets - otherwise end-users will quickly discover that this new class of address space sucks. >> Still, we can predict that there will be such large packets >> early on in many communications. Simply dropping them doesn't >> seem right to me. Dropping them with a too-low PMTU value >> being sent to the sending host would screw up that host's later >> packets, making them shorter than they really need to be. I >> think fragmenting them at first is the best approach. > > If we mandate that *TRs support 1500-byte user traffic without > fragmentation this wouldn't be any issue in practice. But then we wouldn't be able to put ITRs and ETRs in all the places they need to be. The whole system would be much more restricted in scope and would not be deployed as widely or as flexibly. >>>> Later, if more such packets need to be sent, the ITR and >>>> ETR can work on determining the real PMTU. I do this with >>>> probe packets, rather than traffic packets. > >>> Even more overhead... > >> Yes. However I can't see a way of probing the PMTU in any >> other way. ICMP can't be relied upon, and if I tried to use >> only traffic packets, I would have to risk those packets not >> arriving. Instead, IPTM fragments the traffic packets and >> sends its own probe packets. This means there is no fancy >> overhead in traffic packets - they are not intended to be used >> for PMTUD at all. > > I REALLY don't like this: generating singalling traffic when > there is no data traffic is a very bad precedent. However, we > probably need to probe for reachability in some way or another, > if we can do the MTU stuff along with that i may be tolerable. In my proposal, and I think in Fred's, there are no probe packets unless the ITR perceives that there is likely to be further longer packets to handle. So yes, there would be one or more large probe packets. ITRs might evolve some smarts about what length probe packets to try, perhaps based on comparing notes with nearby ITRs who have tunneled to this ETR recently, or by some downloaded "cheatsheet" suggesting good lengths to try, for each of the BPG prefixes in which ETRs have been found. Also, an ITR might be happy with getting a response from an ETR to a relatively long probe packet and leave it at that - rather than send another somewhat longer packet, and wait a few seconds before deciding that was too long. That would lead to longer delays before it could inform the sending host with a PTB message, prolonging the time in which the ITR needs to fragment longer traffic packets. This might lead to a generalised minor underestimate for PMTUs, in addition to the overhead imposed by tunneling. However, that might be preferable to spending a few seconds pushing larger and larger probe packets at the ETR until one size is perceived as never generating a response. Some of these matters are also discussed in the other thread "MTU, jumboframes, ITR & ETR placement, ITR function in hosts" - please see my next message. - Robin -- to unsubscribe send a message to [EMAIL PROTECTED] with the word 'unsubscribe' in a single line as the message text body. archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg
