Erik Nordmark writes: > James Carlson wrote: > > Kacheong Poon writes: > >> For example, suppose we want to use the TCP MD5 option. I > >> think it will work nicely with MDT. But I don't know if there > >> is (or ever will be?) a hardware out there doing LSO which can > >> fill in the MD5 option value. As another example, MDT can > >> support the ECN nonce in RFC 3168 nicely. I doubt that there > >> is a hardware out there doing LSO that can do ECN nonce. > > > > Yep, and, as I mentioned, IPsec is pretty problematic with LSO, but > > "easy" with MDT. > > Do the double-quotes mean that it is really hard, or something different?
Conceptually easy, because MDT allows the upper level software to construct the proper headers, but hard in implementation, because we ended up with lots of squirrelly loops and duplicated code paths. > Today we have to disable LSO and MDT when we add ESP or AH to the > packet. It is not impossible to make all of the ESP, AH and other IPsec > code have special handling for M_MULTIDATA but that by itself might not > buy the MDT performance improvement. For the MDT DMA/IOMMU benefits to > appear the result of applying ESP needs to result in one contiguous > payload instead of N separate encrypted payloads. Thus this would result > in more overhead just to manage the multiple packets in the M_MULTIDATA. If I understand what you're saying, I think the IPsec offload issues are somewhat separate. We shouldn't be using STREAMS to send messages into IPsec at all, so "optimizing" that path by use of MDT doesn't make a lot of sense to me. On the other hand, you'd have to make sure that IPsec has the same kinds of duplicate code paths as in TCP and other places: it has to loop in order to construct the right headers and (for ESP) the separate encrypted data segments as well. The issues are a bit more obvious for new TCP headers and the like. > I'm personally a fan of having a minimal number of instructions in the > data paths and the M_MULTIDATA code doesn't satisfy that - it needs to > manage all the metadata which is extra work compared to a soft LSO > implementation. > > So while LSO has some limitations, it is a more tractable way to > amortize DMA/IOMMU overhead than MDT. LSO looks to the rest of the software like a middlebox in the network, and that buys with it all of the technology change problems as we've had in the past. Every time you make a transport change, you've got to modify that middlebox to do something appropriate with that transport bit, and that costs extra. Or you end up with "orphan" protocols (like IPsec, SCTP, or IPv6) that just aren't supported and always run in the slow path. Both answers are bad. Simply accumulating writes avoids those problems, and that's what I think is slightly attractive about MDT. Not that I'm wild about MDT, given the overhead. For what it's worth, I think optimizing the path down into (and back from) the driver is a better long-term solution. It doesn't require things MDT. And it'll work for everybody, unlike LSO. -- James Carlson, Solaris Networking <[EMAIL PROTECTED]> Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 _______________________________________________ networking-discuss mailing list [email protected]
