Hello Tom et al., hmm, looking at the original email from Erik I wonder: have we made any progress?
For the "extensions & hardware" discussion: there is obviously no simple truth. And ECMP is a well-defined task, this does not mean much for parsing general TLV. I understood Erik's comment as a "avoid TLVs if we can do it otherwise, e.g. with fixed+flag fields". Which is not far away from your idea to constrain, I think; nevertheless it requires to know today - at least roughly - what extensions you need. For the aspects where you know that you don't know yet I see two options: (1) come up with a generic TLV scheme (2) define a new header when you need it Solution (1) is the obvious choice when options/TLVs are optional. When new fields are mandatory for the functionality then you may effectively end up with (2) - a "header plus mandatory TLV plus TLV sequence rules" is not different from a new header that looks like "old header plus TLV fields". Reading through the various mails and drafts I come up with this first conclusion: - VNI field of at least 24 bit is required (*) - ECMP hashing is needed, either by an explicit field or by offloading it into the UDP source port value - protocol field is needed. Personally I like to think of a "next-header" as this would allow to point to the payload protocol as well as to a TLV. Saves bits in the initial header design. The open question is 8 or 16 bit. - version field. In case of an IP/UDP encapsulation I wonder if another UDP destination port would not do the same (?), again saving bits. - OAM. Proposals so far use a flag. With a protocol or next-header field I wonder if an "OAM" value could not do the same as the OAM flag. Actually it could do more as some may want a "punt, don't forward" OAM while others may have plans for a "punt & forward" OAM. This would be simply two different protocol/next-header values. (*: Tom, you mentioned some VNI hierarchy but I don't think we came to a conclusion how many bits this would need and if the hierarchy requires dataplane support or can be handled by the control plane) > But it would be nice (desirable) if a new protocol (with good support for > extensibility) can be implemented using existing silicon, even if that silicon can't > handle the extensions themselves. That facilitates incremental deployment. agree. Which practically means VxLAN or NVGRE (?). I would still prefer we think this from the requirements before mapping it onto one of the existing schemes. Not that I expect we come up with something fundamentally different (e.g. Geneve also looks "suspiciously" similar to VxLAN ;-) but it may help asking if fields are really required. Regards, Marc On Fri, 7 Nov 2014 09:18:12 -0800, Tom Herbert wrote: > On Thu, Nov 6, 2014 at 10:33 AM, Erik Nordmark <[email protected]> wrote: >> On 11/4/14 4:39 PM, Tom Herbert wrote: >>> >>> On Tue, Nov 4, 2014 at 3:46 PM, Erik Nordmark <[email protected]> wrote: >>>> >>>> On 10/26/14 1:20 PM, Tom Herbert wrote: >>>>> >>>>> On Fri, Oct 24, 2014 at 5:44 PM, Erik Nordmark <[email protected]> wrote: >>>>>> >>>>>> >>>>>> It would be good for the NVO3 WG to have a clear understanding of what >>>>>> data >>>>>> needs to be carried with each encapsulated frame. That helps determine >>>>>> how >>>>>> flexible and extensible the packet format needs to be. >>>>>> The experience with extensibility for protocols that are in the >>>>>> dataplane >>>>>> (be it IPv4 options, IPv6 extension headers, TRILL options, etc) is >>>>>> that >>>>>> they don't tend to get implemented in hardware. And the dataplane >>>>>> protocols >>>>>> tend to have a mixture of hardware and software implementations - which >>>>>> is >>>>>> different than TCP which is mostly software. >>>>> >>>>> I don't believe this is always true. We have verified that at least >>>>> two NICs and one switch chip are capable of parsing any combination of >>>>> keyid, sequence number, and checksum fields in GRE for the purposes of >>>>> calculating a flow hash on the inner header. In fact, we've been able >>>>> to overload the sequence number and checksum fields for our own >>>>> options in lieu of HW not supporting a general L3 extensibility >>>>> mechanism (like IP options). >>>>> >>>> Tom, >>>> >>>> I was referring to the case when the IETF defines some options/extensions >>>> mechanism. Those don't tend to get implemented in hardware. >>>> >>>> Your example is where some existing hardware parser can be reused for >>>> some >>>> other purpose by overloading or redefining existing protocol fields. I've >>>> seen that elsewhere as well. But that doesn't refute the point about lack >>>> of >>>> hardware implementations for options/extensions. >>>> >>> I suspect there are several devices that implement keyid option in GRE >>> and nvgre (for VNID). >> >> But the VNID isn't an optional extension so it is quite natural that >> hardware that implements nvgre would implement it. >> > > Erik, > > The VNID uses the keyid which is optional in the underlying > encapsulation which is GRE. When we deploy nvgre in the network we now > see two formats of GRE with different header lengths. Both of these > formats are already supported in switches for purposes of EMCP hash, > so we didn't need swap out any hardware to get "support" for nvgre. If > you want to say these are two different versions of a protocol, or > just two different protocols, as opposed to one version with and > without an optional extension-- that is fine, but semantically these > descriptions are equivalent. > >> The issue we see is with IETF standardizing some optional option; in >> general >> hardware implementations do not support those. >> >>> In any case, the requirements document probably >>> needs to expound upon what the hardware requirements of the dataplane >>> are, especially if such requirements will potentially place >>> constraints on other requirements (like extensibility). >> >> >> I definitely don't think (re)using existing hardware should trump other >> requirements. >> >> But it would be nice (desirable) if a new protocol (with good support for >> extensibility) can be implemented using existing silicon, even if that >> silicon can't handle the extensions themselves. That facilitates >> incremental >> deployment. >> > Agreed. I think nvgre/GRE demonstrates a possible direction for that. > Purposely limit the number of possible variants of the header, but > allow a means to (sparingly) add new variants. This works well in GRE > since number of fields that can be added is limited, and fields are > always well ordered (for instance with 5 bits, we'd have up to 32 > header combinations which seems reasonable to put into a TCAM for > parsing). > > A constrained extensibility approach precludes open-ended > extensibility that one might get with a rich set of TLVs and vendor > specific options. Proposals like NSH and geneve seem to allow for > that. This also should be considered in the requirements. > > Tom > >> Regards, >> Erik >> > _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
