Hi Ilango, Sorry for the delayed response.
On Thu, Dec 12, 2019 at 08:00:04PM +0000, Ganga, Ilango S wrote: > Hello Benjamin, > > Thanks for your review and comments. Please see below for our responses to > some of your comments, enclosed in <Response> </Response>. Let us know if you > are satisfied with this resolution. > We will send responses to your other comments separately. > > Thanks, > Ilango Ganga > Geneve Editor > > From: Benjamin Kaduk via Datatracker <[email protected]> > Sent: Wednesday, December 4, 2019 4:37 PM > To: The IESG <[email protected]> > Cc: [email protected]; Matthew Bocci <[email protected]>; > [email protected]; [email protected]; [email protected] > Subject: Benjamin Kaduk's Discuss on draft-ietf-nvo3-geneve-14: (with DISCUSS > and COMMENT) > > Benjamin Kaduk has entered the following ballot position for > draft-ietf-nvo3-geneve-14: Discuss > > 1. Security and transit devices comments: > > Benjamin> “to what extent the Geneve architecture includes support for > middleboxes that inspect (but do not modify!) the Geneve header and inner > payload” > > IG> <Response> 1. Looks like you are interpreting middlebox as a transit > device, but middle box has a broader meaning including service functions > (appliances). So we clarify that the transit devices are forwarding elements > like switches and routers (see section 1.2 for definition) . We reiterate > that only NVEs can generate and terminate Geneve headers and transit devices > can only interpret Geneve headers and that this is optional. A transit device > not interpreting or not able to interpret options can forward the packets > like any other UDP packet. So it does not affect the forwarding of Geneve > packets end to end. An example use case of a transit device (like a > switch/router) is it may interpret information in Geneve headers to perform > ECMP forwarding. If a transit device is not able interpret it will still > forward based on outer UDP header information. > </Response> I'm not sure whether I was consciously interpreting "middlebox" as a transit device specifically, but I'm happy to hear that we're envisioning a strict separation between "transit device" that functions analogously to a switch or a router and "middlebox" as service functions or appliances that are principal entities and participants in the protocol. The discussion here (and some clarifications by Martin on the telechat) do a lot to help reassure me that we are not setting ourselves up for failure. I will try to make some textual suggestions that would help other readers avoid the confusion/concerns that I had. > Benjamin> “to what extent the Geneve architecture is intended to be > applicable to scenarios where (end-to-end per-tunnel) underlay > confidentiality protection is necessary” > > IG> <Response> 2. NVE to NVE end to end encryption (underlay > confidentiality) is prevalent in multi-tenant data centers use cases (see > section 6.1). In these cases a transit device will still forward Geneve like > any other UDP/IP packets. Hence does not affect the forwarding operation of a > transit device. > </Response> Ok. > Benjamin> "Interposing advanced middleboxes" and "service interposition" are > conceived as possible uses for Geneve metadata in Sections 1 and 2.2 as a > consideration for why structured tagging is needed on the data plane and not > just the control plane, which to me suggests that such usage is considered a > first-class use case for Geneve. > > IG> <Response> 3. In the “service interposition” use case noted above the > reference middle box is a network service function, this is not a transit > device. The service would implement a NVE function to generate and terminate > Geneve headers. Hence this is consistent with Geneve specifications. If data > confidentiality is required by the data center operator this is NVE to NVE > encryption as expected by the Geneve/NVO3 architecture. > </Response> I think we should clarify in the text that these NVE functions are terminating and generating Geneve headers, akin to a "back to back user agent" in the SIP world (e.g., RFC 7092); even just "interposing advanced middleboxes that terminate and reencapsulate Geneve traffic" would go a long way. In Section 2.2 we could use "terminating/reencapsulating service interposition". I thought there was another place in the text that discussed this general sort of thing but couldn't find it by searching; I'll try to read through the -15 when it appears to see if I notice anything then. > Benjamin> “Section 6.1.1 discusses encryption for traffic traversing > untrusted links between geographically separated data centers (though perhaps > in this case an encrypted tunnel would be used just for that untrusted > transit and leaving the in-datacenter traffic visible to middleboxes), but > Section 6.1 discusses cases where the tenant may expect the service provider > to provide confidentiality as part of the service. Would this be above or > below the Geneve encapsulation? > ” > IG> <Response> 4. In the scenario described in 6.1.1 (geographically > separated data centers) the Geneve packets will be transported over an > encrypted service (e.g. a VPN service). > <Response 5>. In the scenario described in 6.1 (multi-tenancy) encryption > may be used either at the Geneve payload level or at the IP/UDP level in > which case the Geneve header will not be visible to transit devices. > </Response> Okay. (In either case, if an interposing service is terminating/reencapsulating Geneve, then the encryption will be appropriately targetted anyway.) > Benjamin> "The consideration from Section 6.1 that the provider of the > underlay and the provider of the overlay may not be the same could be taken > to imply that the overlay provider itself wants (cryptographic) protection > from the underlay provider. I don't have a clear picture of how these > considerations interact. (I also note that, since DTLS is mentioned, DTLS > 1.3 is going the way of TLS 1.3 and not defining any authentication-only > ciphersuites, so if authentication-only service is desired, DTLS may not be > the way of the future, leaving IPsec AH as the leading candidate.)" > > IG> <Response> 6. We expect that the overlay network provider will provide > the cryptographic protection with the entire payload being encrypted. Okay. I don't know whether it makes sense to try to convey this expectation in the document as well, or leave it unspecified. > </Response> > > 2. UDP Checksum with IPv6 comment: > > Benjamin> Section 4.3.1 > 2. If Geneve is used with zero UDP checksum over IPv6 then such > tunnel endpoint implementation MUST meet all the requirements > specified in section 4 of [RFC6936] and requirements 1 as > specified in section 5 of [RFC6936]. > > "This seems to implicitly be saying that the other numbered requirements in > Section 5 of RFC 6936 can be ignored, which is updating the behavior of a > standards-track document. We need to either be explicit about the update or > justify why (the rest of) that applicability statement is not applicable > here. If, as the paragraph following the enumerated list says, the > requirements specified in RFC 6936 continue to apply in full, why do we need > to call out a MUST-level requirement here?" > > IG> <Response> 7. Requirement 1 is a consideration for Geneve implementers, > which is why it is highlighted. The others are design considerations for the > protocol itself, which have already been incorporated or are not applicable > to Geneve. It is not implying that the other requirements should be ignored. > We have tried to indicate that with the text stating that all requirements > continue to apply. This text is a resolution to TSVART early review (David > Black). The text is very similar to RFC 8086 (and RFC 7510). > </Response> Thanks for helping me understand the history and precedent here (Martin also helped with this). I'd suggest rewording slightly to clarify that requirement 1 of Section 5 of RFC 6936 is mentioned due to its particular relevance (as opposed to its exclusive relevance), but I do not consider this point to be at the DISCUSS-level anymore. Thanks, Ben > </End of Responses> > > -----Original Message----- > From: Benjamin Kaduk via Datatracker <[email protected]> > Sent: Wednesday, December 4, 2019 4:37 PM > To: The IESG <[email protected]> > Cc: [email protected]; Matthew Bocci <[email protected]>; > [email protected]; [email protected]; [email protected] > Subject: Benjamin Kaduk's Discuss on draft-ietf-nvo3-geneve-14: (with DISCUSS > and COMMENT) > > Benjamin Kaduk has entered the following ballot position for > draft-ietf-nvo3-geneve-14: Discuss > > When responding, please keep the subject line intact and reply to all email > addresses included in the To and CC lines. (Feel free to cut this > introductory paragraph, however.) > > > Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html > for more information about IESG DISCUSS and COMMENT positions. > > > The document, along with other ballot positions, can be found here: > https://datatracker.ietf.org/doc/draft-ietf-nvo3-geneve/ > > > > ---------------------------------------------------------------------- > DISCUSS: > ---------------------------------------------------------------------- > > This first point is a "discuss discuss" for which I'd like to get a sense of > what the rest of the IESG feels. I've read the discussion at > https://mailarchive.ietf.org/arch/msg/last-call/ywRKREnxWAlunHR7MSaTM4ScsDs > but I'm left with a similar sense of uncertainty that Daniel has as to > whether the question is fully resolved. Specifically, "the question" > that I have in mind is to what extent the Geneve architecture includes > support for middleboxes that inspect (but do not modify!) the Geneve header > and inner payload, to what extent the Geneve architecture is intended to be > applicable to scenarios where (end-to-end per-tunnel) underlay > confidentiality protection is necessary, and whether those requirements are > both strong enough to be deemed an internal inconsistency of > requirements/applicability. "Interposing advanced middleboxes" and "service > interposition" are conceived as possible uses for Geneve metadata in Sections > 1 and 2.2 as a consideration for why structured tagging is needed on the data > plane and not just the control plane, which to me suggests that such usage is > considered a first-class use case for Geneve. Section 6.1.1 discusses > encryption for traffic traversing untrusted links between geographically > separated data centers (though perhaps in this case an encrypted tunnel would > be used just for that untrusted transit and leaving the in-datacenter traffic > visible to middleboxes), but Section 6.1 discusses cases where the tenant may > expect the service provider to provide confidentiality as part of the > service. Would this be above or below the Geneve encapsulation? > Might some customers insist on one or the other? The consideration from > Section 6.1 that the provider of the underlay and the provider of the overlay > may not be the same could be taken to imply that the overlay provider itself > wants (cryptographic) protection from the underlay provider. I don't have a > clear picture of how these considerations interact. (I also note that, since > DTLS is mentioned, DTLS 1.3 is going the way of TLS 1.3 and not defining any > authentication-only ciphersuites, so if authentication-only service is > desired, DTLS may not be the way of the future, leaving IPsec AH as the > leading candidate.) > > Some other section-by-section discuss-level points follow, mostly > self-contained/localized issues. > > Section 3.5.1 > > o Some options may be defined in such a way that the position in the > option list is significant. Options MUST NOT be changed by > transit devices. > > o An option SHOULD NOT be dependent upon any other option in the > packet, i.e., options can be processed independently of one > another. [...] > > As was already noted, I don't see how these two requirements are > self-consistent. > > size. A particular option is specified to have either a fixed > length, which is constant, or a variable length, which may change > over time or for different use cases. This property is part of the > definition of the option and conveyed by the 'Type'. For fixed > > This text is written as if this specification is going to specify further > substructure for the "Type", with respect to certain types that have fixed > length and others that may vary. Otherwise the property would be attached to > the option value and not the type value, in my understanding. With the > current way the registry is laid out it seems like we need to explicitly say > that the entity allocating the option class value needs to specify the > interpretation of the 'type' field when used with that option class. > > Section 4.3.1 > > 2. If Geneve is used with zero UDP checksum over IPv6 then such > tunnel endpoint implementation MUST meet all the requirements > specified in section 4 of [RFC6936] and requirements 1 as > specified in section 5 of [RFC6936]. > > This seems to implicitly be saying that the other numbered requirements in > Section 5 of RFC 6936 can be ignored, which is updating the behavior of a > standards-track document. We need to either be explicit about the update or > justify why (the rest of) that applicability statement is not applicable > here. If, as the paragraph following the enumerated list says, the > requirements specified in RFC 6936 continue to apply in full, why do we need > to call out a MUST-level requirement here? > > 4. The Geneve tunnel endpoint that encapsulates the tunnel MAY use > different IPv6 source addresses for each Geneve tunnel that uses > Zero UDP checksum mode in order to strengthen the decapsulator's > check of the IPv6 source address (i.e the same IPv6 source > address is not to be used with more than one IPv6 destination > address, irrespective of whether that destination address is a > unicast or multicast address). When this is not possible, it is > RECOMMENDED to use each source address for as few Geneve tunnels > that use zero UDP checksum as is feasible. > > This functionality is not usable without some mechanism to signal from > encapsulator to decapsulator that it is in use. > > The requirement to check the source IPv6 address in addition to the > destination IPv6 address, [...] > > I do not see this specified as a requirement, only a MAY-level suggestion. > > Section 4.6 > > o When performing LSO, a NIC MUST replicate the entire Geneve header > and all options, including those unknown to the device, onto each > resulting segment. However, a given option definition may > override this rule and specify different behavior in supporting > devices. [...] > > This second sentence makes the MUST in the first no longer a MUST. > > > ---------------------------------------------------------------------- > COMMENT: > ---------------------------------------------------------------------- > > Section 2.2.1 > > recipient. As new functionality becomes sufficiently well defined to > add to tunnel endpoints, supporting options can be designed using > ordering restrictions and other techniques to ease parsing. > > I'm having trouble parsing the second half of this sentence -- what does > "supporting options" mean as a noun? > > Further, either tunnel endpoints or transit devices MAY use offload > capabilities of NICs such as checksum offload to improve the > performance of Geneve packet processing. The presence of a Geneve > variable length header SHOULD NOT prevent the tunnel endpoints and > transit devices from using such offload capabilities. > > I agree with the directorate reviewer that this implementation guidance is > unenforcable as normative keywords. > > Section 3.1, 3.2 > > If we're going to give concrete values for the IPv4 protocol/IPv6 NextHeader > (17) and destination port (6081), shouldn't we also use the concreve value > for Geneve protocol type (0x6558) that corresponds to the inner ethernet > frame? > > I'd also suggest some visual distinction that the "Variable Length Options" > do in fact have variable length, perhaps using the '~' > character in vertical lines. > Similarly, the original ethernet payload need not be 4-byte-aligned and the > figure could make that more prominent. > > It's a little awkward to expand FCS on second usage, not first usage. > > Section 3.4 > > The critical bit allows hardware implementations the flexibility > to handle options processing in the hardware fastpath or in the > exception (slow) path without the need to process all the options. > For example, a critical option such as secure hash to provide > Geneve header integrity check must be processed by tunnel > endpoints and typically processed in the hardware fastpath. > > I think I'm failing to make a connection between some of these steps. > How does having a critical bit let a header integrity check happen in the > hardware fastpath while deferring other option processing to software? > > Transit devices MUST maintain consistent forwarding behavior > irrespective of the value of 'Opt Len', including ECMP link > selection. These devices SHOULD be able to forward packets > containing options without resorting to a slow path. > > There seem to be two broad aspects in play here. First, requiring > insensitivity to "Opt Len" might be because the value would change as a > packet traverses the network. I think this is forbidden by virtue of transit > devices not being allowed to add/delete options, but please confirm. Second, > this affects the ability of transit devices to look past the geneve header to > the inner ethernet header and payload. Given the substantial discussion > we've had in the broader IETF about IPv6 extension headers and the inability > of hardware to examine such variable-length chains to get to the actual upper > layer protocol (with the result that extension headers are largely unusuable > on substantial portions of the internet), it seems like we might conclude > from this statement that either we expect transit devices to not inspect the > upper-layer content or there's a significant chance that this requirement > will be ignored (possibly just by capping the 'Opt Len' > value that is supported), or both. What makes this setup different from > IPv6 EH such that we expect hardware compliance and a usable deployment? > This is particularly poigniant given that we claim this to be a requirement > on transit devices but allow (in Section 4.5) for endpoints to use profiles > that have a restricted maximum length for the options. > If such profiles are common, the incentive for transit devices to slip and > use the lower maximum length increases. > > Section 3.5 > > The high order bit of the option type indicates that this is a > critical option. If the receiving tunnel endpoint does not > recognize this option and this bit is set then the packet MUST be > dropped. If the 'C' bit (critical bit) is set in any option then > the 'C' bit in the Geneve base header MUST also be set. Transit > devices MUST NOT drop packets on the basis of this bit. The > > nit: since we mention the Geneve header, one might claim that "this bit" > in "MUST NOT drop packets on the basis of this bit" is ambiguous (but since > we said this before for the Geneve header one, I assume we're talking about > the one in the Type field now). > > Section 4.4.1 > > It is strongly RECOMMENDED that Path MTU Discovery ([RFC1191], > [RFC8201]) be used by setting the DF bit in the IP header when Geneve > packets are transmitted over IPv4 (this is the default with IPv6). > > Is it the default or the only specified behavior for IPv6? > > Section 4.4.3 > > outside of the scope of this document. When physical multicast is in > use, the 'C' bit in the Geneve header may be used with groups of > devices with heterogeneous capabilities as each device can interpret > only the options that are significant to it if they are not critical. > > Please double-check this sentence, particularly the "may be used". If the > intent is, as written, to note that the packets with the 'C' bit set might > take paths with heterogenous paths, I suggest being more explicit about the > consequences that the traffic might only be delivered to some but not all > endpoints. > > Section 6 > > untrusted boundaries. In addition, tunnel endpoints should only be > operated in environments controlled by the service provider, such as > the hypervisor itself rather than within a customer VM. > > Can you say a bit more about how this "should only be operated in > environments controlled by the service provider" meshes with the note in > Section 4.1 that "[i]t is intended for use in public or private data center > environments" (specifically the "public data center" portion) and the note in > Section 6.1 that the provider of the overlay may not be the same as the > provider of the underlay? > > Section 6.1.1 > > traversing public networks. Any Geneve overlay data leaving the data > center network beyond the operator's security domain SHOULD be > secured by encryption mechanisms such as IPsec or other VPN > mechanisms to protect the communications between the NVEs when they > are geographically separated over untrusted network links. > > Since we use "mechanisms" in both the IPsec clause and the "other VPN" > clause, the "encryption" does not automatically bind to both clauses from a > grammatical perspective. Given that "VPN" is currently in use for both > encrypted and non-encrypted schemes (much to my chagrin), please clarify that > the other VPN mechanisms also need to provide cryptographic confidentiality > protection. (Replacing "VPN mechanisms" > with "VPN technologies" would probably suffice.) > > Section 6.2 > > network. To prevent such attacks, an NVE MUST NOT propagate Geneve > packets beyond the NVE to tenant systems and SHOULD employ packet > > We also care about not propagating Geneve packets from the tenant systems > past the NVE, right? > > filtering mechanisms so as not to forward unauthorized traffic > between TSs in different tenant networks. > > What does "TS" stand for, here? > > Section 10.2 > > RFCs 1191, 2460 (er, 8200), 6040, and 8201 should be listed as normative > references. > > [ETYPES] The IEEE Registration Authority, "IEEE 802 Numbers", 2013, > <http://www.iana.org/assignments/ieee-802-numbers/ieee- > 802-numbers.xml>. > > Hmm, firefox claims the content of this resource is invalid XML, sigh. > > _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
