Hi Pankaj, Please see my reply in-line below.
Regards, Lucy From: Pankaj Garg [mailto:[email protected]] Sent: Thursday, August 22, 2013 4:18 PM To: Lucy yong; [email protected] Cc: [email protected]; [email protected] Subject: RE: comments on draft-sridharan-virtualization-nvgre-03 Hi Lucy, My comments are inline. Thanks Pankaj From: [email protected] [mailto:[email protected]] On Behalf Of Lucy yong Sent: Tuesday, August 20, 2013 2:49 PM To: [email protected] Cc: [email protected]; [email protected] Subject: [nvo3] comments on draft-sridharan-virtualization-nvgre-03 Hi Authors, I read this new version and have a major concern on the ECMP support. * The new version adds ECMP support section (4.1) and require switches and routers SHOULD provide ECMP on the NVGRE packet using the outer frame fields and entire Key field (32-bit). This means that the solution not only requires the change on the endpoint devices but also the change on the intermediate switches and routers, which put the high bar for the network. Rare switches and routers today support GRE key based load balance. [PG] - This requirement has always been there as without using the key field, there is not enough entropy for ECMP in the network. There are two options for GRE based network virtualization, either use the outer frame fields and the key field or calculate entropy based on the inner packet fields. Many of the newer current generation switches and router silicon supports ECMP hashing for NVGRE packets using outer frame fields and key field. However, the older devices would not be able to do that. [Lucy] There is another option beside these two. That is described in the draft mentioned below, which has no change request on intermediate routers/switches. Many older devices are there now. I don't see the reason to replace them for supporting this feature. In addition, both described options create complex logic to existing LB process. Today switches/routers widely support 5 tuple based LB, which we should leverage instead of adding unnecessary complex to it. * To support GRE Key based load balancing, the hardware has to check the GRE Key presence and the protocol type to avoid a mis-operation on other GRE usages, which is bad. * Using 8 bits in the Key field for flow entropy is not sufficient for all the applications. [PG] - The entropy is defined by 4 fields in NVGRE, outer source IP, outer destination IP, VSID and FlowId. The VSID is the only thing that is static for one virtual network. All other fields can change and that can provide additional entropy needed in the large networks. [Lucy] I mean that one virtual network may contain many and many flows, mapping all the flows into 8 bits flow entropy does not give a good granularity, which may potentially cause uneven load balancing issue. Hashing based load balancing is based on statistical. Without a large amount of flow entropy, it will not work well. * Directly disclosing VSID to underlying network can be a security concern in some cases. [PG] - In theory, it is possible to use IPsec as the transport to protect the GRE header, however then you lose ECMP etc. I am also not sure why this is a concern since the virtual network ID is disclosed in the network in all network virtualization tunneling proposals out there, correct? [Lucy] My point is that the fields that are processed by underlying network (intermediate nodes) should not be the fields belonging to the virtual network. If we want the overlay virtual network is decoupled from the physical infrastructure, we should do that consistently. To let overlay and underlying operate on the same field on packets is bed idea, IMHO. It creates some dependency and restriction for both networks. For example, what if virtual networks span across several WAN networks, and some WAN operator want to create its own flow entropy by rehashing some fields on the packets? Your two options do not give that flexibility. IMHO, it is important to have flow entropy field on packet header for the underlying network and allow the end point to fill the flow entropy there. * The draft further suggests the interim solution for ECMP support (without any hardware upgrade). It is to assign multiple physical addresses (PA, outer address) to NVGRE end point and use policy controlling the choice of PA to be used. This makes operation complex. [PG] - There is some operational overhead, but I believe it is minimum as one can assign multiple source IP to each NVE to gain additional entropy for ECMP. The tunneling operation can select the source IP based on the hash of inner packet headers. [Lucy] I don't believe that is simple to manage. We can have a better solution to support ECMP. Most switches and routers today support 5 tuple based load balance. Five tuple are IP src/dst addr, tcp|udp src/dst ports, and IP protocol type. draft-yong-tsvwg-gre-in-udp-encap-01 proposes the gre-in-udp encapsulation for GRE encapsulated protocols to be tunneled over IP networks where ECMP exists. This solution supports 16 bits flow entropy, does not require any change on intermediate switches and routers, and applies to any GRE encapsulated protocol. It also gives the ingress end point flexibility to generate the flow entropy without explicitly exposing VSID. I highly recommend NVGRE proposal to adopt this method for ECMP support. [PG] - Thanks, I would take a look at the proposal, however I treat this as another tunneling protocol in addition to three existing ones i.e. NVGRE, VxLAN and STT. What are the specific advantages of this over other protocols? Also many of current generation NIC hardware support offloads for the existing three protocols. Wouldn't using GRE-IN-UDP break those offloads? [Lucy] If NIC hardware support VxLAN which use UDP port for the flow entropy, it should be easy to support gre-in-udp encaps as well. Will be great to see chip vendor comment on this. Please take a look at the gre-in-udp encapsulation proposal (below), welcome comment on it. Regards, Lucy Here is the draft. The TSVWG will adopt it as WG draft soon. http://datatracker.ietf.org/doc/draft-yong-tsvwg-gre-in-udp-encap/ Regards, Lucy
_______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
