So this is about payload flow forwarding. Will a payload flow be forwarded to any CPU core, or must be designated to one of cores? Lucy
From: Lizhong Jin [mailto:[email protected]] Sent: Tuesday, September 24, 2013 9:27 AM To: Lucy yong Cc: [email protected]; [email protected] Subject: RE: [nvo3] LAG/ECMP load-balancing problems facing overlay networks On Sep 24, 2013 10:12 PM, "Lucy yong" <[email protected]<mailto:[email protected]>> wrote: > > Lizhong, > > > > Thank you very much for the detail explanation. > > > > The explanation further confirms that using gre-in-udp for NVGRE will > simplify NIC LB implementation. It requires no change on the flow table > structure. [Lizhong] there maybe some misunderstanding here. The key of flow table is the customer payload 4 tuple, not tunnel header 4 tuple. I don't think the discussion confirms anything. Regards Lizhong > > > > Thanks, > > Lucy > > > > From: Lizhong Jin [mailto:[email protected]<mailto:[email protected]>] > Sent: Tuesday, September 24, 2013 8:59 AM > To: Lucy yong > Cc: [email protected]<mailto:[email protected]>; > [email protected]<mailto:[email protected]> > > Subject: Re: [nvo3] LAG/ECMP load-balancing problems facing overlay networks > > > > > > ---------- Forwarded message ---------- > From: Lucy yong <[email protected]<mailto:[email protected]>> > To: Jeff Wheeler <[email protected]<mailto:[email protected]>> > Cc: "[email protected]<mailto:[email protected]>" > <[email protected]<mailto:[email protected]>> > Date: Mon, 23 Sep 2013 17:06:43 +0000 > Subject: Re: [nvo3] LAG/ECMP load-balancing problems facing overlay networks > Jeff, > > I agree that supporting LAG/ECMP at the NIC->host interface is the issue for > both VXLAN/NVGRE. However if NVGRE uses gre-in-udp, it will make future NIC > enhancement for this capability much simpler since both encapsulations use > udp src port for the flow entropy that is used by LB. Why do you think that > inner payload inspection is necessary for ECMP LB? > > [Lizhong] the NIC LB is quite different with the router/switch. The NIC LB is > called flow steering, and it does not rely on a hash value to steer traffic, > but rely on a flow table to steer traffic to a specific CPU core. Usually the > key of flow table is the 4 tuple of payload, src/dst IP add and UDP/TCP port. > That's why the NIC always has to parse payload header. What I have to point > out is, how to steer the VXLAN/NVGRE traffic is still under development, and > maybe different with the one I described. Hope the explanation helps. > > Regards > Lizhong > > > > Thank, > Lucy > > > > -----Original Message----- > From: [email protected]<mailto:[email protected]> > [mailto:[email protected]<mailto:[email protected]>] On Behalf Of > Jeff Wheeler > Sent: Sunday, September 22, 2013 1:13 PM > Cc: [email protected]<mailto:[email protected]> > Subject: [nvo3] LAG/ECMP load-balancing problems facing overlay networks > > On Sat, Sep 21, 2013 at 11:57 AM, Lucy yong > <[email protected]<mailto:[email protected]>> wrote: > > * Requirement: For performance reasons, multipath over LAG and ECMP > > paths SHOULD be supported. > > > > VXLAN supports common five tuple based LB. NVGRE requests LB to use > > GRE header, which is not commonly supported by underlying IP network. > > I'd like to note that it is becoming increasingly common for the underlying > L3 or L2 network to support load-balancing based on GRE inner-header fields. > In fact, this can be largely relied on for high-bandwidth GRE tunnels over > the public Internet (!) today, and within many datacenter networks. > > Where this is not so reliable is in the NIC->Host interface. This is an > issue facing NVGRE, and in some circumstance, VXLAN also. > > For example, the most popular 10GE server NIC chipsets will deliver all NVGRE > packets to a host DMA ring buffer based on the GRE outer-header only. One > example is the Intel 82599, which everyone must be familiar with to have an > informed opinion on vRouter-related topics. Even if the NIC is configured > with several DMA rings, and the vRouter is able to use different CPU cores to > service those rings for distribution of the work, all NVGRE traffic (having > same outer-header) will arrive at only one DMA ring, and other CPU cores may > not be utilized. > > If vRouters are used to support network-heavy applications, NIC vendors must > be encouraged to support additional header inspection which may then be used > for load-balancing across host DMA rings. One would hope same vendors will > eventually implement hardware offload of VXLAN/NVGRE entirely, and these two > requirements share a common underlying need for the NIC to understand the > VXLAN/NVGRE encapsulation. In other words, if a NIC vendor plans to support > deeper header inspection for load-balancing, they have already done some of > the work needed to support hardware offload. > > This list is very focused on procedural items and support in the networks; > but nearly zero attention is paid to NIC->Host interface issues. For overlay > technologies to deliver acceptable performance for network-heavy workloads, > NIC->Host interface must be more intelligent than it is today. > Next-generation NICs must implement new capabilities. It is therefore useful > to consider NICs in any discussion of load-balancing across the datacenter > network, if for no other reason than to foster greater understanding of this > challenge. > -- > Jeff S Wheeler <[email protected]<mailto:[email protected]>> > Sr Network Operator / Innovative Network Concepts > _______________________________________________ > nvo3 mailing list > [email protected]<mailto:[email protected]> > https://www.ietf.org/mailman/listinfo/nvo3 > > > _______________________________________________ > nvo3 mailing list > [email protected]<mailto:[email protected]> > https://www.ietf.org/mailman/listinfo/nvo3 > > >
_______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
