> > ---------- Forwarded message ---------- > From: Jeff Wheeler <[email protected]> > To: > Cc: "[email protected]" <[email protected]> > Date: Sun, 22 Sep 2013 14:13:00 -0400 > Subject: [nvo3] LAG/ECMP load-balancing problems facing overlay networks > On Sat, Sep 21, 2013 at 11:57 AM, Lucy yong <[email protected]> wrote: > > ยท Requirement: For performance reasons, multipath over LAG and > ECMP > > paths SHOULD be supported. > > > > VXLAN supports common five tuple based LB. NVGRE requests LB to use GRE > > header, which is not commonly supported by underlying IP network. > > I'd like to note that it is becoming increasingly common for the > underlying L3 or L2 network to support load-balancing based on GRE > inner-header fields. In fact, this can be largely relied on for > high-bandwidth GRE tunnels over the public Internet (!) today, and > within many datacenter networks. > > Where this is not so reliable is in the NIC->Host interface. This is > an issue facing NVGRE, and in some circumstance, VXLAN also. > > For example, the most popular 10GE server NIC chipsets will deliver > all NVGRE packets to a host DMA ring buffer based on the GRE > outer-header only. One example is the Intel 82599, which everyone > must be familiar with to have an informed opinion on vRouter-related > topics. Even if the NIC is configured with several DMA rings, and the > vRouter is able to use different CPU cores to service those rings for > distribution of the work, all NVGRE traffic (having same outer-header) > will arrive at only one DMA ring, and other CPU cores may not be > utilized. > > If vRouters are used to support network-heavy applications, NIC > vendors must be encouraged to support additional header inspection > which may then be used for load-balancing across host DMA rings. One > would hope same vendors will eventually implement hardware offload of > VXLAN/NVGRE entirely, and these two requirements share a common > underlying need for the NIC to understand the VXLAN/NVGRE > encapsulation. In other words, if a NIC vendor plans to support > deeper header inspection for load-balancing, they have already done > some of the work needed to support hardware offload. > > This list is very focused on procedural items and support in the > networks; but nearly zero attention is paid to NIC->Host interface > issues. For overlay technologies to deliver acceptable performance > for network-heavy workloads, NIC->Host interface must be more > intelligent than it is today. Next-generation NICs must implement new > capabilities. It is therefore useful to consider NICs in any > discussion of load-balancing across the datacenter network, if for no > other reason than to foster greater understanding of this challenge. > [Lizhong] agree with above point. NIC plays a key role for server performance now. Currently only STT could perfect support LB without NIC update. I am not proposing STT to NVO3, I mean using TCP-Like encapsulation does adapt the capability of current NIC. The fragmented UDP frame is also a potential threat for LB of VXLAN. For some NIC, even the non-fragmented UDP frame could not support LB. So from the legacy NIC point of view, we could not say UDP has better LB than GRE.
Regards Lizhong > -- > Jeff S Wheeler <[email protected]> > Sr Network Operator / Innovative Network Concepts > > > _______________________________________________ > nvo3 mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/nvo3 > >
_______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
