Himanshu and I had ever discussed extending our IS-IS VPLS proposal so as to support other data encapsulation formats such as VXLAN and NVGRE, in addition to the existing MAC-in-MPLS-in-IP/GRE/UDP encapsulation formats as described in this draft. As you may have already noticed before, one major concern about the deployment of VXLAN in a multi-tenant cloud data center environment is its requirement for a strong multicast capability in the IP underlying networks. With some IS-IS extensions, the DCVPN membership could be discovered automatically and as a result, the DCVPN solution could easily support ingress replication capability. In this way, DC operators would have enough flexibility to make a tradeoff between states and bandwidth on a per tenant basis according to the particular conditions of each tenant (e.g., the amount of PE/NVE nodes, the volume of BUM traffic and so on). Besides, the special encapsulation format could be signaled automatically between PE/NVE nodes. Finally, in the case where multiple tunnel endpoint addresses are required to facilitate the load-balancing of DC VPN traffic across the IP core networks (e.g., an additional IPsec encapsulation may be required after the VXLAN encapsulation due to security considerations in some cases, or NVGRE may need such capability so as to add more entropy info to the DCVPN traffic between a PE/NVE node pair), IS-IS extension could be used as well to advertise multiple tunnel endpoint addresses in the form of one or more prefixes.
We are considering extending our IS-IS VPLS proposal for the above purposes now, any comments and suggestions are appreciated. Best regards, Xiaohu 发件人: [email protected] [mailto:[email protected]] 代表 Balus, Florin Stelian (Florin) 发送时间: 2012年8月31日 4:20 收件人: Ivan Pepelnjak 抄送: [email protected]; Somesh Gupta 主题: Re: [nvo3] performance limitations with virtual switch as the nvo3 end point Actually this is different: the service encapsulation varies not the tunnel. In MPLS VPNs the MPLS service label is always present. To address the multiple service encapsulations the control plane in charge with service Instance ad, rib and fib must indeed have a mechanism to indicate the type of encap used. On Aug 30, 2012, at 11:47 AM, "Ivan Pepelnjak" <[email protected]<mailto:[email protected]>> wrote: Yes & Yes. It’s like MPLS/VPN where a PE-router can reach some PE routers through an LSP, others through MPLS-over-GRE encapsulation. We will probably need a control-plane mechanism to indicate supported/preferred encapsulation methods for each peer NVE. From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Somesh Gupta Sent: Thursday, August 30, 2012 6:47 PM To: [email protected]<mailto:[email protected]> Subject: Re: [nvo3] performance limitations with virtual switch as the nvo3 end point BTW, should the standard support multiple encapsulation types? And should/can a single L2-CUG support multiple encapsulation types? From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Somesh Gupta Sent: Thursday, August 30, 2012 9:37 AM To: Martin Casado; Ayandeh, Siamack Cc: smith, erik; [email protected]<mailto:[email protected]>; David LE GOFF Subject: Re: [nvo3] performance limitations with virtual switch as the nvo3 end point totally agree about decoupling the encap and the control plane – probably needs some abstraction of mappings to achieve that. From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Martin Casado Sent: Thursday, August 30, 2012 8:48 AM To: Ayandeh, Siamack Cc: David LE GOFF; [email protected]<mailto:[email protected]>; smith, erik Subject: Re: [nvo3] performance limitations with virtual switch as the nvo3 end point For the most part, drop performance impact is similar to TSO today (partial coalescing can be done on the receive side). Here is a relevant snippet from a discussion on this at (http://networkheresy.com/2012/03/04/network-virtualization-encapsulation-and-stateless-tcp-transport-stt/) " the semantics are very similar (to TSO) in that received packets can be batched in consecutive sequences and passed to the guest as legitimate TCP frames (just like TSO today). However, with STT, the outer frame is what is segmented, where with other tunneling protocols presumably it would be the inner TCP frame. There are clear trade-offs between the two approaches. With STT, if the first packets drops, then we’re hosed. On the other hand, segmenting the inner header (with L2) would likely require duplicating the TCP header in each packet which would be less efficient byte-for-byte." Again, it's worth pointing out that STT was designed for use when x86 is on both ends. The header is 32 bit aligned, the spec isn't parsimonious with bit sizes in header fields, and the fields are opaque based on the assumption that they'll be interpreted by software on either side that is evolving relatively quickly. This makes sense in some environments, and not in others. From our (Nicira's) standpoint, using a more flexible encap makes sense when we own both sides of the communication since we are often often evolving our control plane (header bits are useful for all sorts of stuff, datapath state versioning, multi-hop logical topologies, carrying additional information like logical inport, or logical output port, etc.). Also, it is generally only deployed in the datacenter fabric, so abusing TCP isn't a huge issue since no middleboxes should be on route. For deployment environments with middleboxe, GRE is clearly more suitable (and we support that too). Of course, whenever an end point is an ASIC or a third party device we don't control, clearly something like VXLAN or NVGRE is preferable. In general, I think it is a good idea to decouple the control plane and the encap so there is more flexibility to map the right technology to the right deployment environment. .m On 8/30/12 7:25 AM, Ayandeh, Siamack wrote: Hi Erik, Thanks for the post. Do you by any chance have any data on impact of packet loss on STT performance if application is running TCP? Would the application resend the entire segment!? Thanks, Siamack From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of smith, erik Sent: Wednesday, August 29, 2012 6:48 PM To: David LE GOFF; [email protected]<mailto:[email protected]> Subject: Re: [nvo3] performance limitations with virtual switch as the nvo3 end point Hi David, a few months ago we did some basic performance testing with OVS and were pretty happy with the results. For one reason or another we were under the impression that using OVS to encap/decap would limit our total throughput to 4-6 Gbps and this turned out to not be the case. In our configuration, we were able to demonstrate 20 Gbps over a bonded pair of 10GbE NICs using STT for the overlay. Our testing wasn’t exactly scientific but I also found an interesting blog post by Martin Cassado that our limited testing seems to corroborate. I haven’t done any testing with VMware and VXLAN. However, if you’re experiencing limited performance with OVS on <insert your favorite Linux distro here>, I would suggest playing around with Jumbo frames (starting from within the guest) and working your way out to the physical interfaces. For additional information, refer to the following: 1) Martin Cassado’s blog: ( http://networkheresy.com/2012/06/08/the-overhead-of-software-tunneling/ ) 2) I posted something to my blog a bit less detailed (but with diagrams) earlier this week ( http://brasstacksblog.typepad.com/brass-tacks/2012/08/network-virtualization-networkings-21st-century-equivalent-to-the-space-race.html ) Specifically, the final three diagrams.. Erik From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of David LE GOFF Sent: Wednesday, August 29, 2012 9:16 AM To: [email protected]<mailto:[email protected]> Subject: [nvo3] performance limitations with virtual switch as the nvo3 end point Hi Folks, Did anyone experienced some performance limitations in Labs with the virtual switch function as the bottleneck when dealing with network overlays? I mean with the tunnel end point located on the hypervisor (virtual switch), setting up Tagging, QoS, ACL, encryption/decryption, etc. require significant CPUs. I know there is not yet official nvo3 implementation there, though VSphere 5 announced it with VXLAN recently but at any chance if some studies have been done, I would be glad to read those. I know STT has been built to overcome such challenges thanks to the NIC offload capabilities… These studies may also drive the brainstorming about which protocol we may use/build? Thank you! david le goff. _______________________________________________ nvo3 mailing list [email protected]<mailto:[email protected]> https://www.ietf.org/mailman/listinfo/nvo3 -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Martin Casado Nicira Networks, Inc. www.nicira.com<http://www.nicira.com> cell: 650-776-1457 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ _______________________________________________ nvo3 mailing list [email protected]<mailto:[email protected]> https://www.ietf.org/mailman/listinfo/nvo3
_______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
