On 8/28/12 12:21 AM, "Ivan Pepelnjak" <[email protected]> wrote:
>Dimitri, > >We're more in agreement than it might seem. I might have my doubts about >the operational viability of the OpenStack-to-baremetal use case you >described below, but I'm positive someone will try to do that as well. > >In any case, regardless of whether we're considering VMs or bare-metal >servers, in the simplest scenario the server-to-NVE connection is a >point-to-point link, usually without VLAN tagging. Unfortunately not always. If the NVE is the ToR and the server is part of a blade system, the server-to-NVE connection is actually multiplexed. Same holds for "fabric extender" type of architectures. One could require that the NVE must always be one hop away and in a p2p connection, but this would limit options. Also, if you want to consider multi-homed servers to dual NVEs, there are some additional complexities to consider, especially if we are looking for active/active configurations. > >In the VM/hypervisor case, NVE is implemented in the hypervisor soft >switch; in the baremetal server case, it has to be implemented in the >ToR switch. > >It's important to keep in mind the limitations of the ToR switches to >ensure whatever solution we agree upon will be implementable in ToR >switches as well, but it makes absolutely no sense to assume NVE will >not be in the hypervisor (because someone wants to support a customer >having a decade-old VLAN-only hypervisor soft switch). > >As for ToR switch capabilities, Dell has demonstrated NVGRE support and >Arista is right now showing off a hardware VXLAN VTEP prototype, so I >guess it's safe to assume next-generation merchant silicon will support >GRE- and UDP-based encapsulations well before we'll agree on what NVO3 >solution should be. > >Finally, can at least some of us agree that the topology that makes most >sense is a direct P2P link between (VM or bare-metal) server and NVE >using VLAN tagging only when a server participating in multiple L2 CUGs >has interface limitations? Yes, VLAN hand offs are perfectly fine and cover most cases, and I don't think they require a p2p link. Dimitri > >Kind regards, >Ivan > >On 8/27/12 6:55 AM, Stiliadis, Dimitrios (Dimitri) wrote: >> Ivan: >> >> I agree and at the same time disagree with some of the statements >> below. I would like to understand your view. >> >> See inline: >> >> On 8/25/12 8:22 AM, "Ivan Pepelnjak"<[email protected]> wrote: >> >>> On 8/24/12 11:11 PM, Linda Dunbar wrote: >>> [...] >>> >>>> But most, if not all, data centers today don't have the Hypervisors >>>> which can encapsulate the NVo3 defined header. The deployment to all >>>> 100% NVo3 header based servers won't happen overnight. One thing for >>>> sure that you will see data centers with mixed types of servers for >>>> very long time. >>>> >>>> If NVEs are in the ToR, you will see mixed scenario of blade servers, >>>> servers with simple virtual switches, or even IEEE802.1Qbg's VEPA. So >>>> it is necessary for NVo3 to deal with the "L2 Site" defined in this >>>> draft. >>> >>> There are two hypothetical ways of implementing NVO3: existing layer-2 >>> technologies (with well-known scaling properties that prompted the >>> creation of NVO3 working group) or something-over-IP encapsulation. >>> >>> I might be myopic, but from what I see most data centers today (at >>>least >>> based on market shares of individual vendors) don't have ToR switches >>> that would be able to encapsulate MAC frames or IP datagrams in UDP, >>>GRE >>> or MPLS envelopes. I am not familiar enough with the commonly used >>> merchant silicon hardware to understand whether that's a software or >>> hardware limitation. In any case, I wouldn't expect switch vendors to >>> roll out NVO3-like something-over-IP solutions any time soon. >>> >>> On the hypervisor front, VXLAN is shipping for months, NVGRE is >>>included >>> in the next version of Hyper-V and MAC-over-GRE is available (with Open >>> vSwitch) for both KVM and Xen. Open vSwitch is also part of standard >>> Linux kernel distribution and thus available to any other Linux-based >>> hypervisor product. >>> >>> So: all major hypervisors have MAC-over-IP solutions, each one using a >>> proprietary encapsulation because there's no standard way of doing it, >>> and yet we're spending time discussing and documenting the history of >>> evolution of virtual networking. Maybe we should be a bit more >>> forward-looking, acknowledge the world has changed, and come up with a >>> relevant hypervisor-based solution. >> >> Correct, and here is where IETF as a standard body fails. There is no >> easy way (any time soon) for a VXLAN based solution to talk to an NVGRE >> or MAC/GRE, or Cloudstack MAC/GRE or STT (you forgot this one), based >> solution. >> Proprietary approaches that drive enterprises to vendor lock ins. And >> instead >> of trying to address the first problem that is about "interoperability", >> we completely throw it under the rug as "not important". And by the time >> we are done with NVO3, there will be a controller lock in as well, and >>the >> death of interoperability.. If I was on the deployment side of the >> solution, >> that's the number one flexibility I would like to see. I don't want to >>be >> forced to buy all my hypervisors from a given vendor, given that not all >> applications are served equally by all hypervisors (for several $$$ >> reasons >> I might add, that can be related with the licensing options of different >> OSes on top of different hypervisors). >> >>> >>> Furthermore, performing something-in-IP encapsulation in the >>>hypervisors >>> greatly simplifies the data center network, removes the need for >>> bridging (each ToR switch can be a L3 switch) and all associated >>> bridging kludges (including large-scale bridging solutions). Maybe we >>> should remember that "Perfection is achieved, not when there is nothing >>> more to add, but when there is nothing left to take away" along with a >>> few lessons from RFC 3439. >>> >>> I am positive a decade from now we'll see ancient servers still using >>> VLAN-only hypervisor switches (or untagged interfaces), so there might >>> definitely be an need for an NVO3-to-VLAN gateway, but we shouldn't >>> continuously focus our efforts on something that's probably going to be >>> a rare corner case a few years from now. >> >> You are absolutely correct. >> I think that if the gateways were trying to solve the interoperability >>with >> legacy servers problem, then obviously they are doomed since they have a >> limited >> life time as you correctly point out. >> >> But I would argue that the only reason for the gateways is rather >> different, >> and it has to do with the point of separation of trust. I believe that >> people >> have several use cases in mind, where there is no hypervisor involved. >> Some examples: ARM/Low power servers where the unit of computation is >>the >> processor and there is no hypervisor, offers of "bare metal servers as >>as >> service", where the handover is the physical wire and what the server >>puts >> on the wire cannot be trusted, etc. >> >> Another real problem described in the last Openstack conference >> by one of the panelists:"We want test&dev and QA systems to run over VMs >> and production systems to run over the same data center network, but on >> bare metal." >> Obviously, they want to scale bare metal usage the same way as VMs >> depending >> on load etc (i.e. drop down the available test&dev resources when >> production needs it). >> Same separation problems etc exist. >> >> So yes, spending too much time worrying about VMs moving around and >>doing >> encapsulations on ToRs is probably a waste of time. But spending a lot >>of >> time >> to understand interoperability between a hypervisor based environments >>and >> use >> cases such as the above that require gateways, I think is a real world >> problem. >> >> Dimitri >> >>> >>> ... or I may be completely wrong. Wouldn't be the first time. >>> Ivan >>> _______________________________________________ >>> nvo3 mailing list >>> [email protected] >>> https://www.ietf.org/mailman/listinfo/nvo3 >> _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
