Hi, the statement around VM cloning and preservation of IP / MAC for each VM is interesting:
*"The example I am most aware of is supported by VMware's vCloud Director, in which vApps (groups of VMs that work together to provide an application) can be cloned multiple times from a template to deploy multiple instances of the application. A tenant can deploy as many copies of the vApp as they like, and each copy retains its original MAC and IP addresses. The VMs within a vApp communicate over an isolated layer 2 network. A NAT device is used to interface these vApps to a network external to the vApp."* This is a novel concept, however I see large scale cloud environments doing this very differently. I have yet to see an intra-tenant address overlap in DCs, as dynamic environments involving cloning of images typically use DHCP for address assignment to the VM, and service orchestration handles the service IP to VM IP mapping (via NAT). Cloning IP and MAC addresses even for intra-tenant use is setting up issues for the future. Kind regards, Truman On Wed, Jul 11, 2012 at 12:14 AM, Larry Kreeger (kreeger) <[email protected] > wrote: > > > From: Aldrin Isaac <[email protected]> > Date: Tuesday, July 10, 2012 8:06 PM > > To: Cisco Employee <[email protected]> > Cc: "LASSERRE, MARC (MARC)" <[email protected]>, David > Black <[email protected]>, "[email protected]" <[email protected]>, Lucy yong < > [email protected]>, Rekhter Yakov <[email protected]> > Subject: Re: [nvo3] call for adoption: draft-lasserre-nvo3-framework-02 > > Larry, will try again as my attempt to give some background/perspective > before I gave my answer to your question seems to have backfired. See > inline. -- aldrin > > On Jul 9, 2012, at 11:48 PM, Larry Kreeger (kreeger) wrote: > > Hi Aldrin, > > I'm afraid this didn't help give me the answers to my questions – at > least directly > > > I think the problem is a matter of perspective/experience. If the > perspective of NVO3 is through the lens of VLAN/VXLAN/NVGRE/VNET/OTV where > the VNID has global/network/domain-wide scope and is used as-is on the > wire, then your questions are quite reasonable as they are based on the > limitations of these solutions. However if the perspective is through the > lens of IPVPN and E-VPN then the restrictions of one VN-per-VNI and one > VNI-per-VN-per-NVE are not reasonable since IPVPN/EVPN inherently have > support for multiple VN-per-VNI and multiple VNI-per-VN-per-NVE. > > First, you discuss multiple VNIs on the same NVE connected to the same > VN. I did not see this illustrated in your diagram, so I didn't have > questions about it. From your description, you seem to be making an > assumption that there is some kind of firewall policy associated with the > VNI. I don't see the benefit of applying firewall policies to VNIs. It > seems more straight forward to apply them to the VAPs. > > > This is indeed in my illustration. I'm resending it. If you look at > NVE1 and NVE2 you will see that the 2 L2VNI on each of these NVE share a > "DC<n> mesh" L2VN. The illustration depicts both multiple VN per VNI as > well as multiple VNI on a single NVE being members of the same VN. > > > OK, now that you point it out, I see it. I didn't recognize it for what > it was. > > > Regarding features such as filtering implemented at the forwarding table > (such as VACL) -- L2 ACL on standard dot1q switches are usually implemented > with VACL not per-port ACL. > > > Yes, but a VACL would apply to all traffic on the VLAN. If you want > finer grain, then you would move down to a Port ACL. > > But to be clear, VACL is only one possible example of a feature that > can be implemented at the forwarding table. Another thing that might be > implemented at the forwarding table level might be features like PVLAN > (although I prefer that pvlan solution be implemented using overlapping VN > model as shown in my illustration). > > > Regarding the topic of my questions (one VNI connected to multiple > VNs), I am not clear on the answers to my questions based on your example. > Based on your example, it seems like you are assuming that addresses > cannot overlap between two VNs (or at least two that have been associated > with the same VNI). It also seems like you are assuming a VN doesn't > necessarily represent a subnet (I'm thinking of an L2 VN), but perhaps more > like a Closed User Group. If a VN connects VNI connects to multiple VNs > then it is a member of multiple CUGs. Did I interpret that correctly? > > > Overlapping address space is generally an inter-tenant issue, not an > intra-tenant issue. What reasonable example exists where a single tenant > might choose to use the same IP to address different end stations? > > > The example I am most aware of is supported by VMware's vCloud Director, > in which vApps (groups of VMs that work together to provide an application) > can be cloned multiple times from a template to deploy multiple instances > of the application. A tenant can deploy as many copies of the vApp as they > like, and each copy retains its original MAC and IP addresses. The VMs > within a vApp communicate over an isolated layer 2 network. A NAT device > is used to interface these vApps to a network external to the vApp. > > > Generally an L2VNI will be associated to a single subnet. In the cases > where pvlan is used, a single subnet might have multiple CUGs. As you are > more than likely well aware, in pvlan a "community" vlan represents a CUG > yet it's end-station members are generally on the same subnet as > end-station on another community VLAN that shares the same primary vlan. > The members of both community vlan will share a "primary" vlan where > generally the gateway is present. In EVPN this could be implemented by the > DC network operator as multiple overlapping VPN, i.e two mesh VPN for the > two community vlan CUGs and a hub-and-spoke VPN for the primary VLAN. My > illustration depicts this. In this solution approach, a single VNI is a > member of multiple VN where each VN represents a CUG. How would full pvlan > feature be implemented with VXLAN solution? Would it be implemented at the > VN, VNI or VAP level? > > > VXLAN does not support the concept of Private VLANs. It provides the > same services as a plain old VLAN. If I wanted to implement a CUG with > VXLANs, I would use something else (like a zone-based firewall) in > conjunction with it. > > > Hopefully I did a better job with my response this time :} > > > Yes, thank you! …and now that you have, I would question whether this > functionality (Closed User Groups) needs to be built into the overlay > connectivity itself, or can it be achieved independently (e.g. using a > zone-based firewall). > > > > Thanks, Larry > > > > From: Aldrin Isaac <[email protected]> > Date: Saturday, July 7, 2012 8:08 AM > To: Cisco Employee <[email protected]> > Cc: "LASSERRE, MARC (MARC)" <[email protected]>, David > Black <[email protected]>, "[email protected]" <[email protected]>, Lucy yong < > [email protected]>, Rekhter Yakov <[email protected]> > Subject: Re: [nvo3] call for adoption: draft-lasserre-nvo3-framework-02 > > Hi Larry, > > I view the VAP as just a virtualized host-facing network port, and the > VNI as an instance of a forwarding table. > > > Wrt multiple VNI on a single NVE that are members of the same VN: > > Let me try the following example. Let's assume you have 10 TES on a > hypervisor/NVE that need to be on the same VN. If you need a certain > behavior implemented at the forwarding table (say an L2VNI equivalent of a > "VACL") for 5 of those TES and another behavior for the other 5 TES then > you can have two VNI on the same NVE that are members of the same VPN. > > Another better example would be where the first 5 TES additionally need > to be a part of another VN. > > > Wrt a single VNI being a member of more than one VN: > > Consider a network overlay solution where at the center is a pub-sub > system with VNI as clients of this system. The VNI publish routes with one > or more topic-IDs and also subscribe to one or more topic-IDs (and thereby > receive routes that were published with those topic-IDs). Let's assume > each topic-ID can be considered a VN. By publishing and subscribing with a > topic-ID a VNI is effectively participating (becomes a member) in the topic > (VN). > > If a VNI is a member of (subscribes to) two VN (topic-IDs) it will > receive routes from VNI that published routes with both VN (topic-IDs). If > a route is associated to both VN then the VNI (or the big brain in the sky) > has to pick one based on some tie breaker. However there are two cases (1) > the route represents the same destination in which case it doesn't matter. > Also generally in the case of (1) the pub-sub system delivers to the VNI > one route with two topic-ID on it (in case of BGP that would be two Route > Targets). Case (2) is where the route indeed represent different > destinations -- this is a case of operator error. The disambiguation will > have to happen somewhere -- either in the network or in the host. > > Furthermore, in L2VN a "VNI route" would probably be used for > identifying where to send BUM, while unicast MAC routes would be used for > forwarding known unicast. In my illustration, the BUM traffic would be > sent to all VNI represented by "VNI routes" in the ingress VNI regardless > of how many VN the VNI is a member of. > > I'm hoping this email will also address Dave's earlier email. > > Best -- aldrin > > > On Jul 6, 2012, at 7:34 PM, Larry Kreeger (kreeger) wrote: > > Hi Aldrin, > > My interpretation of the model in the framework document is that a VNI > is the intersection of a single VN with a single NVE, with the VAP > determining a connection to a single VNI. Your picture seems to show a > single VAP connecting to a single VNI where two VNs intersect at the same > VNI. Given that, how would the VAP differentiate which VN to associate > ingress packets with? For example, if one of the VMs at the bottom sent an > ARP to the L2VNI, which VN would the broadcast be sent to? Since each VN > is allowed to support overlapping addresses, how would a unicast frame sent > by that VM on the VAP be delivered to the correct overlapping destination > address in the two VNs? > > Thanks, Larry > > From: Aldrin Isaac <[email protected]> > Date: Monday, July 2, 2012 7:29 PM > To: "LASSERRE, MARC (MARC)" <[email protected]>, David > Black <[email protected]> > Cc: "[email protected]" <[email protected]>, Lucy yong <[email protected]>, > Aldrin Isaac <[email protected]> > Subject: Re: [nvo3] call for adoption: draft-lasserre-nvo3-framework-02 > > (resending from my correct mailto email with minor edits) > > > Hi Marc/David, > > I've attached a "proof-of-concept" design to express the points I have > raised. Hopefully a picture is worth a thousand words. It is a PDF file > since I simply don't have the time to draw it as ascii art. > > The illustration tries to capture (1) multiple VN per VNI (and > relatedly, single interface on TES) and (2) VNIF. In this POC the L2-based > subnet comprise of 3 VNs to force VMs to remain in their availability zone > (DC) but allows them to communicate with other subnets or Internet via > gateways in both DC (with preference for local DC). This illustration > could represent a single tenant of a cloud DC provider or infrastructure > owned and operated by an enterprise. > > The illustration also tries to capture a more mobile L3-based VN where > routing information is in the form of host-routes with aggregation of those > routes at the gateways. > > Best -- aldrin > > > > > On Jun 26, 2012, at 11:51 PM, Aldrin Isaac wrote: > > Hi Marc, > > My comments inline, > > > On Jun 26, 2012, at 9:43 AM, LASSERRE, MARC (MARC) wrote: > > Hi Aldrin, > > See my comments below. > > Thanks, > Marc > > ------------------------------ > *From:* Aldrin Isaac [mailto:[email protected]<[email protected]>] > > *Sent:* Tuesday, June 26, 2012 3:24 PM > *To:* LASSERRE, MARC (MARC) > *Cc:* Aldrin Isaac; [email protected]; [email protected]; Lucy yong > *Subject:* Re: [nvo3] call for adoption: draft-lasserre-nvo3-framework-02 > > Hi Marc, > > Thanks for the clarifications. > > Wrt my first question, should we assume that there isn't yet a consensus > on whether a VNI can support more than one type of VN? > > It does not preclude it. I'd like to hear how much interest there is to > support this capability. > > > Actually I'm not convinced mixing types is good. Needed to clarify > where things stood today. > > > Regarding one-to-one mapping of VNI to VN, this implies that an > end-station that needs to be a part of more than one VN will require a > separate interface into each associated VNI. This implies that the > end-station will require a potentially complex routing table (depending on > how aggregateable the address space within a VN is). Unfortunately as an > operator I don't find this an acceptable trade off. Can we please address > the issue of managing multiple interfaces and complex tables in > end-systems? I believe the problem statement needs to capture this. > > This is not any different than with existing VPN technologies. Supporting > multiple interfaces to connect to different subnets and/or nvo3 domains > does not make routing really complex. > Have you had specific problems with past technologies? > > > If you are referring to VRFs and the like, then I don't see host-facing > VNI as equivalent. For one, most PE connect to a CE which generally tend > to be a router, not a host -- the difference here is network versus system > administration. > > Even in cases where the PE connects to an access subnet (as opposed to a > CE router), there are scenarios where the subnet-connecting VRF is a member > of multiple VPNs. For example: a regional extranet DMZ subnet connected to > a VRF which is both a member of a customer-facing regional hub-and-spoke > VPN as well as an internal DMZ mesh. The advantage of this approach is > that the servers do not need to have an interface on a subnet connected to > an internal DMZ VRF which is a member of the internal mesh VPN and yet > another separate interface on a subnet connected to another DMZ VRF which > is a member of the customer-facing regional hub-and-spoke VPN. In > addition, the servers would only need a default route as opposed to the > complex management of static routes on the servers which can become quite > challenging to manage at scale. > > Even in a PE to CE-router scenario, there is also the concept of "shared > VRF" model and "interface VRF" model. In the shared VRF model multiple CE > are connected to the same VRF and each VRF is connected to one VPN. Here > if a CE needs to be connected to another VPN you provision yet another > interface to connect to the associated VRF on the PE, the CE and all the > associated routing configuration. OTOH, with the interface VRF model each > VRF has only one CE connected (sometimes each interface of the CE gets it's > own VRF on the PE). In this model enabling an additional VPN for the CE > only involves importing and exporting an additional RT pair at it's VRF. > > In a DC/overlay setting, a particular scenario where multiple VN per VNI > are involved would be a DMZ subnet with public addressing provided by the > ISP. The DMZ would need to support different CUGs with a common > gateway/firewall to efficiently share the ISP assigned prefix. In this > scenario, it is very common that "private vlans" are used (informational > RFC 5517). This would need to be implemented with at least two VNs per VNI > -- a hub-and-spoke VN with the gateway/firewall as the hub and multiple > "community" VNs where each community VN represents a CUG wherein members > need to communicate with one another. > > Furthermore, in an enterprise setting it is generally expected that the > network bears the network-related complexity. I suppose in outfits where > the same folks deal with both sides (network and host) this may not matter > -- but my understanding is that these folks are a minority. > > > > Best -- aldrin > > On Tuesday, June 26, 2012, LASSERRE, MARC (MARC) wrote: > > Hi Aldrin, > > See my comments below. > > Thanks, > Marc > > > -----Original Message----- > > From: Aldrin Isaac [mailto:[email protected]] > > Sent: Monday, June 25, 2012 8:23 PM > > To: LASSERRE, MARC (MARC) > > Cc: Lucy yong; [email protected]; [email protected] > > Subject: Re: [nvo3] call for adoption: > > draft-lasserre-nvo3-framework-02 > > > > Hi Marc, > > > > Looking for more clarity on the following... > > > > Should we assume that a VNI can only support L2VNs (VNI = > > L2VNI) or L3VNs (VNI = L3VNI) but not both? > > > > draft-mity-nvo3-use-cases describes a "virtual network > > instance interconnecting interface" (VNIF) to interconnect > > VNI of different type. An example of a VNIF would be an IRB > > (integrated routing and bridging) interface that is common on > > "L3 switches". Is this something that draft-lasserre captures? > > While the current draft focuses on L2VNs and L3VNs as distinct services, > no assumptions have been made about the ability to support IRB. > A couple of sentences could be added to highlight this possibility. > > > > > Are there any restriction to the topology of a VN or topology > > created using a set of VNs. I.e. can (1) a VNI be a member > > of multiple VN and (2) can a VN be either hub-and-spoke or > > full-mesh? Does draft-lasserre support (1) and (2)? > > A VN is represented by a VNI created within NVEs that participate in this > VN. Hence there is a one-to-one mapping between a VN and VNI. > > Hub-and-spoke is discussed briefly in this draft. More details are > provided in draft-bl-nvo3-dataplane-requirements. > > > > > Best -- aldrin > > > > > > On Jun 25, 2012, at 1:14 PM, <[email protected]> wrote: > > > > > Extracting a couple of items for additional comment: > > > > > >> 2) section 2.3, why call it NVE service type? Should we call it VN > > >> (or VNI) service type? One NVE may terminate both L2 VNI > > and L3 VNI. > > >> NVE is equivalent to PE in L2VPN/L3VPN. We did not have > > service type > > >> for PE and had service type for VPN. > > >> > > >> [ml] Like with L2 and L3 PEs, there are L2 and L3 NVEs. > > >> [[LY]] Should we consider this as VN service type? I > > understand the > > >> description but have trouble with the title of NVE service type. > > > > > > I think VN service type makes sense, as an NVE could conceivably > > > support both > > > L2 and L3 services for the same end system. OTOH, mixing L2 and L3 > > > service types on the same VN requires that the > > encapsulation indicate > > > the service type (for correct decap processing), and that > > would also be useful to note. > > > > > >> 7) It is important to state that the major difference > > between other > > >> overlay network technology and NVO3 is that > > >> the client edges of the NVO3 network are individual virtualized > > >> hosts and not network sites. > > >> > > >> [ml] NVO3 can cope with both virtualized and non-virtualized hosts. > > >> [[LY]] Yeah, that is more precise. The point is that the > > client edge > > >> is not network sites like CEs in a VPN. > > > > > > Actually the NVE can be in a network node. See Figure 3 and the > > > discussion of "traditional physical server" in the last > > paragraph on > > > p.10, including the statement that the NVE can be in the > > ToR. FWIW, > > > the Storage Systems example in that paragraph is fine, although one > > > can envision storage systems that contain NVEs. > > > > > > Thanks, > > > --David > > > > > >> -----Original Message----- > > >> From: [email protected] [mailto:[email protected]] > > On Behalf > > >> Of Lucy yong > > >> Sent: Monday, June 25, 2012 11:52 AM > > >> To: LASSERRE, MARC (MARC); Benson Schliesser; [email protected] > > >> Cc: Bocci, Matthew (Matthew) > > >> Subject: Re: [nvo3] call for adoption: > > >> draft-lasserre-nvo3-framework-02 > > >> > > >> Marc, > > >> > > >> Please see inline below with [[LY]]. > > >> > > >> Lucy > > >> > > >> -----Original Message----- > > >> From: LASSERRE, MARC (MARC) > > [mailto:[email protected]] > > >> Sent: Monday, June 25, 2012 9:23 AM > > >> To: Lucy yong; Benson Schliesser; [email protected] > > >> Cc: Bocci, Matthew (Matthew) > > >> Subject: RE: [nvo3] call for adoption: > > >> draft-lasserre-nvo3-framework-02 > > >> > > >> Hi Lucy, > > >> > > >> Thanks for your comments. > > >> See my comments below prefixed with [ml]. > > >> > > >> Marc > > >> > > >> -----Original Message----- > > >> From: Lucy yong [mailto:[email protected]] > > >> Sent: Saturday, June 23, 2012 1:11 AM > > >> To: LASSERRE, MARC (MARC); Benson Schliesser; [email protected] > > >> Cc: Bocci, Matthew (Matthew) > > >> Subject: RE: [nvo3] call for adoption: > > >> draft-lasserre-nvo3-framework-02 > > >> > > >> Marc, > > >> > > >> Here are some comments on this draft: > > >> > > >> 1) The VN (or VNI?) in the doc. is an overlay network that is > > >> transported over the tunnels. These tunnels are provided by the > > > > > > > _______________________________________________ > nvo3 mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/nvo3 > >
_______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
