Ivan, See in-line... > -----Original Message----- > From: [email protected] [mailto:[email protected]] On Behalf Of > Ivan Pepelnjak > Sent: Tuesday, September 04, 2012 12:11 PM > To: Balus, Florin Stelian (Florin) > Cc: [email protected]; 'Somesh Gupta' > Subject: Re: [nvo3] Support for multi-homed NVEs > > Here's (very approximately) one way to make it work (assuming L2 S+D > matching in hypervisor vSwitch, with multiple entries resulting in > packet replication). Pretty easy to implement with OpenFlow (more so if > you have OF 1.1+ with multiple lookup tables). > [FB>] Before jumping in the solution space, for NVE resiliency to an external network (NVE1, NVE2 gateways connected on their access side to GW1, GW2 in an external domain) it might be useful to have a look at previous IETF work on this - see http://datatracker.ietf.org/wg/l2vpn/, BGP MH and EVPN drafts.
> A) for every known destination MAC, install (*,D) entry pointing to > destination NVE IP address [FB>] You are just talking about the ingress NVE... must consider also the L2 forwarding in the egress NVE towards the external network (say NVE1,2->GW1,2). Then the other direction GW1/GW2 -> NVE1/NVE2. > B) for multicast MAC addresses, install (*,MC) entries pointing to all > NVEs in the same VN [FB>] This is not enough - if packets are flooded to NVE1 and NVE2 in my example, duplication/loops may occur - see the inclusive mcast and split-horizon procedures described in EVPN draft. > C) for every source MAC (= VM), select one of the NVO3 GWs and install > (S,*) entry pointing to that GW. > > No need to know external MAC or IP addresses; just use the source-MAC- > based "default route" toward one of the NVO3 gateways. > > Nicira does this on L2, Midokura does it on L3. What's wrong with the > shape of this wheel? ;) [FB>] difficult to discuss a solution based on email text... if you feel strongly about it you/them should submit a NVO3 proposal on this gateway resiliency... I believe though there is more to it than what you listed as A,B,C. Florin > > Ivan > > On 9/4/12 7:31 PM, Balus, Florin Stelian (Florin) wrote: > > We need to talk about it: the wheel you refer to below might not work > > on all the roads... :) > > > > For example when you go to external non-NVO3 domains you do not know > all the external MAC/IP addresses so you can't program them from a > centralized controller in the egress direction from NVO3. Also the non- > NVO3 gateways are not controlled by the NVO3 control plane so you can't > program their FIBs from the controller either. Broadcast and Multicast > handling in general might require special consideration as well. > > > > A number of mechanisms can be considered to address the external > network use case - see BGP MH for VPLS and/or BGP EVPN work in IETF ... > > It looks like we need to add access multi-homing in the framework and > follow up on use cases in data and control plane requirements so that > NVO3 can do a proper solution gap analysis at the end. > > > >> -----Original Message----- > >> From: Ivan Pepelnjak [mailto:[email protected]] > >> Sent: Tuesday, September 04, 2012 9:58 AM > >> To: Balus, Florin Stelian (Florin) > >> Cc: 'Somesh Gupta'; [email protected] > >> Subject: Re: [nvo3] Support for multi-homed NVEs > >> > >> Ah, that other can of worms ;) Mine was simpler. > >> > >> On the underlay side, we might decide that NVEs have a single IP > >> address or multiple IP addresses (like some NVGRE load balancing > >> proposals). If we decide NVEs have a single IP address (potential > per > >> virtual network segment), then the rest is implementation details > >> (and we're back to MLAG/SMLT land for true redundancy). > Alternatively > >> we might implement the option of having multiple IP addresses per > >> NVE, and the NVEs might use the IP-address-per-link option (thus no > >> need for L2 or MLAG at all). > >> > >> On the overlay side, the real problem (as you stated) is the multi- > >> homing of NVO3-to-legacy gateways. I don't see any other need for > >> overlay NVE multihoming. > >> > >> BTW, Nicira has nicely solved the NVO3 gateway multihoming - the > >> whole > >> NVO3 network works exactly like VMware's vSwitch: split horizon > >> bridging (thus no forwarding loops through NVO3), with every VM MAC > >> address being dynamically assigned to one of the gateways, which > also > >> solves the return path issues (dynamic MAC learning in legacy > network > >> takes care of that). Maybe we should just use the wheel that has > >> already been invented? > >> > >> Kind regards, > >> Ivan > >> > >> On 9/4/12 6:45 PM, Balus, Florin Stelian (Florin) wrote: > >>> I understand the discussion below is about the NVE multi-homing > >> towards the IP core, on the tunnel side. > >>> We did not focus in the framework draft on the core redundancy as > in > >> our opinion there was no need to standardize anything here. There > are > >> no differences from what is available today in regular IP networks: > >> if NVEs are multi-homed directly to the next IP router, regular > >> routing will take care of it. If there is Ethernet switching in > >> between NVE and the next IP hop, L2 resiliency mechanisms need to be > >> employed. From what I read below it looks more of an implementation > >> discussion than a standardization requirement. Am I right? > >>> By Multi-homed NVEs one can also understand a set of NVEs > >>> multi-homed > >> on the access side to other devices. That is a discussion we need to > >> have in my opinion. An use case example: NVO3 network - NVE GWs > >> multi- homed to external non-NVO3 networks. Handoff can be VLANs, > >> VPLS PWs, or BGP EVPN labels... > >>> I think the latter is worth discussing although there are some > >> mechanisms and some standardization initiatives in place already. > >>> > >>>> -----Original Message----- > >>>> From: [email protected] [mailto:[email protected]] On > >>>> Behalf Of Ivan Pepelnjak > >>>> Sent: Friday, August 31, 2012 12:23 AM > >>>> To: 'Somesh Gupta'; [email protected] > >>>> Subject: Re: [nvo3] Support for multi-homed NVEs > >>>> > >>>> This is definitely an interesting can of worms ;) > >>>> > >>>> While I don't think we should go down the path of IP-A/IP-B > >>>> networks similar to some other DC technology, we will face the > >>>> reality of > >> some > >>>> NVE elements (hypervisor soft switches) not being underlay IP > >> routers. > >>>> We could either: > >>>> > >>>> (A) ignore the issue and expect the network designer to solve it > >>>> using any one of the existing NIC teaming/MLAG kludges while > >>>> retaining a single encapsulation IP address per NVE; > >>>> > >>>> (B) provide support for multiple encapsulation addresses per NVE > so > >> a > >>>> multi-homed NVE could have one IP address per physical interface > >>>> and send and receive nvo3-encapsulated frames using more than one > >> address. > >>>> Option (A) is the easy way out similar to existing MPLS/VPN > >>>> behavior and would fit well with existing DC deployments. It would > >>>> also > >> retain > >>>> all the server-to-ToR multihoming complexity. > >>>> > >>>> Option (B) would reduce the complexity of the underlay DC network > >>>> (which would become a simple L3 network with single-homed IP > >>>> addresses), but we'd have to deal with a bunch of additional > >> problems > >>>> (peer IP address liveliness check). > >>>> > >>>> Just speculating ... > >>>> Ivan > >>>> > >>>>> -----Original Message----- > >>>>> From: [email protected] [mailto:[email protected]] On > >> Behalf > >>>>> Of Somesh Gupta > >>>>> Sent: Friday, August 31, 2012 6:58 AM > >>>>> To: [email protected] > >>>>> Subject: [nvo3] Support for multi-homed NVEs > >>>>> > >>>>> I did not see any mention of multi-homed NVEs in > >>>>> draft-lasserre-nvo3- framework-03.txt. NVEs are connected > together > >>>>> by an L3 network - does that mean only one? > >>>>> Can it be multi-homes to two L3 networks? > >>>>> > >>>>> Somesh > >>>>> _______________________________________________ > >>>>> nvo3 mailing list > >>>>> [email protected] > >>>>> https://www.ietf.org/mailman/listinfo/nvo3 > >>>> _______________________________________________ > >>>> nvo3 mailing list > >>>> [email protected] > >>>> https://www.ietf.org/mailman/listinfo/nvo3 > _______________________________________________ > nvo3 mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/nvo3 _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
