\> > I thought that we are discussing if it is necessary for NVo3 to consider cross > subnet communications. For the example you give where all applications in the > DC are all in one subnet and those applications don't communicate with peers > in other subnets, I would think they are pretty small DC, which is not really > what NVo3 is targeted for, is it? >
On the contrary. Lots of very large DCs are there for such applications. (most Big Data related data centers are like that). Dimitri > Linda > > > -----Original Message----- > > From: Stiliadis, Dimitrios (Dimitri) [mailto:dimitri.stiliadis@alcatel- > > lucent.com] > > Sent: Monday, July 09, 2012 5:41 PM > > To: Linda Dunbar; Joel M. Halpern > > Cc: Pedro Roque Marques; [email protected]; [email protected]; > > [email protected] > > Subject: RE: [nvo3] inter-CUG traffic [was Re: call for adoption: > > draft-narten-nvo3-overlay-problem-statement-02] > > > > There is no DMZ in such applications. They don't reside on the DMZ. > > > > Applications maintain Tbytes of data in a hadoop cluster and running > > mapreduce > > on them. There is a single application and single tenant on this whole > > cluster. > > mapreduce nodes will send lots of data between the map and reduce > > phase > > and there is no need for any L3 separation in these cases. > > > > This is just one example. We can't possibly make the assumption that > > any > > app in the DC is related to a web-service on the DMZ. > > > > > > Dimitri > > > > > > > > > > > > > > > -----Original Message----- > > > From: Linda Dunbar [mailto:[email protected]] > > > Sent: Monday, July 09, 2012 2:59 PM > > > To: Stiliadis, Dimitrios (Dimitri); Joel M. Halpern > > > Cc: Pedro Roque Marques; [email protected]; [email protected]; > > [email protected] > > > Subject: RE: [nvo3] inter-CUG traffic [was Re: call for adoption: > > draft- > > > narten-nvo3-overlay-problem-statement-02] > > > > > > Dimitrios, > > > > > > Do you mean that all applications in "a large hddop/mapredure > > cluster" are in > > > one subnet? > > > And the applications in "one cluster" never communicate with another > > > "cluster" in the same data center? Really? > > > I would think applications in DMZ would be at least put in a > > different subnet. > > > > > > Linda > > > > > > > -----Original Message----- > > > > From: Stiliadis, Dimitrios (Dimitri) > > [mailto:dimitri.stiliadis@alcatel- > > > > lucent.com] > > > > > What kind of DCs which only have intra subnet communication? > > > > > > > > A large hadoop/mapreduce cluster will not bother about different > > > > subnets, > > > > and the traffic requirements can be huge. Thousands of enterprise > > > > applications > > > > will also do the same, since they assume they are in a protected > > > > environment. > > > > > > > > I agree with Joel's earlier statement : > > > > > > > > > I have seen descriptions of data centers for which it is almost > > > > "all", > > > > > and other data centers for which it is negligible. > > > > > > > > Dimitri > > > > > > > > > > > > > > > > > > Linda > > > > > > -----Original Message----- > > > > > > From: Joel M. Halpern [mailto:[email protected]] > > > > > > Sent: Monday, July 09, 2012 3:00 PM > > > > > > To: Linda Dunbar > > > > > > Cc: Pedro Roque Marques; [email protected]; [email protected]; > > > > > > [email protected] > > > > > > Subject: Re: [nvo3] inter-CUG traffic [was Re: call for > > adoption: > > > > > > draft-narten-nvo3-overlay-problem-statement-02] > > > > > > > > > > > > I have not seen data to suggest "most". Do you have a source > > for > > > > that? > > > > > > I have seen descriptions of data centers for which it is > > almost > > > > "all", > > > > > > and other data centers for which it is negligible. > > > > > > Yours, > > > > > > Joel > > > > > > > > > > > > On 7/9/2012 3:51 PM, Linda Dunbar wrote: > > > > > > > Joel, > > > > > > > > > > > > > > I agree with you that VPN related WG (e.g. L2VPN, L3VPN) may > > not > > > > need > > > > > > to address too much on across VN communications. > > > > > > > > > > > > > > But most traffic in Data Center are across subnets. Given > > NVo3 is > > > > for > > > > > > identifying issues associated with data centers, I think that > > cross > > > > > > Subnet traffic should not be ignored. > > > > > > > > > > > > > > Linda > > > > > > > > > > > > > >> -----Original Message----- > > > > > > >> From: [email protected] [mailto:[email protected]] > > On > > > > Behalf > > > > > > Of > > > > > > >> Joel M. Halpern > > > > > > >> Sent: Friday, June 29, 2012 9:32 PM > > > > > > >> To: Pedro Roque Marques > > > > > > >> Cc: [email protected]; [email protected]; [email protected] > > > > > > >> Subject: Re: [nvo3] inter-CUG traffic [was Re: call for > > adoption: > > > > > > >> draft-narten-nvo3-overlay-problem-statement-02] > > > > > > >> > > > > > > >> I would not be so bold as to insist that all deployments can > > > > safely > > > > > > >> ignore inter-VPN intra-data-center traffic. But there are > > MANY > > > > > > cases > > > > > > >> where that is not an important part of the traffic mix. > > > > > > >> So I was urging that we not mandate optimal inter-subnet > > routing > > > > as > > > > > > >> part > > > > > > >> of the NVO3 requirements. > > > > > > >> I would not want to prohibit it either, as there are > > definitely > > > > > > cases > > > > > > >> where it matters, some along the lines you alude to. > > > > > > >> > > > > > > >> Yours, > > > > > > >> Joel > > > > > > >> > > > > > > >> On 6/29/2012 9:40 PM, Pedro Roque Marques wrote: > > > > > > >>> Joel, > > > > > > >>> A very common model currently is to have a 3 tier app where > > > > each > > > > > > tier > > > > > > >> is in its VLAN. You will find that web-servers for instance > > > > don't > > > > > > >> actually talk much to each other... although they are on the > > > > same > > > > > > VLAN > > > > > > >> 100% of their traffic goes outside VLAN. Very similar story > > > > applies > > > > > > to > > > > > > >> app logic tier. The database tier may have some replication > > > > traffic > > > > > > >> within its VLAN but hopefully than is less than the requests > > > > that it > > > > > > >> serves. > > > > > > >>> > > > > > > >>> There isn't a whole lot of intra-CUG/subnet traffic under > > that > > > > > > >> deployment model. A problem statement that assumes > > (implicitly) > > > > that > > > > > > >> most or a significant part of the traffic stays local to a > > > > > > >> VLAN/subnet/CUG is not a good match for the common 3-tier > > > > > > application > > > > > > >> model. Even if you assume that web and app tiers use a > > > > > > VLAN/subnet/CUG > > > > > > >> per tenant (which really is an application in enterprise) > > the > > > > > > database > > > > > > >> is typically common for a large number of apps/tenants. > > > > > > >>> > > > > > > >>> Pedro. > > > > > > >>> > > > > > > >>> On Jun 29, 2012, at 5:26 PM, Joel M. Halpern wrote: > > > > > > >>> > > > > > > >>>> Depending upon what portion of the traffic needs inter- > > region > > > > > > >> handling (inter vpn, inter-vlan, ...) it is not obvious that > > > > > > "optimal" > > > > > > >> is an important goal. As a general rule, perfect is the > > enemy > > > > of > > > > > > good. > > > > > > >>>> > > > > > > >>>> Yours, > > > > > > >>>> Joel > > > > > > >>>> > > > > > > >>>> On 6/29/2012 7:54 PM, [email protected] wrote: > > > > > > >>>>> Pedro, > > > > > > >>>>> > > > > > > >>>>>> Can you please describe an example of how you could set > > up > > > > such > > > > > > >>>>>> straightforward routing, assuming two Hosts belong to > > > > different > > > > > > >> "CUGs" such > > > > > > >>>>>> that these can be randomly spread across the DC ? My > > > > question > > > > > > is > > > > > > >> where is the > > > > > > >>>>>> "gateway", how is it provisioned and how can traffic > > paths > > > > be > > > > > > >> guaranteed to > > > > > > >>>>>> be optimal. > > > > > > >>>>> > > > > > > >>>>> Ok, I see your point - the routing functionality is > > > > > > straightforward > > > > > > >> to move over, > > > > > > >>>>> but ensuring optimal pathing is significantly more work, > > as > > > > noted > > > > > > >> in another one > > > > > > >>>>> of your messages: > > > > > > >>>>> > > > > > > >>>>>> Conceptually, that means that the functionality of the > > > > "gateway" > > > > > > >> should be > > > > > > >>>>>> implemented at the overlay ingress and egress points, > > rather > > > > > > than > > > > > > >> requiring > > > > > > >>>>>> a mid-box. > > > > > > >>>>> > > > > > > >>>>> Thanks, > > > > > > >>>>> --David > > > > > > >>>>> > > > > > > >>>>> > > > > > > >>>>>> -----Original Message----- > > > > > > >>>>>> From: Pedro Roque Marques > > [mailto:[email protected]] > > > > > > >>>>>> Sent: Friday, June 29, 2012 7:38 PM > > > > > > >>>>>> To: Black, David > > > > > > >>>>>> Cc: [email protected]; [email protected] > > > > > > >>>>>> Subject: Re: [nvo3] inter-CUG traffic [was Re: call for > > > > adoption: > > > > > > >> draft- > > > > > > >>>>>> narten-nvo3-overlay-problem-statement-02] > > > > > > >>>>>> > > > > > > >>>>>> > > > > > > >>>>>> On Jun 29, 2012, at 4:02 PM, <[email protected]> wrote: > > > > > > >>>>>> > > > > > > >>>>>>>> There is an underlying assumption in NVO3 that > > isolating > > > > > > tenants > > > > > > >> from > > > > > > >>>>>>>> each other is a key reason to use overlays. If 90% of > > the > > > > > > >> traffic is > > > > > > >>>>>>>> actually between different tenants, it is not > > immediately > > > > > > clear > > > > > > >> to me > > > > > > >>>>>>>> why one has set up a system with a lot of "inter > > tenant" > > > > > > traffic. > > > > > > >> Is > > > > > > >>>>>>>> this is a case we need to focus on optimizing? > > > > > > >>>>>>> > > > > > > >>>>>>> A single tenant may have multiple virtual networks with > > > > routing > > > > > > >> used to > > > > > > >>>>>>> provide/control access among them. The crucial thing > > is to > > > > > > avoid > > > > > > >> assuming > > > > > > >>>>>>> that a tenant or other administrative entity has a > > single > > > > > > virtual > > > > > > >> network > > > > > > >>>>>>> (or CUG in Pedro's email). For example, consider > > moving a > > > > > > >> portion of > > > > > > >>>>>>> a single data center that uses multiple VLANs and > > routers > > > > to > > > > > > >> selectively > > > > > > >>>>>>> connect them into an nvo3 environment - each VLAN gets > > > > turned > > > > > > >> into a virtual > > > > > > >>>>>>> network, and the routers now route among virtual > > networks > > > > > > instead > > > > > > >> of VLANs. > > > > > > >>>>>>> > > > > > > >>>>>>> One of the things that's been pointed out to me in > > private > > > > is > > > > > > >> that the level > > > > > > >>>>>>> of importance that one places on routing across virtual > > > > > > networks > > > > > > >> may depend > > > > > > >>>>>>> on one's background. If one is familiar with VLANs and > > > > views > > > > > > >> nvo3 overlays > > > > > > >>>>>>> providing VLAN-like functionality, IP routing among > > virtual > > > > > > >> networks is a > > > > > > >>>>>>> straightforward application of IP routing among VLANs > > (e.g., > > > > > > the > > > > > > >> previous > > > > > > >>>>>>> mention of L2/L3 IRB functionality that is common in > > data > > > > > > center > > > > > > >> network > > > > > > >>>>>>> switches). > > > > > > >>>>>> > > > > > > >>>>>> Can you please describe an example of how you could set > > up > > > > such > > > > > > >>>>>> straightforward routing, assuming two Hosts belong to > > > > different > > > > > > >> "CUGs" such > > > > > > >>>>>> that these can be randomly spread across the DC ? My > > > > question > > > > > > is > > > > > > >> where is the > > > > > > >>>>>> "gateway", how is it provisioned and how can traffic > > paths > > > > be > > > > > > >> guaranteed to > > > > > > >>>>>> be optimal. > > > > > > >>>>>> > > > > > > >>>>>>> OTOH, if one is familiar with VPNs where access > > among > > > > > > >>>>>>> otherwise-closed groups has to be explicitly configured, > > > > > > >> particularly > > > > > > >>>>>>> L3 VPNs where one cannot look to L2 to help with > > grouping > > > > the > > > > > > end > > > > > > >> systems, > > > > > > >>>>>>> this sort of cross-group access can be a significant > > area > > > > of > > > > > > >> functionality. > > > > > > >>>>>> > > > > > > >>>>>> Considering that in a VPN one can achieve inter-CUG > > traffic > > > > > > >> exchange without > > > > > > >>>>>> an gateway in the middle via policy, it is unclear why > > you > > > > > > suggest > > > > > > >> that "look > > > > > > >>>>>> to L2" would help. > > > > > > >>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>> Thanks, > > > > > > >>>>>>> --David > > > > > > >>>>>>> > > > > > > >>>>>>>> -----Original Message----- > > > > > > >>>>>>>> From: [email protected] [mailto:nvo3- > > [email protected]] > > > > On > > > > > > >> Behalf Of > > > > > > >>>>>> Thomas > > > > > > >>>>>>>> Narten > > > > > > >>>>>>>> Sent: Friday, June 29, 2012 5:56 PM > > > > > > >>>>>>>> To: Pedro Roque Marques > > > > > > >>>>>>>> Cc: [email protected] > > > > > > >>>>>>>> Subject: [nvo3] inter-CUG traffic [was Re: call for > > > > adoption: > > > > > > >> draft-narten- > > > > > > >>>>>>>> nvo3-overlay-problem-statement-02] > > > > > > >>>>>>>> > > > > > > >>>>>>>> Pedro Roque Marques <[email protected]> writes: > > > > > > >>>>>>>> > > > > > > >>>>>>>>> I object to the document on the following points: > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> 3) Does not discuss the requirements for inter-CUG > > > > traffic. > > > > > > >>>>>>>> > > > > > > >>>>>>>> Given that the problem statement is not supposed to be > > the > > > > > > >>>>>>>> requirements document,, what exactly should the > > problem > > > > > > >> statement say > > > > > > >>>>>>>> about this topic? > > > > > > >>>>>>>> > > > > > > >>>>>>>> <[email protected]> writes: > > > > > > >>>>>>>> > > > > > > >>>>>>>>> Inter-VN traffic (what you refer to as inter-CUG > > traffic) > > > > is > > > > > > >> handled > > > > > > >>>>>>>>> by a straightforward application of IP routing to the > > > > inner > > > > > > IP > > > > > > >>>>>>>>> headers; this is similar to the well-understood > > > > application > > > > > > of > > > > > > >> IP > > > > > > >>>>>>>>> routing to forward traffic across VLANs. We should > > talk > > > > > > about > > > > > > >> VRFs > > > > > > >>>>>>>>> as something other than a limitation of current > > > > approaches - > > > > > > >> for > > > > > > >>>>>>>>> VLANs, VRFs (separate instances of routing) are > > > > definitely a > > > > > > >>>>>>>>> feature, and I expect this to carry forward to nvo3 > > VNs. > > > > In > > > > > > >>>>>>>>> addition, we need to make changes to address > > Dimitri's > > > > > > comments > > > > > > >>>>>>>>> about problems with the current VRF text. > > > > > > >>>>>>>> > > > > > > >>>>>>>> Pedro Roque Marques <[email protected]> writes: > > > > > > >>>>>>>> > > > > > > >>>>>>>>> That is where again the differences between different > > > > types > > > > > > of > > > > > > >>>>>>>>> data-centers do play in. If for instance 90% of a VMs > > > > traffic > > > > > > >>>>>>>>> happens to be between the Host OS and a network > > attached > > > > > > >> storage > > > > > > >>>>>>>>> file system run as-a-Service (with the appropriate > > multi- > > > > > > tenent > > > > > > >>>>>>>>> support) then the question of where are the routers > > > > becomes a > > > > > > >> very > > > > > > >>>>>>>>> important issue. In a large scale data-center where > > the > > > > Host > > > > > > VM > > > > > > >> and > > > > > > >>>>>>>>> the CPU that hosts the filesystem block can be > > randomly > > > > > > spread > > > > > > >>>>>>>>> where is the router ? > > > > > > >>>>>>>> > > > > > > >>>>>>>> Where is what router? Are you assuming the Host OS and > > NAS > > > > are > > > > > > >> in the > > > > > > >>>>>>>> different VNs? And hence, traffic has to (at least > > > > > > conceptually) > > > > > > >> exit > > > > > > >>>>>>>> one VN and reenter another whenever there is HostOS - > > NAS > > > > > > >> traffic? > > > > > > >>>>>>>> > > > > > > >>>>>>>>> Is every switch a router ? Does it have all the CUGs > > > > present ? > > > > > > >>>>>>>> > > > > > > >>>>>>>> The underlay can be a mixture of switches and > > routers... > > > > that > > > > > > is > > > > > > >> not > > > > > > >>>>>>>> our concern. So long as the underlay delivers traffic > > > > sourced > > > > > > by > > > > > > >> an > > > > > > >>>>>>>> ingress NVE to the appropriate egress NVE, we are good. > > > > > > >>>>>>>> > > > > > > >>>>>>>> If there are issues with the actual path taken being > > > > > > suboptimal > > > > > > >> in > > > > > > >>>>>>>> some sense, that is an underlay problem to solve, not > > for > > > > the > > > > > > >> overlay. > > > > > > >>>>>>>> > > > > > > >>>>>>>>> In some DC designs the problem to solve is the inter- > > CUG > > > > > > >>>>>>>>> traffic. With L2 headers being totally irrelevant. > > > > > > >>>>>>>> > > > > > > >>>>>>>> There is an underlying assumption in NVO3 that > > isolating > > > > > > tenants > > > > > > >> from > > > > > > >>>>>>>> each other is a key reason to use overlays. If 90% of > > the > > > > > > >> traffic is > > > > > > >>>>>>>> actually between different tenants, it is not > > immediately > > > > > > clear > > > > > > >> to me > > > > > > >>>>>>>> why one has set up a system with a lot of "inter > > tenant" > > > > > > traffic. > > > > > > >> Is > > > > > > >>>>>>>> this is a case we need to focus on optimizing? > > > > > > >>>>>>>> > > > > > > >>>>>>>> But in any case, if one does have inter-VN traffic, > > that > > > > will > > > > > > >> have to > > > > > > >>>>>>>> get funneled through a "gateway" between VNs, at least > > > > > > >> conceptually, I > > > > > > >>>>>>>> would assume that an implementation of overlays would > > > > provide > > > > > > at > > > > > > >> least > > > > > > >>>>>>>> one, and likely more such gateways on each VN. How > > many > > > > and > > > > > > >> where to > > > > > > >>>>>>>> place them will presumably depend on many factors but > > > > would be > > > > > > >> done > > > > > > >>>>>>>> based on traffic patterns and network layout. I would > > not > > > > > > think > > > > > > >> every > > > > > > >>>>>>>> NVE has to provide such functionality. > > > > > > >>>>>>>> > > > > > > >>>>>>>> What do you propose needs saying in the problem > > statement > > > > > > about > > > > > > >> that? > > > > > > >>>>>>>> > > > > > > >>>>>>>> Thomas > > > > > > >>>>>>>> > > > > > > >>>>>>>> _______________________________________________ > > > > > > >>>>>>>> nvo3 mailing list > > > > > > >>>>>>>> [email protected] > > > > > > >>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3 > > > > > > >>>>>>> > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>>>> _______________________________________________ > > > > > > >>>>> nvo3 mailing list > > > > > > >>>>> [email protected] > > > > > > >>>>> https://www.ietf.org/mailman/listinfo/nvo3 > > > > > > >>>>> > > > > > > >>>> > > > > > > >>> > > > > > > >>> > > > > > > >> > > > > > > >> _______________________________________________ > > > > > > >> nvo3 mailing list > > > > > > >> [email protected] > > > > > > >> https://www.ietf.org/mailman/listinfo/nvo3 > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > nvo3 mailing list > > > > > [email protected] > > > > > https://www.ietf.org/mailman/listinfo/nvo3 _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
