Hi Kireeti, I guess another dimension is that a VM in NV does not need to communicate with all other VMs in CUG, why NVE need to have all the endpoint locations even hardware is afford. They are difference between host and switch. Will NVE on server behave more like a group of hosts or a switch?
Lucy -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Kireeti Kompella Sent: Wednesday, September 26, 2012 5:01 PM To: Thomas Narten Cc: Kireeti Kompella; [email protected] Subject: Re: [nvo3] Push or pull? Hi Thomas, I'd like to focus on the following couple of points. Note that the goal is not whether push is better than pull; the goal is to find a metric where doing the work to achieve pull is worthwhile. On Sep 26, 2012, at 11:04 , Thomas Narten <[email protected]> wrote: > And what about for very large VNs where there is a lot of VM mobility? > Should all NVEs be required to get update info even for destinations > they don't care about? > >> Here's my back-of-the-envelop calculation for memory, normalized to a >> VM. Let's say a VM has 10,000 friends in the DC that it might possibly >> want to talk to, but only one that it really wants to talk to. Let's >> say that a FIB route entry takes 100 bytes. That adds up to a possible >> total of 1MB vs. an actual of 100 bytes. Is 1MB really something one >> should optimize, especially considering that the VM has probably been >> allocated 4GB? > > Are you really arguing that the difference between 1MB and 100 bytes > is just noise? Yes. But see below. > And who says this is in conventional memory on a host? > I could see this being done in the ASIC... I helped build some of those ASICs: eminently doable, fairly cost effective, simpler than implementing a new encap. But in any case, I thought the point of overlays was to shield VM scale from ASICs (whole separate conversation). So, here's my attempt to quantify the cost of push by normalizing the cost per VM. This is all very naive and preliminary (for now), but the numbers, such as they are, may speak for themselves. Let's go back to the case of a VN with 10,000 VMs (each with 4GB RAM); as the worst case, assume each VM is on a different hypervisor. 1) assuming 100 bytes per route, the memory cost per VM (as above) is 1MB for FIB space. For a 4GB VM, that's a "tax" of 0.025% 2) For VM motion, each VM moves needs an update to 10,000 NVEs. Assume each update is ~1000 bytes; that's a total of 10MB sent. Moving the VM is 4GB (or more) sent. The "tax" is 0.25%. 3) Processing an update message is a very small number of cycles. I don't know enough to compute the "tax" for this, but I'd guess very small. Clearly, there are other dimensions to this problem; folks might have better numbers or estimates. But so far, push is looking pretty good. So, is there a metric whereby push really hurts, and considering pull becomes a necessity? Or is the suggestion to explore pull models as due diligence, irrespective of the need? Or ...? Thanks, Kireeti. _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3 _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
