Any chance to apply on 2.2.x ? 2012/5/25 Edison Su <[email protected]>
> Two bugs are been created: > http://bugs.cloudstack.org/browse/CS-15095 > http://bugs.cloudstack.org/browse/CS-15094 > Let me fix it on the master branch, but not sure will it be pushed into > next release(3.0.3). > > > -----Original Message----- > > From: Diego Spinola Castro [mailto:[email protected]] > > Sent: Friday, May 25, 2012 5:07 AM > > To: [email protected] > > Subject: RE: Vmware CPU Cap > > > > Edison. What we are going to do? I believe this is a important path > > inspite > > of cs isnt using the right way of resource allocation. > > > > []'s > > Diego Castro > > Em 23/05/2012 19:29, "Edison Su" <[email protected]> escreveu: > > > > > ...Sorry, I am not read your last email carefully... > > > As I said before, in VMware, the limit parameter is set on the whole > > VM, > > > not per core. So we need to set the limit as " core_number*core_speed > > ", > > > which means > > > the VM as a whole will not use as much as " core_number*core_speed " > > > shares. It doesn't break the rule 2. > > > > > > > -----Original Message----- > > > > From: Tamas Monos [mailto:[email protected]] > > > > Sent: Wednesday, May 23, 2012 3:20 PM > > > > To: [email protected] > > > > Subject: RE: Vmware CPU Cap > > > > > > > > I'm afraid I don't follow. > > > > I was talking about multiple core VMs and not single core ones. > > > > I said if I have a 4 core VM with 3Ghz core speed and I run 4 > > cpuburn > > > > threads I could get approximately the performance of a single core > > at > > > > 12Ghz. > > > > > > > > -----Original Message----- > > > > From: Edison Su [mailto:[email protected]] > > > > Sent: 23 May 2012 23:06 > > > > To: [email protected] > > > > Subject: RE: Vmware CPU Cap > > > > > > > > Single core means there is only one scheduling entity, that > > hypervisor > > > > can schedule for the VM, so there is no parallel processing at all. > > > > Hypervisor can not schedule a single core VM to multiple physical > > cpu > > > > cores at the same time. > > > > > > > > > > > > > -----Original Message----- > > > > > From: Tamas Monos [mailto:[email protected]] > > > > > Sent: Wednesday, May 23, 2012 2:59 PM > > > > > To: [email protected] > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > Hi, > > > > > > > > > > Before we would touch that rule 2: > > > > > While utilizing 12Ghz on a single core is not possible it is > > possible > > > > > to have effective 12Ghz on 4 cores at 3Ghz via parallel > > processing. > > > > > If I start 4 cpuburn threads on a quad core it will use all four > > 3Ghz > > > > > cores flat out "giving similar speed as 1 core@12Ghz would be"*. > > As > > > > > vmware uses the limit parameter as limit to the whole CPU > > utilization > > > > > rule 2 approach would prevent me effectively utilizing the CPU on > > > > > vmware because if we say the CPU speed in Mhz cannot be larger > > than > > > > > the core speed I would be limited to an effective speed of core- > > > > > speed/number_of_cores. This would mean 3000/4=750Mhz per core on > > a > > > > > 4vCPU VM. If you set "core_number*core_speed" as limit you > > already > > > > > broke rule 2 as the CPU is set higher than its core speed. > > > > > > > > > > I believe it good as it is now. > > > > > > > > > > Regards > > > > > > > > > > -----Original Message----- > > > > > From: Prachi Damle [mailto:[email protected]] > > > > > Sent: 23 May 2012 22:36 > > > > > To: [email protected] > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > Yeah, rule#1 and #3 are considered by the current Host Allocator; > > > > > rule#2 to compare just the speed is missing. > > > > > > > > > > Prachi > > > > > > > > > > -----Original Message----- > > > > > From: Anthony Xu [mailto:[email protected]] > > > > > Sent: Wednesday, May 23, 2012 2:06 PM > > > > > To: [email protected] > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > Checked the code, I think we miss rule 2 in allocator. > > > > > > > > > > Anthony > > > > > > > > > > > -----Original Message----- > > > > > > From: Anthony Xu [mailto:[email protected]] > > > > > > Sent: Wednesday, May 23, 2012 1:50 PM > > > > > > To: [email protected] > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > I'm pretty sure we have that before we redesign allocator. > > > > > > > > > > > > Anthony > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Alex Huang [mailto:[email protected]] > > > > > > > Sent: Wednesday, May 23, 2012 1:44 PM > > > > > > > To: [email protected] > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > I'm surprised. I definitely recall having that conversation > > > > > > > before about never exceeding the physical limits of the > > > > > > > hypervisor. I'm pretty sure we have test cases against that > > as > > > > well. > > > > > > > > > > > > > > --Alex > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: Edison Su [mailto:[email protected]] > > > > > > > > Sent: Wednesday, May 23, 2012 1:39 PM > > > > > > > > To: [email protected] > > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > > > I think it's a general issue for all the hypervisors we > > > > supported. > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: Alex Huang [mailto:[email protected]] > > > > > > > > > Sent: Wednesday, May 23, 2012 1:32 PM > > > > > > > > > To: [email protected] > > > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > > > > > That's because no one wrote a host allocator for VmWare. > > > > > > > > > > > > > > > > > > The relationship between DeploymentPlanner and > > HostAllocator > > > > > and > > > > > > > > > StoragePoolAllocator is as follows: > > > > > > > > > > > > > > > > > > DeploymentPlanner deals with heuristics and affinity > > rules of > > > > > > > placing > > > > > > > > > a VM. Once it determines a set of hosts that matches, it > > > > then > > > > > > asks > > > > > > > > > the HostAllocator to work on the limitations of the type > > of > > > > > > > hypervisor. > > > > > > > > > There was never a HostAllocator written for VmWare. It > > just > > > > > > > > > uses > > > > > > > the > > > > > > > > > one that was written originally for XenServer. Hence the > > bug. > > > > > > > > > > > > > > > > > > There's similar differentiation for DeploymentPlanner and > > > > > > > > > StoragePoolAllocator and similar bugs exists, especially > > if > > > > > > someone > > > > > > > > > adds a completely new type of storage pool but decides to > > > > just > > > > > > > reuse > > > > > > > > > the current set of StoragePoolAllocators. > > > > > > > > > > > > > > > > > > --Alex > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: Edison Su [mailto:[email protected]] > > > > > > > > > > Sent: Wednesday, May 23, 2012 1:24 PM > > > > > > > > > > To: [email protected]; > > > > '[email protected]' > > > > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > > > > > > > Thanks for your input, I find another bug in > > cloudstack... > > > > > > > > > > Let's dive little bit deeper into how the CPU > > allocation > > > > > works. > > > > > > > > > > VMware(the same for other hypervisors(Xen/KVM)) uses > > > > > > proportional > > > > > > > > > share > > > > > > > > > > based scheduling algorithm([1],[2]). It means "If a > > virtual > > > > > > > machine > > > > > > > > > has twice > > > > > > > > > > as many shares of a resource as another virtual machine, > > it > > > > > is > > > > > > > > > entitled to > > > > > > > > > > consume twice as much of that resource when these two > > > > > > > > > > virtual > > > > > > > > > machines > > > > > > > > > > are competing for resources." > > > > > > > > > > There are two questions: > > > > > > > > > > 1. Where is the share coming from? In Cloudstack, the > > share > > > > > is > > > > > > > > > calculated > > > > > > > > > > from (CPU MHz * CPU cores). > > > > > > > > > > 2. How the share of a VM is mapped to the physical CPU > > by > > > > > > > > > > the > > > > > > > > > hypervisor? > > > > > > > > > > Of cause, it depends on the hypervisor implementation, > > but > > > > > > there > > > > > > > are > > > > > > > > > some > > > > > > > > > > general rules that we need to follow, as we are living > > in > > > > > > > > > > the > > > > > > > real > > > > > > > > > world:) > > > > > > > > > > Rule 1: num of cpu core of a VM should be <= num of > > cpu > > > > > > > > > > core > > > > > > > of > > > > > > > > > the > > > > > > > > > > hypervisor host. It doesn't make sense to running a 12 > > core > > > > > VM > > > > > > on > > > > > > > a > > > > > > > > > single > > > > > > > > > > core host, even some hypervisors, such as KVM, can do > > that. > > > > > > > > > Hypervisor will > > > > > > > > > > be busy at scheduling VCPU(the VCPU context switch is > > much > > > > > > > heavier > > > > > > > > > than > > > > > > > > > > process context switch), thus bad performance for the > > VM. > > > > > > > > > > Rule 2: The frequency of a VCPU should be <= the > > > > > > > > > > frequency > > > > > > of > > > > > > > the > > > > > > > > > host > > > > > > > > > > hypervisor host. Can you run an one core * 12Ghz VM on > > a > > > > > > > > > > 3Ghz > > > > > > > > > > * > > > > > > 4 > > > > > > > > > core > > > > > > > > > > physical CPU? Nope, in an unit time, the max freq of > > VCPU > > > > > > > > > > can > > > > > > get > > > > > > > is > > > > > > > > > the max > > > > > > > > > > freq of physical cpu core, that's the physical LAW. > > > > > > > > > > Rule 3: the total share of VMs <= total share of > > host > > > > > > > > > hypervisor(in case of > > > > > > > > > > no CPU overcommit). > > > > > > > > > > > > > > > > > > > > Currently, in cloudstack host allocator, only rule 3 is > > > > > > > > > > taken > > > > > > > into > > > > > > > > > consideration. > > > > > > > > > > > > > > > > > > > > [1] www.cs.uiuc.edu/class/sp10/cs423/lectures/17-VMM- > > > > > > resource.pdf > > > > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > http://www.vmware.com/pdf/vsphere4/r40/vsp_40_resource_mgmt.pdf > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: Tamas Monos [mailto:[email protected]] > > > > > > > > > > > Sent: Tuesday, May 22, 2012 5:41 PM > > > > > > > > > > > To: [email protected] > > > > > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > I still don't think this is an issue as the CPU Mhz > > limit > > > > > > > > > > > and > > > > > > > the > > > > > > > > > > > number of cores are independent. > > > > > > > > > > > CPU manufacturers sell 2,4,6 cores at 3Ghz and not > > > > > > > > > > > 6,12,18Ghz > > > > > > > CPUs. > > > > > > > > > > > > > > > > > > > > > > So I think it is good how it works but the > > > > > > > "number_of_cores*Mhz" > > > > > > > > > while > > > > > > > > > > > allocating should not multiply so that is the bug :) > > What > > > > > > > vmware > > > > > > > > > does > > > > > > > > > > > with multiplying the cores with the core speed is bad > > as > > > > I > > > > > > > can't > > > > > > > > > have > > > > > > > > > > > a 1vCPU VM at 12Ghz on a 3Ghz quad core if you know > > where > > > > > > > > > > > I'm > > > > > > > > > coming > > > > > > > > > > > from.... > > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: Tamas Monos [mailto:[email protected]] > > > > > > > > > > > Sent: 23 May 2012 01:27 > > > > > > > > > > > To: [email protected] > > > > > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > I have confused myself too because if I have a look > > the > > > > > > > database > > > > > > > > > and > > > > > > > > > > > dump the service_offerings table the limit for me is > > set > > > > > > > > > > > to > > > > > > > > > > > 2000,3000,4000 and not 1000. > > > > > > > > > > > So a Mhz limit set to 4000 and 4 cores will end up as > > a > > > > > quad > > > > > > > core > > > > > > > > > box > > > > > > > > > > > at 4Ghz. I remember now I had to set CPU > > overprovisioning > > > > > to > > > > > > 4 > > > > > > > as > > > > > > > > > > > CloudStack took away 4x4Ghz=16Ghz of the available > > CPU. > > > > > > > > > > > So Cloudstack sets the VM CPU Mhz limit what the > > actual > > > > > > > > > > > limit > > > > > > > is > > > > > > > > > set > > > > > > > > > > > to in the offering but does not multiply the Mhz > > limit by > > > > > > > > > > > the > > > > > > > > > number > > > > > > > > > > > of cores when setting the limit on vmware. However > > takes > > > > > > > > > > > away "number_of_cores*Mhz limit" from the available > > CPU > > > > > > > > > > > capacity > > > > > > > when > > > > > > > > > > > allocating. > > > > > > > > > > > > > > > > > > > > > > I'm getting confused myself so I'm not sure if this > > is > > > > bug > > > > > > now > > > > > > > or > > > > > > > > > not > > > > > > > > > > > either :) I'm using 3.0.3 > > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: Edison Su [mailto:[email protected]] > > > > > > > > > > > Sent: 23 May 2012 00:32 > > > > > > > > > > > To: [email protected]; Tamas Monos > > > > > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > > > > > > > > > But in Diego'case, the limit is set as 1000Mhz, while > > his > > > > > > > service > > > > > > > > > > > offering is 1000Mhz * 2 cores. > > > > > > > > > > > Which version of cloudstack are you using? Maybe it's > > a > > > > > > > regression. > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > > From: Tamas Monos [mailto:[email protected]] > > > > > > > > > > > > Sent: Tuesday, May 22, 2012 4:26 PM > > > > > > > > > > > > To: [email protected] > > > > > > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > I have a service offering with 1000Mhz limit and 4 > > cpu > > > > > > cores. > > > > > > > > > That > > > > > > > > > > > > sums up to 4000Mhz. > > > > > > > > > > > > Cloudstack sets the limit to 4Ghz on the virtual > > > > machine > > > > > > and > > > > > > > > > > > > when > > > > > > > > > I > > > > > > > > > > > > put load on it vmware balances the load between the > > 4 > > > > > > > > > > > > cores > > > > > > > > > allowing > > > > > > > > > > > > them to use 1000Mhz each. > > > > > > > > > > > > I do not see any bugs here. > > > > > > > > > > > > > > > > > > > > > > > > Apologies if the "meant per CPU core" was incorrect. > > > > > > > > > > > > What I meant > > > > > > > > > is > > > > > > > > > > > > described above. > > > > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > > From: Edison Su [mailto:[email protected]] > > > > > > > > > > > > Sent: 23 May 2012 00:05 > > > > > > > > > > > > To: [email protected] > > > > > > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > > > > > > > > > > > Nope, from the document, the limit is set on whole > > vm: > > > > > > > > > > > > CPU Limits > > > > > > > > > > > > > > > > > > > > > > > > When a CPU Limit is set on a virtual machine > > resource > > > > > > > settings, > > > > > > > > > the > > > > > > > > > > > > virtual machine is deliberately held from being > > > > > > > > > > > > scheduled > > > > > > to > > > > > > > a > > > > > > > > > PCPU > > > > > > > > > > > > when it has used up its allocated CPU resource. > > This > > > > > > happens > > > > > > > > > > > > regardless of the CPU utilization. If the limit is > > set > > > > > > > > > > > > to 500MHz, the virtual machine is descheduled from > > the > > > > > > > > > > > > PCPU > > > > > > and > > > > > > > has > > > > > > > > > > > > to wait before > > > > > > > > > > > it > > > > > > > > > > > > is allowed to be scheduled again. As such, the > > virtual > > > > > > > machine > > > > > > > > > might > > > > > > > > > > > > experience performance degradation. > > > > > > > > > > > > > > > > > > > > > > > > Note: For an SMP virtual machine, the sum of all > > vCPUs > > > > > > cannot > > > > > > > > > exceed > > > > > > > > > > > > the specified limit. For example, 4 vCPU virtual > > > > machine > > > > > > with > > > > > > > a > > > > > > > > > > > > limit of 1200MHz and equal load among vCPUs would > > > > result > > > > > > > > > > > > in > > > > > > a > > > > > > > > > > > > max > > > > > > > > > of > > > > > > > > > > > > 300MHz per vCPU. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://kb.vmware.com/selfservice/documentLinkInt.do?micrositeID=&pop > > > > > > > > > > u > > > > > > > > > > p > > > > > > > > > > > > = > > > > > > > > > > > > true&languageId=&externalID=1033115 > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > > > From: Tamas Monos [mailto:[email protected]] > > > > > > > > > > > > > Sent: Tuesday, May 22, 2012 3:43 PM > > > > > > > > > > > > > To: [email protected] > > > > > > > > > > > > > Subject: RE: Vmware CPU Cap > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > No I don't think this is a bug. When you set > > 1000Mhz > > > > > > > > > > > > > as > > > > > > CPU > > > > > > > > > > > > > cap > > > > > > > > > > > that > > > > > > > > > > > > > is meant per core. vmWare will limit each CPU > > core to > > > > > > > 1000Mhz. > > > > > > > > > > > > > As you gave 2 CPU cores that is 2000Mhz effective. > > > > > > > > > > > > > That > > > > > > is > > > > > > > how > > > > > > > > > > > > > vmware works. > > > > > > > > > > > > > > > > > > > > > > > > > > I have setup my offerings all to 1000Mhz as speed > > and > > > > > > just > > > > > > > > > > > > > increasing the number of cores. > > > > > > > > > > > > > 1 core ends up being 1x1000Mhz > > > > > > > > > > > > > 2 core ends up being 2x1000Mhz=2000Mhz ... > > > > > > > > > > > > > ... > > > > > > > > > > > > > > > > > > > > > > > > > > I'm actually using it in production and works > > quite > > > > > well > > > > > > as > > > > > > > > > > > > Cloudstack > > > > > > > > > > > > > "allocates" 4000Mhz when I'm using 4x1000Mhz > > cores > > > > and > > > > > > > vmware > > > > > > > > > > > > cleverly > > > > > > > > > > > > > balances between the cores as you put load on it > > and > > > > > not > > > > > > > > > letting > > > > > > > > > > > any > > > > > > > > > > > > > of the cores above the set limit of 1000Mhz. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > > > From: Diego Spinola Castro > > > > > > [mailto:[email protected]] > > > > > > > > > > > > > Sent: 22 May 2012 19:34 > > > > > > > > > > > > > To: [email protected] > > > > > > > > > > > > > Subject: Re: Vmware CPU Cap > > > > > > > > > > > > > > > > > > > > > > > > > > I forgot the link: > > > > > > > > > > > > > http://imageshack.us/photo/my- > > > > images/848/cpulimit.png/ > > > > > > > > > > > > > > > > > > > > > > > > > > 2012/5/22 Diego Spinola Castro > > > > > <[email protected]> > > > > > > > > > > > > > > > > > > > > > > > > > > > I believe that is a bug with cpu cap and vmware. > > > > > > > > > > > > > > > > > > > > > > > > > > > > To reproduce: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Create a offering with 2 cores and 1000mhz. > > > > > > > > > > > > > > Enable CPU CAP. > > > > > > > > > > > > > > > > > > > > > > > > > > > > After created instance , cs create a vm with 2 > > > > cores > > > > > > and > > > > > > > > > 1000mhz > > > > > > > > > > > > > > of > > > > > > > > > > > > > limit. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't know for sure if is a bug, but vmware > > gives > > > > > > > 1000mhz > > > > > > > > > > > shared > > > > > > > > > > > > > > with cores. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Diego > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
