It was enough to just stop VMs and start again (assuming you change
cpu.overprovisioning on the cluster level - i.e. settings tab of the actual
cluster). If global settings changed, then yes, restart of mgmt server and
again stop/start existing VMs, but definitely NOT removing and re-adding
the host.
Glad you have it solved.
On Sun, Mar 10, 2019, 17:29 Fariborz Navidan <mdvlinqu...@gmail.com> wrote:
> Hello All,
>
> I solved the issue. When you have running VMs, after changing
> cpu.overprovising.factor and restart management server, you need to stop
> all VMs, put the host in Maintenance mode and remove it. The host will be
> automatically re-added to the cluster and you will get additional CPU
> capacity as I wanted.
>
> Thank you for all your ideas.
>
> Best Regards
>
> On Sun, Mar 10, 2019 at 7:18 PM Andrija Panic <andrija.pa...@gmail.com>
> wrote:
>
> > Please send compete log to pastebin.com or such...and link here.
> >
> > On Sun, Mar 10, 2019, 14:36 Rafael Weingärtner <
> > rafaelweingart...@gmail.com>
> > wrote:
> >
> > > Ah, this happens for a simple reason.
> > >
> > > When you allocate a VM with over-provision of 1, you are charging by
> > that,
> > > and later if you change the over-provisioning to two; what would you
> > expect
> > > to happen with that VM that was allocated when over-provisioning was
> set
> > to
> > > 1? ACS is doing that do maintain the resources originally allocated by
> > the
> > > VM. If you want to change this, you need to restart the VMs.
> > >
> > > On Sun, Mar 10, 2019 at 10:19 AM Fariborz Navidan <
> mdvlinqu...@gmail.com
> > >
> > > wrote:
> > >
> > > > I guess the issue is that allocated cpu is also grown by
> > overprovisioning
> > > > factor such thatusage percentage remains the same. So, by increasing
> > cpu
> > > > overprovisiong factor you don't gain access to un-utilized cpu
> > capacity.
> > > My
> > > > real cpu utilization on the host is 31% but 84.82% is allocated and
> by
> > > > increasing the factor, the usage is also increased!
> > > >
> > > > On Sun, Mar 10, 2019 at 4:40 PM Fariborz Navidan <
> > mdvlinqu...@gmail.com>
> > > > wrote:
> > > >
> > > > > Even with cpu overprovisioning factor of 20 set on both cluster
> and
> > > > > global settings, still I cannot deploy new VMs. It show 672GHz in
> > > > > dashboardinstead of 33.6GHz (actual capacity) but still I get in
> the
> > > log
> > > > > "The host is in avoid list".
> > > > >
> > > > > Please advise.
> > > > > Thanks
> > > > >
> > > > > On Sun, Mar 10, 2019 at 4:04 PM Andrija Panic <
> > andrija.pa...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Cpu.overprovisioning.ratio or such - set it to 2.0 - means
> > CloudStack
> > > > will
> > > > >> see your cluster having 200% cpu of that it really has, 3.5= 350%
> > etc.
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Sun, Mar 10, 2019, 10:24 Fariborz Navidan <
> mdvlinqu...@gmail.com
> > >
> > > > >> wrote:
> > > > >>
> > > > >> > I've already set
> cluster.cpu.allocated.capacity.disablethreshold
> > to
> > > > >> > maximum value in both cluster and global settings. Is it
> possible
> > to
> > > > to
> > > > >> > makeACS ignore cpu capacity check?
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Sun, Mar 10, 2019 at 12:43 PM Andrija Panic <
> > > > andrija.pa...@gmail.com
> > > > >> >
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Try to o provision CPU on a cluster level, thought I can't say
> > if
> > > > >> > shortage
> > > > >> > > of CPU or RAM, on my mobile atm...
> > > > >> > >
> > > > >> > > On Sun, Mar 10, 2019, 10:03 Andrija Panic <
> > > andrija.pa...@gmail.com>
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > > Cannot allocate cluster list [1] for vm creation since their
> > > > >> allocated
> > > > >> > > > percentage crosses the disable capacity threshold defined at
> > > each
> > > > >> > > cluster/
> > > > >> > > > at global value for capacity Type : 1, skipping these
> clusters
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Sat, Mar 9, 2019, 20:54 Fariborz Navidan <
> > > > mdvlinqu...@gmail.com>
> > > > >> > > wrote:
> > > > >> > > >
> > > > >> > > >> Any idea?
> > > > >> > > >>
> > > > >> > > >> On Sat, Mar 9, 2019 at 10:05 PM Fariborz Navidan <
> > > > >> > mdvlinqu...@gmail.com
> > > > >> > > >
> > > > >> > > >> wrote:
> > > > >> > > >>
> > > > >> > > >> > Hi,
> > > > >> > > >> >
> > > > >> > > >> > Another noticeable issue about system is that cpu cores
> > > > >> allocation
> > > > >> > is
> > > > >> > > >> 19/8
> > > > >> > > >> > (237%) which may be thereason for error. However the RAM
> > and
> > > > CPU
> > > > >> > > >> > utilization is fairly low and I should be able to create
> > more
> > > > >> VMs.
> > > > >> > How
> > > > >> > > >> can
> > > > >> > > >> > I change settings for cpu cores overprovisioning factor?
> > > > >> > > >> >
> > > > >> > > >> > Thanks
> > > > >> > > >> >
> > > > >> > > >> > On Sat, Mar 9, 2019 at 9:53 PM Fariborz Navidan <
> > > > >> > > mdvlinqu...@gmail.com>
> > > > >> > > >> > wrote:
> > > > >> > > >> >
> > > > >> > > >> >> Any idea?
> > > > >> > > >> >>
> > > > >> > > >> >> On Sat, Mar 9, 2019 at 8:38 PM Fariborz Navidan <
> > > > >> > > mdvlinqu...@gmail.com
> > > > >> > > >> >
> > > > >> > > >> >> wrote:
> > > > >> > > >> >>
> > > > >> > > >> >>> Memory Total: 62.69 GB
> > > > >> > > >> >>> Memory Allocated: *14.25 GB*
> > > > >> > > >> >>> Memory Used: 19.03 GB
> > > > >> > > >> >>>
> > > > >> > > >> >>> There are also lots of storage available. As the log
> > > states,
> > > > >> it is
> > > > >> > > >> >>> complaining about resource type 1 (cpu).
> > > > >> > > >> >>>
> > > > >> > > >> >>> Please advise.
> > > > >> > > >> >>>
> > > > >> > > >> >>> Thanks
> > > > >> > > >> >>>
> > > > >> > > >> >>> On Sat, Mar 9, 2019 at 8:16 PM Ivan Kudryavtsev <
> > > > >> > > >> >>> kudryavtsev...@bw-sw.com> wrote:
> > > > >> > > >> >>>
> > > > >> > > >> >>>> What about RAM and storage?
> > > > >> > > >> >>>>
> > > > >> > > >> >>>> сб, 9 мар. 2019 г., 9:22 Fariborz Navidan <
> > > > >> mdvlinqu...@gmail.com
> > > > >> > >:
> > > > >> > > >> >>>>
> > > > >> > > >> >>>> > I have already set
> > > > >> > > cluster.cpu.allocated.capacity.disablethreshold
> > > > >> > > >> to
> > > > >> > > >> >>>> 1
> > > > >> > > >> >>>> > and cpu.overprovisioning.factor to 10 both in global
> > > > >> settings
> > > > >> > and
> > > > >> > > >> >>>> cluster
> > > > >> > > >> >>>> > level.
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>> > Best Regards
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>> > On Sat, Mar 9, 2019 at 6:47 PM Fariborz Navidan <
> > > > >> > > >> >>>> mdvlinqu...@gmail.com>
> > > > >> > > >> >>>> > wrote:
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>> > > Hello
> > > > >> > > >> >>>> > >
> > > > >> > > >> >>>> > > This is host's cpu resource statistics:
> > > > >> > > >> >>>> > >
> > > > >> > > >> >>>> > > CPU Utilized: 30.2%
> > > > >> > > >> >>>> > > CPU Allocated for VMs: 84.82%
> > > > >> > > >> >>>> > >
> > > > >> > > >> >>>> > > On Sat, Mar 9, 2019 at 4:20 PM Ivan Kudryavtsev <
> > > > >> > > >> >>>> > kudryavtsev...@bw-sw.com>
> > > > >> > > >> >>>> > > wrote:
> > > > >> > > >> >>>> > >
> > > > >> > > >> >>>> > >> Looks like your cluster is pretty full. Increase
> > > > >> thresholds
> > > > >> > in
> > > > >> > > >> >>>> cluster
> > > > >> > > >> >>>> > >> vars
> > > > >> > > >> >>>> > >> or add resources.
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> > >> сб, 9 мар. 2019 г., 6:55 Rafael Weingärtner <
> > > > >> > > >> >>>> > rafaelweingart...@gmail.com
> > > > >> > > >> >>>> > >> >:
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> > >> > In the log there is this message:
> > > > >> > > >> >>>> > >> > > Cannot allocate cluster list [1] for vm
> > creation
> > > > >> since
> > > > >> > > their
> > > > >> > > >> >>>> > allocated
> > > > >> > > >> >>>> > >> > percentage crosses the disable capacity
> threshold
> > > > >> defined
> > > > >> > at
> > > > >> > > >> each
> > > > >> > > >> >>>> > >> > cluster/at global value for capacity Type : 1,
> > > > skipping
> > > > >> > > these
> > > > >> > > >> >>>> clusters
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >> > What is the status of your cluster's host?
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >> > On Sat, Mar 9, 2019 at 7:14 AM Fariborz
> Navidan <
> > > > >> > > >> >>>> > mdvlinqu...@gmail.com>
> > > > >> > > >> >>>> > >> > wrote:
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >> > > Hello,
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > > Here is the log for VM deployment:
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:34,984 DEBUG
> > [c.c.a.ApiServlet]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578)
> (logid:113f467d)
> > > > >> > > ===START===
> > > > >> > > >> >>>> > >> 137.74.35.65
> > > > >> > > >> >>>> > >> > > -- GET
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> command=deployVirtualMachine&response=json&zoneid=bc4565d8-4029-4dbd-93eb-47137ff45188&templateid=4c10414a-51f2-4859-b673-e92c3f32cbfd&hypervisor=KVM&rootdisksize=10&serviceofferingid=cd1235b8-acdf-4ce3-a3ac-3f7ddcea8870&displayname=centos&name=centos&_=1552125695843
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:34,989 DEBUG
> [c.c.a.ApiServer]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> CIDRs
> > > > >> > > >> >>>> > >> from
> > > > >> > > >> >>>> > >> > > which account
> > > > >> > > >> >>>> 'Acct[27cd01ef-3907-11e9-87ab-a4bf012ed1a6-admin]' is
> > > > >> > > >> >>>> > >> > allowed
> > > > >> > > >> >>>> > >> > > to perform API calls: 0.0.0.0/0,::/0
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,040 DEBUG
> > > > >> [c.c.u.AccountManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> Access
> > > > >> > > >> >>>> > >> > granted
> > > > >> > > >> >>>> > >> > > to
> > > Acct[27cd01ef-3907-11e9-87ab-a4bf012ed1a6-admin]
> > > > >> to
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.quota.vo.ServiceOfferingVO$$EnhancerByCGLIB$$c370fca@68de8a96
> > > > >> > > >> >>>> > >> > > by AffinityGroupAccessChecker
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,042 DEBUG
> > > > >> [c.c.u.AccountManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> Access
> > > > >> > > >> >>>> > >> > granted
> > > > >> > > >> >>>> > >> > > to
> > > Acct[27cd01ef-3907-11e9-87ab-a4bf012ed1a6-admin]
> > > > >> to
> > > > >> > > null
> > > > >> > > >> by
> > > > >> > > >> >>>> > >> > > AffinityGroupAccessChecker
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,046 DEBUG
> > > > >> [c.c.n.NetworkModelImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> Service
> > > > >> > > >> >>>> > >> > > SecurityGroup is not supported in the network
> > > > id=204
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,058 DEBUG
> > > > >> [c.c.n.NetworkModelImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> Service
> > > > >> > > >> >>>> > >> > > SecurityGroup is not supported in the network
> > > > id=204
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,074 DEBUG
> > > > >> [c.c.v.UserVmManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > >> Rootdisksize
> > > > >> > > >> >>>> > >> > > override validation successful. Template root
> > > disk
> > > > >> size
> > > > >> > > 8GB
> > > > >> > > >> >>>> Root
> > > > >> > > >> >>>> > disk
> > > > >> > > >> >>>> > >> > size
> > > > >> > > >> >>>> > >> > > specified 10GB
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,081 DEBUG
> > > > >> [c.c.v.UserVmManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > >> Allocating
> > > > >> > > >> >>>> > >> > in
> > > > >> > > >> >>>> > >> > > the DB for vm
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,098 DEBUG
> > > > >> > > >> [c.c.v.VirtualMachineManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > >> Allocating
> > > > >> > > >> >>>> > >> > > entries for VM: VM[User|i-2-68-VM]
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,099 DEBUG
> > > > >> > > >> [c.c.v.VirtualMachineManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > >> Allocating
> > > > >> > > >> >>>> > >> > > nics for VM[User|i-2-68-VM]
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,100 DEBUG
> > > > >> > > >> [o.a.c.e.o.NetworkOrchestrator]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > >> Allocating
> > > > >> > > >> >>>> > >> > nic
> > > > >> > > >> >>>> > >> > > for vm VM[User|i-2-68-VM] in network
> > > > >> Ntwk[204|Guest|7]
> > > > >> > > with
> > > > >> > > >> >>>> > requested
> > > > >> > > >> >>>> > >> > > profile NicProfile[0-0-null-null-null
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,106 DEBUG
> > > > >> [c.c.n.NetworkModelImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> Service
> > > > >> > > >> >>>> > >> > > SecurityGroup is not supported in the network
> > > > id=204
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,106 DEBUG
> > > > >> > > >> [c.c.v.VirtualMachineManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > >> Allocating
> > > > >> > > >> >>>> > >> > > disks for VM[User|i-2-68-VM]
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,108 DEBUG
> > > > >> > > [o.a.c.e.o.VolumeOrchestrator]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> Using
> > > > >> > > >> >>>> > >> root
> > > > >> > > >> >>>> > >> > > disk size of 10737418240 Bytes for volume
> > ROOT-68
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,111 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > Updating
> > > > >> > > >> >>>> > >> > > resource Type = volume count for Account = 2
> > > > >> Operation =
> > > > >> > > >> >>>> increasing
> > > > >> > > >> >>>> > >> > Amount
> > > > >> > > >> >>>> > >> > > = 1
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,116 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > Updating
> > > > >> > > >> >>>> > >> > > resource Type = primary_storage count for
> > Account
> > > > = 2
> > > > >> > > >> >>>> Operation =
> > > > >> > > >> >>>> > >> > > increasing Amount = 10737418240
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,118 DEBUG
> > > > >> > > >> [c.c.v.VirtualMachineManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > >> Allocation
> > > > >> > > >> >>>> > >> > > completed for VM: VM[User|i-2-68-VM]
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,118 DEBUG
> > > > >> [c.c.v.UserVmManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > >> Successfully
> > > > >> > > >> >>>> > >> > > allocated DB entry for VM[User|i-2-68-VM]
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,119 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > Updating
> > > > >> > > >> >>>> > >> > > resource Type = user_vm count for Account = 2
> > > > >> Operation
> > > > >> > =
> > > > >> > > >> >>>> increasing
> > > > >> > > >> >>>> > >> > Amount
> > > > >> > > >> >>>> > >> > > = 1
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,120 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > Updating
> > > > >> > > >> >>>> > >> > > resource Type = cpu count for Account = 2
> > > > Operation =
> > > > >> > > >> >>>> increasing
> > > > >> > > >> >>>> > >> Amount
> > > > >> > > >> >>>> > >> > = 1
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,122 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > Updating
> > > > >> > > >> >>>> > >> > > resource Type = memory count for Account = 2
> > > > >> Operation =
> > > > >> > > >> >>>> increasing
> > > > >> > > >> >>>> > >> > Amount
> > > > >> > > >> >>>> > >> > > = 512
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,128 DEBUG
> > > > >> [c.c.n.NetworkModelImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> Service
> > > > >> > > >> >>>> > >> > > SecurityGroup is not supported in the network
> > > > id=204
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,130 DEBUG
> > > > >> [c.c.n.NetworkModelImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> Service
> > > > >> > > >> >>>> > >> > > SecurityGroup is not supported in the network
> > > > id=204
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,138 INFO
> > > > >> > > [o.a.c.f.j.i.AsyncJobMonitor]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508)
> > > > >> > (logid:4c7ba6b4)
> > > > >> > > >> Add
> > > > >> > > >> >>>> > job-508
> > > > >> > > >> >>>> > >> > into
> > > > >> > > >> >>>> > >> > > job monitoring
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,139 DEBUG
> > > > >> > > >> [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> submit
> > > > >> > > >> >>>> > >> async
> > > > >> > > >> >>>> > >> > > job-508, details: AsyncJobVO {id:508, userId:
> > 2,
> > > > >> > > accountId:
> > > > >> > > >> 2,
> > > > >> > > >> >>>> > >> > > instanceType: VirtualMachine, instanceId: 68,
> > > cmd:
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >>
> > org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin,
> > > > >> > > >> >>>> > >> cmdInfo:
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> {"httpmethod":"GET","templateid":"4c10414a-51f2-4859-b673-e92c3f32cbfd","ctxAccountId":"2","uuid":"f8926b46-c679-4e75-8410-bbe5e612bd93","cmdEventType":"VM.CREATE","serviceofferingid":"cd1235b8-acdf-4ce3-a3ac-3f7ddcea8870","response":"json","ctxUserId":"2","hypervisor":"KVM","displayname":"centos","name":"centos","zoneid":"bc4565d8-4029-4dbd-93eb-47137ff45188","rootdisksize":"10","ctxStartEventId":"995","id":"68","ctxDetails":"{\"interface
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.vm.VirtualMachine\":\"f8926b46-c679-4e75-8410-bbe5e612bd93\",\"interface
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.dc.DataCenter\":\"bc4565d8-4029-4dbd-93eb-47137ff45188\",\"interface
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.offering.ServiceOffering\":\"cd1235b8-acdf-4ce3-a3ac-3f7ddcea8870\",\"interface
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.template.VirtualMachineTemplate\":\"4c10414a-51f2-4859-b673-e92c3f32cbfd\"}","_":"1552125695843"},
> > > > >> > > >> >>>> > >> > > cmdVersion: 0, status: IN_PROGRESS,
> > > processStatus:
> > > > 0,
> > > > >> > > >> >>>> resultCode: 0,
> > > > >> > > >> >>>> > >> > > result: null, initMsid: 279278805450982,
> > > > >> completeMsid:
> > > > >> > > null,
> > > > >> > > >> >>>> > >> lastUpdated:
> > > > >> > > >> >>>> > >> > > null, lastPolled: null, created: null}
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,141 DEBUG
> > [c.c.a.ApiServlet]
> > > > >> > > >> >>>> > >> > > (qtp788117692-18:ctx-f51fd578 ctx-a82cdf12)
> > > > >> > > (logid:113f467d)
> > > > >> > > >> >>>> > ===END===
> > > > >> > > >> >>>> > >> > > 137.74.35.65 -- GET
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> command=deployVirtualMachine&response=json&zoneid=bc4565d8-4029-4dbd-93eb-47137ff45188&templateid=4c10414a-51f2-4859-b673-e92c3f32cbfd&hypervisor=KVM&rootdisksize=10&serviceofferingid=cd1235b8-acdf-4ce3-a3ac-3f7ddcea8870&displayname=centos&name=centos&_=1552125695843
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,142 DEBUG
> > > > >> > > >> [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508)
> > > > >> > (logid:57397166)
> > > > >> > > >> >>>> Executing
> > > > >> > > >> >>>> > >> > > AsyncJobVO {id:508, userId: 2, accountId: 2,
> > > > >> > instanceType:
> > > > >> > > >> >>>> > >> > VirtualMachine,
> > > > >> > > >> >>>> > >> > > instanceId: 68, cmd:
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >>
> > org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin,
> > > > >> > > >> >>>> > >> cmdInfo:
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> {"httpmethod":"GET","templateid":"4c10414a-51f2-4859-b673-e92c3f32cbfd","ctxAccountId":"2","uuid":"f8926b46-c679-4e75-8410-bbe5e612bd93","cmdEventType":"VM.CREATE","serviceofferingid":"cd1235b8-acdf-4ce3-a3ac-3f7ddcea8870","response":"json","ctxUserId":"2","hypervisor":"KVM","displayname":"centos","name":"centos","zoneid":"bc4565d8-4029-4dbd-93eb-47137ff45188","rootdisksize":"10","ctxStartEventId":"995","id":"68","ctxDetails":"{\"interface
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.vm.VirtualMachine\":\"f8926b46-c679-4e75-8410-bbe5e612bd93\",\"interface
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.dc.DataCenter\":\"bc4565d8-4029-4dbd-93eb-47137ff45188\",\"interface
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.offering.ServiceOffering\":\"cd1235b8-acdf-4ce3-a3ac-3f7ddcea8870\",\"interface
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.template.VirtualMachineTemplate\":\"4c10414a-51f2-4859-b673-e92c3f32cbfd\"}","_":"1552125695843"},
> > > > >> > > >> >>>> > >> > > cmdVersion: 0, status: IN_PROGRESS,
> > > processStatus:
> > > > 0,
> > > > >> > > >> >>>> resultCode: 0,
> > > > >> > > >> >>>> > >> > > result: null, initMsid: 279278805450982,
> > > > >> completeMsid:
> > > > >> > > null,
> > > > >> > > >> >>>> > >> lastUpdated:
> > > > >> > > >> >>>> > >> > > null, lastPolled: null, created: null}
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,157 DEBUG
> > > > >> [c.c.n.NetworkModelImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Service SecurityGroup is not supported in the
> > > > network
> > > > >> > > id=204
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,159 DEBUG
> > > > >> [c.c.n.NetworkModelImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Service SecurityGroup is not supported in the
> > > > network
> > > > >> > > id=204
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,173 DEBUG
> > > > >> > > >> >>>> [c.c.d.DeploymentPlanningManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > DeploymentPlanner allocation algorithm: null
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,173 DEBUG
> > > > >> > > >> >>>> [c.c.d.DeploymentPlanningManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Trying to allocate a host and storage pools
> > from
> > > > >> dc:1,
> > > > >> > > >> >>>> > >> > > pod:null,cluster:null, requested cpu: 500,
> > > > requested
> > > > >> > ram:
> > > > >> > > >> >>>> 536870912
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,173 DEBUG
> > > > >> > > >> >>>> [c.c.d.DeploymentPlanningManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > Is
> > > > >> > > >> >>>> > >> > > ROOT volume READY (pool already allocated)?:
> No
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,176 DEBUG
> > > > >> > > >> >>>> [c.c.d.DeploymentPlanningManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Deploy avoids pods: [], clusters: [], hosts:
> []
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,176 DEBUG
> > > > [c.c.d.FirstFitPlanner]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Searching all possible resources under this
> > > Zone: 1
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,177 DEBUG
> > > > [c.c.d.FirstFitPlanner]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Listing clusters in order of aggregate
> > capacity,
> > > > that
> > > > >> > have
> > > > >> > > >> >>>> (atleast
> > > > >> > > >> >>>> > >> one
> > > > >> > > >> >>>> > >> > > host with) enough CPU and RAM capacity under
> > this
> > > > >> Zone:
> > > > >> > 1
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,179 DEBUG
> > > > [c.c.d.FirstFitPlanner]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Removing from the clusterId list these
> clusters
> > > > from
> > > > >> > avoid
> > > > >> > > >> >>>> set: []
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,181 DEBUG
> > > > [c.c.d.FirstFitPlanner]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Cannot allocate cluster list [1] for vm
> > creation
> > > > >> since
> > > > >> > > their
> > > > >> > > >> >>>> > allocated
> > > > >> > > >> >>>> > >> > > percentage crosses the disable capacity
> > threshold
> > > > >> > defined
> > > > >> > > at
> > > > >> > > >> >>>> each
> > > > >> > > >> >>>> > >> > cluster/
> > > > >> > > >> >>>> > >> > > at global value for capacity Type : 1,
> skipping
> > > > these
> > > > >> > > >> clusters
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,181 DEBUG
> > > > [c.c.d.FirstFitPlanner]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > No
> > > > >> > > >> >>>> > >> > > clusters found after removing disabled
> clusters
> > > and
> > > > >> > > >> clusters in
> > > > >> > > >> >>>> > avoid
> > > > >> > > >> >>>> > >> > list,
> > > > >> > > >> >>>> > >> > > returning.
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,183 DEBUG
> > > > >> [c.c.v.UserVmManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Destroying vm VM[User|i-2-68-VM] as it failed
> > to
> > > > >> create
> > > > >> > on
> > > > >> > > >> >>>> Host with
> > > > >> > > >> >>>> > >> > > Id:null
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,189 DEBUG
> > > > >> > [c.c.c.CapacityManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > VM
> > > > >> > > >> >>>> > >> > > state transitted from :Stopped to Error with
> > > event:
> > > > >> > > >> >>>> > >> > > OperationFailedToErrorvm's original host id:
> > null
> > > > new
> > > > >> > host
> > > > >> > > >> id:
> > > > >> > > >> >>>> null
> > > > >> > > >> >>>> > >> host
> > > > >> > > >> >>>> > >> > id
> > > > >> > > >> >>>> > >> > > before state transition: null
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,199 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Updating resource Type = volume count for
> > Account
> > > > = 2
> > > > >> > > >> >>>> Operation =
> > > > >> > > >> >>>> > >> > > decreasing Amount = 1
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,202 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Updating resource Type = primary_storage
> count
> > > for
> > > > >> > Account
> > > > >> > > >> = 2
> > > > >> > > >> >>>> > >> Operation
> > > > >> > > >> >>>> > >> > =
> > > > >> > > >> >>>> > >> > > decreasing Amount = 10737418240
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,204 WARN [o.a.c.alerts]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > AlertType:: 8 | dataCenterId:: 1 | podId::
> > null |
> > > > >> > > >> clusterId::
> > > > >> > > >> >>>> null |
> > > > >> > > >> >>>> > >> > > message:: Failed to deploy Vm with Id: 68, on
> > > Host
> > > > >> with
> > > > >> > > Id:
> > > > >> > > >> >>>> null
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,206 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Updating resource Type = user_vm count for
> > > Account
> > > > =
> > > > >> 2
> > > > >> > > >> >>>> Operation =
> > > > >> > > >> >>>> > >> > > decreasing Amount = 1
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,208 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Updating resource Type = cpu count for
> Account
> > =
> > > 2
> > > > >> > > >> Operation =
> > > > >> > > >> >>>> > >> decreasing
> > > > >> > > >> >>>> > >> > > Amount = 1
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,209 DEBUG
> > > > >> [c.c.a.m.AgentManagerImpl]
> > > > >> > > >> >>>> > >> > > (AgentManager-Handler-12:null) (logid:) SeqA
> > > > >> 11-96232:
> > > > >> > > >> >>>> Processing
> > > > >> > > >> >>>> > Seq
> > > > >> > > >> >>>> > >> > > 11-96232: { Cmd , MgmtId: -1, via: 11, Ver:
> > v1,
> > > > >> Flags:
> > > > >> > > 11,
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":12,"_loadInfo":"{\n
> > > > >> > > >> >>>> > >> > > \"connections\": []\n}","wait":0}}] }
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,210 DEBUG
> > > > >> [c.c.a.m.AgentManagerImpl]
> > > > >> > > >> >>>> > >> > > (AgentManager-Handler-12:null) (logid:) SeqA
> > > > >> 11-96232:
> > > > >> > > >> Sending
> > > > >> > > >> >>>> Seq
> > > > >> > > >> >>>> > >> > > 11-96232: { Ans: , MgmtId: 279278805450982,
> > via:
> > > > 11,
> > > > >> > Ver:
> > > > >> > > >> v1,
> > > > >> > > >> >>>> > Flags:
> > > > >> > > >> >>>> > >> > > 100010,
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> >
> > > > >> > > >>
> > > > >>
> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> > > > >> > > >> >>>> > >> }
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,210 DEBUG
> > > > >> > > >> [c.c.r.ResourceLimitManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Updating resource Type = memory count for
> > Account
> > > > = 2
> > > > >> > > >> >>>> Operation =
> > > > >> > > >> >>>> > >> > > decreasing Amount = 512
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,215 INFO
> > > > >> > > >> >>>> [o.a.c.a.c.a.v.DeployVMCmdByAdmin]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > >
> > > > >> com.cloud.exception.InsufficientServerCapacityException:
> > > > >> > > >> >>>> Unable to
> > > > >> > > >> >>>> > >> > create a
> > > > >> > > >> >>>> > >> > > deployment for
> > VM[User|i-2-68-VM]Scope=interface
> > > > >> > > >> >>>> > >> com.cloud.dc.DataCenter;
> > > > >> > > >> >>>> > >> > > id=1
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,215 INFO
> > > > >> > > >> >>>> [o.a.c.a.c.a.v.DeployVMCmdByAdmin]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508
> > > > >> ctx-b6238917)
> > > > >> > > >> >>>> > >> (logid:57397166)
> > > > >> > > >> >>>> > >> > > Unable to create a deployment for
> > > > VM[User|i-2-68-VM]
> > > > >> > > >> >>>> > >> > >
> > > > >> com.cloud.exception.InsufficientServerCapacityException:
> > > > >> > > >> >>>> Unable to
> > > > >> > > >> >>>> > >> > create a
> > > > >> > > >> >>>> > >> > > deployment for
> > VM[User|i-2-68-VM]Scope=interface
> > > > >> > > >> >>>> > >> com.cloud.dc.DataCenter;
> > > > >> > > >> >>>> > >> > > id=1
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.reserveVirtualMachine(VMEntityManagerImpl.java:215)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.reserve(VirtualMachineEntityImpl.java:200)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4492)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4057)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4044)
> > > > >> > > >> >>>> > >> > > at
> > > > >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > > >> > > >> Method)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > >> > > >> >>>> > >> > > at
> > > java.lang.reflect.Method.invoke(Method.java:498)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
> > > > >> > > >> >>>> > >> > > at
> > > > >> com.sun.proxy.$Proxy164.startVirtualMachine(Unknown
> > > > >> > > >> Source)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin.execute(DeployVMCmdByAdmin.java:50)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >>
> com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> >
> > > >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > >> > > >> >>>> > >> > > at
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > >> > > >> >>>> > >> > > at java.lang.Thread.run(Thread.java:748)
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,216 DEBUG
> > > > >> > > >> [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508)
> > > > >> > (logid:57397166)
> > > > >> > > >> >>>> Complete
> > > > >> > > >> >>>> > >> async
> > > > >> > > >> >>>> > >> > > job-508, jobStatus: FAILED, resultCode: 530,
> > > > result:
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> > >
> > > > >> > > >> >>>> > >> >
> > > > >> > > >> >>>> > >>
> > > > >> > > >> >>>> >
> > > > >> > > >> >>>>
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":533,"errortext":"Unable
> > > > >> > > >> >>>> > >> > > to create a deployment for
> VM[User|i-2-68-VM]"}
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,221 DEBUG
> > > > >> > > >> [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508)
> > > > >> > (logid:57397166)
> > > > >> > > >> >>>> Publish
> > > > >> > > >> >>>> > >> async
> > > > >> > > >> >>>> > >> > > job-508 complete on message bus
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,221 DEBUG
> > > > >> > > >> [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508)
> > > > >> > (logid:57397166)
> > > > >> > > >> >>>> Wake up
> > > > >> > > >> >>>> > >> jobs
> > > > >> > > >> >>>> > >> > > related to job-508
> > > > >> > > >> >>>> > >> > > 2019-03-09 11:01:35,221 DEBUG
> > > > >> > > >> [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > >> > > >> >>>> > >> > > (API-Job-Executor-1:ctx-0c22fa00 job-508)
> > > > >> > (logid:57397166)
> > > > >> > > >> >>>> Update db
> > > > >>
