weizhouapache commented on issue #4002: server: Search zone-wide storage pool when allocation algothrim is firstfitleastconsumed URL: https://github.com/apache/cloudstack/pull/4002#issuecomment-610005964 > @weizhouapache yes, tested in master and failed with this error: > > ``` > 2020-04-06 19:45:04,506 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7 FirstFitRoutingAllocator) (logid:bc36d8d1) Found a suitable host, adding to list: 1 > 2020-04-06 19:45:04,507 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7 FirstFitRoutingAllocator) (logid:bc36d8d1) Host Allocator returning 1 suitable hosts > 2020-04-06 19:45:04,510 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Checking suitable pools for volume (Id, Type): (129,ROOT) > 2020-04-06 19:45:04,510 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) We need to allocate new storagepool for this volume > 2020-04-06 19:45:04,510 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Calling StoragePoolAllocators to find suitable pools > 2020-04-06 19:45:04,511 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) LocalStoragePoolAllocator trying to find storage pool to fit the vm > 2020-04-06 19:45:04,511 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) ClusterScopeStoragePoolAllocator looking for storage pool > 2020-04-06 19:45:04,515 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Looking for pools in dc: 1 pod:1 cluster:1. Disabled pools will be ignored. > 2020-04-06 19:45:04,517 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Found pools matching tags: [] > 2020-04-06 19:45:04,517 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) No storage pools available for shared volume allocation, returning > 2020-04-06 19:45:04,518 DEBUG [o.a.c.s.a.ZoneWideStoragePoolAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) ZoneWideStoragePoolAllocator to find storage pool > 2020-04-06 19:45:04,521 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Checking if storage pool is suitable, name: null ,poolId: 2 > 2020-04-06 19:45:04,523 INFO [c.c.s.StorageManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Storage pool null (2) does not supply IOPS capacity, assuming enough capacity > 2020-04-06 19:45:04,531 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Checking pool 2 for storage, totalSize: 2164369915904, usedBytes: 1493502525440, usedPct: 0.6900403274253623, disable threshold: 0.85 > 2020-04-06 19:45:04,531 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Destination pool id: 2 > 2020-04-06 19:45:04,551 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Pool ID for the volume with ID 129 is null > 2020-04-06 19:45:04,554 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Found storage pool ps of type NetworkFilesystem with over-provisioning factor 2 > 2020-04-06 19:45:04,555 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Total over-provisioned capacity calculated is 2 * 2164369915904 > 2020-04-06 19:45:04,555 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Total capacity of the pool ps with ID 2 is 4328739831808 > 2020-04-06 19:45:04,559 DEBUG [c.c.s.StorageManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Checking pool: 2 for storage allocation , maxSize : 4328739831808, totalAllocatedSize : 12784435712, askingSize : 8589934592, allocated disable threshold: 0.85 > 2020-04-06 19:45:04,563 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) List of pools in descending order of free capacity: [] > 2020-04-06 19:45:04,563 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) No suitable pools found for volume: Vol[129|vm=123|ROOT] under cluster: 1 > 2020-04-06 19:45:04,563 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) No suitable pools found > 2020-04-06 19:45:04,563 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) No suitable storagePools found under this Cluster: 1 > 2020-04-06 19:45:04,567 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Could not find suitable Deployment Destination for this VM under any clusters, returning. > 2020-04-06 19:45:04,567 DEBUG [c.c.d.FirstFitPlanner] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Searching all possible resources under this Zone: 1 > 2020-04-06 19:45:04,568 DEBUG [c.c.d.FirstFitPlanner] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Listing clusters in order of aggregate capacity, that have (atleast one host with) enough CPU and RAM capacity under this Zone: 1 > 2020-04-06 19:45:04,569 DEBUG [c.c.d.FirstFitPlanner] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Removing from the clusterId list these clusters from avoid set: [1] > 2020-04-06 19:45:04,570 DEBUG [c.c.d.FirstFitPlanner] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) No clusters found after removing disabled clusters and clusters in avoid list, returning. > 2020-04-06 19:45:04,572 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Destroying vm VM[User|i-2-123-VM] as it failed to create on Host with Id:null > 2020-04-06 19:45:04,578 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) VM state transitted from :Stopped to Error with event: OperationFailedToErrorvm's original host id: null new host id: null host id before state transition: null > 2020-04-06 19:45:04,594 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Updating resource Type = volume count for Account = 2 Operation = decreasing Amount = 1 > 2020-04-06 19:45:04,601 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Updating resource Type = primary_storage count for Account = 2 Operation = decreasing Amount = 8589934592 > 2020-04-06 19:45:04,612 WARN [c.c.a.AlertManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) AlertType:: 8 | dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Failed to deploy Vm with Id: 123, on Host with Id: null > 2020-04-06 19:45:04,619 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Updating resource Type = user_vm count for Account = 2 Operation = decreasing Amount = 1 > 2020-04-06 19:45:04,625 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Updating resource Type = cpu count for Account = 2 Operation = decreasing Amount = 1 > 2020-04-06 19:45:04,632 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Updating resource Type = memory count for Account = 2 Operation = decreasing Amount = 512 > 2020-04-06 19:45:04,643 INFO [o.a.c.a.c.u.v.DeployVMCmd] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-123-VM]Scope=interface com.cloud.dc.DataCenter; id=1 > 2020-04-06 19:45:04,643 INFO [o.a.c.a.c.u.v.DeployVMCmd] (API-Job-Executor-1:ctx-acf00d62 job-1045 ctx-c26962c7) (logid:bc36d8d1) Unable to create a deployment for VM[User|i-2-123-VM] > com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-123-VM]Scope=interface com.cloud.dc.DataCenter; id=1 > at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.reserveVirtualMachine(VMEntityManagerImpl.java:225) > at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.reserve(VirtualMachineEntityImpl.java:202) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4766) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4315) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4305) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) > at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) > at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) > at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) > at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > at com.sun.proxy.$Proxy176.startVirtualMachine(Unknown Source) > at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:635) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156) > at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108) > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:586) > at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:534) > at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:834) > 2020-04-06 19:45:04,663 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-1:ctx-acf00d62 job-1045) (logid:bc36d8d1) Complete async job-1045, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":533,"errortext":"Unable to create a deployment for VM[User|i-2-123-VM]"} > ``` @nvazquez good. We got same issue and it's fixed by this PR.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
