Hi Chris, I mean to recreate the system VMs by destroying them.
On Mon, Aug 29, 2022 at 12:38 PM vas...@gmx.de <vas...@gmx.de> wrote: > Hi Slavka, > > didn't tried to restart the SystemVMs. Reading through the docs i had the > impression that "live-patching" would be enough for this.... > Anyway - I tried my luck with the console proxy. > > Which failed utterly. Not starting as it isn't finding any useable storage > pool. - logs at the end of the mail. > > I tried deploying a VM using the same storage pool which worked > wonderfully. > There i noticed that all my custome storage-offerings are unuseable at the > moment. They are not accepting any custome storage sizes anymore... > Throwing failures "VM Creation failed. Volume size: -1GB is out of allowed > tange. Max:3072 Min:1"... > So this is also currently broken.... > > 2022-08-29 10:51:56,189 DEBUG [c.c.v.VmWorkJobDispatcher] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539) (logid:ba10fec5) Run > VM work job: com.cloud.vm.VmWorkStart for VM 83, job origin: 3445 > 2022-08-29 10:51:56,191 DEBUG [c.c.v.VmWorkJobHandlerProxy] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Execute VM work job: > com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":83,"handlerName":"VirtualMachineManagerImpl"} > 2022-08-29 10:51:56,192 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) orchestrating VM start for 'v-83-VM' > com.cloud.vm.VirtualMachineProfile$Param@b66cdd7d set to null > 2022-08-29 10:51:56,198 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) VM instance {id: "83", name: "v-83-VM", uuid: > "b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} state > transited from [Stopped] to [Starting] with event [StartRequested]. VM's > original host: null, new host: null, host before state transition: null > 2022-08-29 10:51:56,198 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Successfully transitioned to start state for VM instance > {id: "83", name: "v-83-VM", uuid: "b1d1f9c6-eafe-4527-9335-e649c646aab0", > type="ConsoleProxy"} reservation id = 6641ccc0-bcef-4b1a-a1f7-8dd8000626ad > 2022-08-29 10:51:56,203 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Deploy avoids pods: null, clusters: null, hosts: null > 2022-08-29 10:51:56,206 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) VM start attempt #1 > 2022-08-29 10:51:56,207 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) DeploymentPlanner allocation algorithm: null > 2022-08-29 10:51:56,207 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Trying to allocate a host and storage pools from dc:1, > pod:null,cluster:null, requested cpu: 500, requested ram: (1.00 GB) > 1073741824 > 2022-08-29 10:51:56,207 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Is ROOT volume READY (pool already allocated)?: No > 2022-08-29 10:51:56,214 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Deploy avoids pods: [], clusters: [], hosts: null > 2022-08-29 10:51:56,215 DEBUG [c.c.d.FirstFitPlanner] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Searching all possible resources under this Zone: 1 > 2022-08-29 10:51:56,216 DEBUG [c.c.d.FirstFitPlanner] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Listing clusters in order of aggregate capacity, that have > (at least one host with) enough CPU and RAM capacity under this Zone: 1 > 2022-08-29 10:51:56,219 DEBUG [c.c.d.FirstFitPlanner] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Removing from the clusterId list these clusters from avoid > set: [] > 2022-08-29 10:51:56,224 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Checking resources in Cluster: 1 under Pod: 1 > 2022-08-29 10:51:56,225 INFO [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Guest VM is requested with > Custom[UEFI] Boot Type false > 2022-08-29 10:51:56,225 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Looking for hosts in dc: 1 > pod:1 cluster:1 > 2022-08-29 10:51:56,227 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) FirstFitAllocator has 4 hosts to > check for allocation: [Host {"id": "7", "name": "srv-5-comp.blabla.blub", > "uuid": "8c753ce1-0253-49c9-9c7f-1f971a719ce6", "type"="Routing"}, Host > {"id": "5", "name": "srv-4-comp.blabla.blub", "uuid": > "134c625d-15ab-461a-a38d-36fb0b979b5d", "type"="Routing"}, Host {"id": "6", > "name": "srv-3-comp.blabla.blub", "uuid": > "6858713b-7e17-46d8-854d-24a13f9c7001", "type"="Routing"}, Host {"id": "1", > "name": "srv-2-comp.blabla.blub", "uuid": > "883f7f62-55f2-42f4-8242-16b6847c4204", "type"="Routing"}] > 2022-08-29 10:51:56,233 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Found 4 hosts for allocation > after prioritization: [Host {"id": "7", "name": "srv-5-comp.blabla.blub", > "uuid": "8c753ce1-0253-49c9-9c7f-1f971a719ce6", "type"="Routing"}, Host > {"id": "5", "name": "srv-4-comp.blabla.blub", "uuid": > "134c625d-15ab-461a-a38d-36fb0b979b5d", "type"="Routing"}, Host {"id": "6", > "name": "srv-3-comp.blabla.blub", "uuid": > "6858713b-7e17-46d8-854d-24a13f9c7001", "type"="Routing"}, Host {"id": "1", > "name": "srv-2-comp.blabla.blub", "uuid": > "883f7f62-55f2-42f4-8242-16b6847c4204", "type"="Routing"}] > 2022-08-29 10:51:56,233 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Looking for speed=500Mhz, > Ram=1024 MB > 2022-08-29 10:51:56,233 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host {id: 7, name: > srv-5-comp.blabla.blub, uuid: 8c753ce1-0253-49c9-9c7f-1f971a719ce6} is KVM > hypervisor type, no max guest limit check needed > 2022-08-29 10:51:56,235 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host: 7 has cpu capability > (cpu:48, speed:2650) to support requested CPU: 1 and requested speed: 500 > 2022-08-29 10:51:56,235 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Checking if host: 7 has enough > capacity for requested CPU: 500 and requested RAM: (1.00 GB) 1073741824 , > cpuOverprovisioningFactor: 1.0 > 2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Hosts's actual total CPU: 127200 > and CPU after applying overprovisioning: 127200 > 2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Free CPU: 37100 , Requested CPU: > 500 > 2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Free RAM: (176.37 GB) > 189373743104 , Requested RAM: (1.00 GB) 1073741824 > 2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host has enough CPU and RAM > available > 2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc CPU from host: > 7, used: 90100, reserved: 0, actual total: 127200, total with > overprovisioning: 127200; requested cpu:500,alloc_from_last_host?:false > ,considerReservedCapacity?: true > 2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc MEM from host: > 7, used: (73.90 GB) 79352037376, reserved: (0 bytes) 0, total: (250.27 GB) > 268725780480; requested mem: (1.00 GB) 1073741824, alloc_from_last_host?: > false , considerReservedCapacity?: true > 2022-08-29 10:51:56,237 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Found a suitable host, adding to > list: 7 > 2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host {id: 5, name: > srv-4-comp.blabla.blub, uuid: 134c625d-15ab-461a-a38d-36fb0b979b5d} is KVM > hypervisor type, no max guest limit check needed > 2022-08-29 10:51:56,239 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host: 5 has cpu capability > (cpu:48, speed:2650) to support requested CPU: 1 and requested speed: 500 > 2022-08-29 10:51:56,239 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Checking if host: 5 has enough > capacity for requested CPU: 500 and requested RAM: (1.00 GB) 1073741824 , > cpuOverprovisioningFactor: 1.0 > 2022-08-29 10:51:56,240 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Hosts's actual total CPU: 127200 > and CPU after applying overprovisioning: 127200 > 2022-08-29 10:51:56,240 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Free CPU: 116100 , Requested > CPU: 500 > 2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Free RAM: (233.77 GB) > 251009040384 , Requested RAM: (1.00 GB) 1073741824 > 2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host has enough CPU and RAM > available > 2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc CPU from host: > 5, used: 11100, reserved: 0, actual total: 127200, total with > overprovisioning: 127200; requested cpu:500,alloc_from_last_host?:false > ,considerReservedCapacity?: true > 2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc MEM from host: > 5, used: (16.50 GB) 17716740096, reserved: (0 bytes) 0, total: (250.27 GB) > 268725780480; requested mem: (1.00 GB) 1073741824, alloc_from_last_host?: > false , considerReservedCapacity?: true > 2022-08-29 10:51:56,241 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Found a suitable host, adding to > list: 5 > 2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host {id: 6, name: > srv-3-comp.blabla.blub, uuid: 6858713b-7e17-46d8-854d-24a13f9c7001} is KVM > hypervisor type, no max guest limit check needed > 2022-08-29 10:51:56,243 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host: 6 has cpu capability > (cpu:48, speed:2650) to support requested CPU: 1 and requested speed: 500 > 2022-08-29 10:51:56,243 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Checking if host: 6 has enough > capacity for requested CPU: 500 and requested RAM: (1.00 GB) 1073741824 , > cpuOverprovisioningFactor: 1.0 > 2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Hosts's actual total CPU: 127200 > and CPU after applying overprovisioning: 127200 > 2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Free CPU: 84300 , Requested CPU: > 500 > 2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Free RAM: (190.02 GB) > 204032835584 , Requested RAM: (1.00 GB) 1073741824 > 2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host has enough CPU and RAM > available > 2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc CPU from host: > 6, used: 42900, reserved: 0, actual total: 127200, total with > overprovisioning: 127200; requested cpu:500,alloc_from_last_host?:false > ,considerReservedCapacity?: true > 2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc MEM from host: > 6, used: (60.25 GB) 64692944896, reserved: (0 bytes) 0, total: (250.27 GB) > 268725780480; requested mem: (1.00 GB) 1073741824, alloc_from_last_host?: > false , considerReservedCapacity?: true > 2022-08-29 10:51:56,244 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Found a suitable host, adding to > list: 6 > 2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host {id: 1, name: > srv-2-comp.blabla.blub, uuid: 883f7f62-55f2-42f4-8242-16b6847c4204} is KVM > hypervisor type, no max guest limit check needed > 2022-08-29 10:51:56,247 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host: 1 has cpu capability > (cpu:48, speed:2650) to support requested CPU: 1 and requested speed: 500 > 2022-08-29 10:51:56,247 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Checking if host: 1 has enough > capacity for requested CPU: 500 and requested RAM: (1.00 GB) 1073741824 , > cpuOverprovisioningFactor: 1.0 > 2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Hosts's actual total CPU: 127200 > and CPU after applying overprovisioning: 127200 > 2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Free CPU: 79000 , Requested CPU: > 500 > 2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Free RAM: (214.02 GB) > 229802639360 , Requested RAM: (1.00 GB) 1073741824 > 2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host has enough CPU and RAM > available > 2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc CPU from host: > 1, used: 48200, reserved: 0, actual total: 127200, total with > overprovisioning: 127200; requested cpu:500,alloc_from_last_host?:false > ,considerReservedCapacity?: true > 2022-08-29 10:51:56,249 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc MEM from host: > 1, used: (36.25 GB) 38923141120, reserved: (0 bytes) 0, total: (250.27 GB) > 268725780480; requested mem: (1.00 GB) 1073741824, alloc_from_last_host?: > false , considerReservedCapacity?: true > 2022-08-29 10:51:56,249 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Found a suitable host, adding to > list: 1 > 2022-08-29 10:51:56,249 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751 > FirstFitRoutingAllocator) (logid:ba10fec5) Host Allocator returning 4 > suitable hosts > 2022-08-29 10:51:56,260 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) VM instance {id: "83", name: "v-83-VM", uuid: > "b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} state > transited from [Starting] to [Stopped] with event [OperationFailed]. VM's > original host: null, new host: null, host before state transition: null > 2022-08-29 10:51:56,260 WARN [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Unable to orchestrate start VM instance {id: "83", name: > "v-83-VM", uuid: "b1d1f9c6-eafe-4527-9335-e649c646aab0", > type="ConsoleProxy"} due to [Unable to create deployment, no usable volumes > found for the VM: 83]. > com.cloud.utils.exception.CloudRuntimeException: Unable to create > deployment, no usable volumes found for the VM: 83 > at > com.cloud.deploy.DeploymentPlanningManagerImpl.findSuitablePoolsForVolumes(DeploymentPlanningManagerImpl.java:1569) > at > com.cloud.deploy.DeploymentPlanningManagerImpl.checkClustersforDestination(DeploymentPlanningManagerImpl.java:1217) > at > com.cloud.deploy.DeploymentPlanningManagerImpl.planDeployment(DeploymentPlanningManagerImpl.java:527) > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1126) > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315) > at jdk.internal.reflect.GeneratedMethodAccessor945.invoke(Unknown Source) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) > at > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439) > at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568) > at > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:829) > 2022-08-29 10:51:56,262 DEBUG [c.c.v.VmWorkJobHandlerProxy] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Done executing VM work job: > com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":83,"handlerName":"VirtualMachineManagerImpl"} > 2022-08-29 10:51:56,263 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Complete async job-3539, jobStatus: FAILED, resultCode: 0, > result: > rO0ABXNyAC9jb20uY2xvdWQudXRpbHMuZXhjZXB0aW9uLkNsb3VkUnVudGltZUV4Y2VwdGlvbgAAAABWTU9yAwABSQALY3NFcnJvckNvZGV4cgAaamF2YS5sYW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0fPho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAKc3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRFeGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0ANRVbmFibGUgdG8gb3JjaGVzdHJhdGUgc3RhcnQgVk0gaW5zdGFuY2Uge2lkOiAiODMiLCBuYW1lOiAidi04My1WTSIsIHV1aWQ6ICJiMWQxZjljNi1lYWZlLTQ1MjctOTMzNS1lNjQ5YzY0NmFhYjAiLCB0eXBlPSJDb25zb2xlUHJveHkifSBkdWUgdG8gW1VuYWJsZSB0byBjcmVhdGUgZGVwbG95bWVudCwgbm8gdXNhYmxlIHZvbHVtZXMgZm91bmQgZm9yIHRoZSBWTTogODNdLnVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAE3NyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH4ABUwADmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfgAFTAAKbW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAAUx3QAA2FwcHQAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAP____9wdAAvamRrLmludGVybmFsLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3I5NDVwdAAGaW52b2tlcHBzcQB-AAwCAAAAK3B0ADFqZGsuaW50ZXJuYWwucmVmbGVjdC5EZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsdAAhRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbC5qYXZhcQB-ABR0AAlqYXZhLmJhc2V0AAcxMS4wLjE2c3EAfgAMAgAAAjZwdAAYamF2YS5sYW5nLnJlZmxlY3QuTWV0aG9kdAALTWV0aG9kLmphdmFxAH4AFHEAfgAYcQB-ABlzcQB-AAwBAAAAa3EAfgAOdAAiY29tLmNsb3VkLnZtLlZtV29ya0pvYkhhbmRsZXJQcm94eXQAGlZtV29ya0pvYkhhbmRsZXJQcm94eS5qYXZhdAAPaGFuZGxlVm1Xb3JrSm9icHBzcQB-AAwBAAAVP3EAfgAOcQB-AA9xAH4AEHEAfgAgcHBzcQB-AAwBAAAAZnEAfgAOdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JwcHNxAH4ADAEAAAJscQB-AA50AD9vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZnJhbWV3b3JrLmpvYnMuaW1wbC5Bc3luY0pvYk1hbmFnZXJJbXBsJDV0ABhBc3luY0pvYk1hbmFnZXJJbXBsLmphdmF0AAxydW5JbkNvbnRleHRwcHNxAH4ADAEAAAAwcQB-AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGUkMXQAG01hbmFnZWRDb250ZXh0UnVubmFibGUuamF2YXQAA3J1bnBwc3EAfgAMAQAAADdxAH4ADnQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHBwc3EAfgAMAQAAAGZxAH4ADnQAQG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHRxAH4AMHQAD2NhbGxXaXRoQ29udGV4dHBwc3EAfgAMAQAAADRxAH4ADnEAfgAzcQB-ADB0AA5ydW5XaXRoQ29udGV4dHBwc3EAfgAMAQAAAC1xAH4ADnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAscQB-AC1wcHNxAH4ADAEAAAI4cQB-AA5xAH4AJ3EAfgAocQB-AC1wcHNxAH4ADAIAAAIDcHQALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgAxcQB-ABhxAH4AGXNxAH4ADAIAAAEIcHQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4ALXEAfgAYcQB-ABlzcQB-AAwCAAAEaHB0ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnEAfgAYcQB-ABlzcQB-AAwCAAACdHB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEJxAH4ALXEAfgAYcQB-ABlzcQB-AAwCAAADPXB0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4ALXEAfgAYcQB-ABlzcgAfamF2YS51dGlsLkNvbGxlY3Rpb25zJEVtcHR5TGlzdHq4F7Q8p57eAgAAeHB4AAAQmncIAAAAAAAAAAB4 > 2022-08-29 10:51:56,264 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Publish async job-3539 complete on message bus > 2022-08-29 10:51:56,264 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Wake up jobs related to job-3539 > 2022-08-29 10:51:56,264 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Update db status for job-3539 > 2022-08-29 10:51:56,266 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751) > (logid:ba10fec5) Wake up jobs joined with job-3539 and disjoin all subjobs > created from job- 3539 > 2022-08-29 10:51:56,276 DEBUG [c.c.v.VmWorkJobDispatcher] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539) (logid:ba10fec5) Done > with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job origin: > 3445 > 2022-08-29 10:51:56,276 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539) (logid:ba10fec5) Done > executing com.cloud.vm.VmWorkStart for job-3539 > 2022-08-29 10:51:56,278 INFO [o.a.c.f.j.i.AsyncJobMonitor] > (Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539) (logid:ba10fec5) > Remove job-3539 from job monitoring > 2022-08-29 10:51:56,285 WARN [c.c.c.ConsoleProxyManagerImpl] > (consoleproxy-1:ctx-88c25b3a) (logid:d39faaa7) Unable to allocate console > proxy standby capacity for zone [1] due to [Unable to orchestrate start VM > instance {id: "83", name: "v-83-VM", uuid: > "b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} due to [Unable > to create deployment, no usable volumes found for the VM: 83].]. > com.cloud.utils.exception.CloudRuntimeException: Unable to orchestrate > start VM instance {id: "83", name: "v-83-VM", uuid: > "b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} due to [Unable > to create deployment, no usable volumes found for the VM: 83]. > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5319) > at jdk.internal.reflect.GeneratedMethodAccessor945.invoke(Unknown Source) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) > at > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439) > at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568) > at > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:829) > 2022-08-29 10:51:56,287 DEBUG [c.c.a.ConsoleProxyAlertAdapter] > (consoleproxy-1:ctx-88c25b3a) (logid:d39faaa7) received console proxy alert > 2022-08-29 10:51:56,288 DEBUG [c.c.a.ConsoleProxyAlertAdapter] > (consoleproxy-1:ctx-88c25b3a) (logid:d39faaa7) Console proxy creation > failure, zone: blabla > 2022-08-29 10:51:56,289 WARN [c.c.a.AlertManagerImpl] > (consoleproxy-1:ctx-88c25b3a) (logid:d39faaa7) alertType=[10] > dataCenterId=[1] podId=[null] clusterId=[null] message=[Console proxy > creation failure. zone: blabla, error details: Unable to orchestrate start > VM instance {id: "83", name: "v-83-VM", uuid: > "b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} due to [Unable > to create deployment, no usable volumes found for the VM: 83].]. > > > > > Am Mo., 29. Aug. 2022 um 09:40 Uhr schrieb Slavka Peleva > <slav...@storpool.com.invalid>: > >> Hi Chris, >> >> Did you recreate the system VMs? In the 4.17 version, the `systemvm.iso` >> is >> deprecated. In its place, you should have the file on agents >> `/usr/share/cloudstack-common/vms/agent.zip`. >> Can you share the complete log if the system VMs are newly created after >> the upgrade? >> >> Best regards, >> Slavka >> >> On Sun, Aug 28, 2022 at 2:16 PM vas...@gmx.de <vas...@gmx.de> wrote: >> >> > Hi everyone, >> > >> > faceing some challanges again after upgradeing to 4.17. >> > Did as explained in the docs. >> > >> > Afterwards I am now not able to migrate system VMs to different hosts. >> > Getting the error >> > "Exception during migrate: org.libvirt.LibvirtException: Cannot access >> > storage file '/usr/share/cloudstack-common/vms/systemvm.iso': No such >> file >> > or directory" >> > >> > Checked the exitstence of the file on the servers: >> > Mangement-Server has a "systemvm.iso.bak" >> > Hosts - no "systemvm.iso" at all present. >> > >> > Maybe someone has an idea on how to move on with this? >> > >> > Regards, >> > Chris >> > >> >