Hi Slavka,
didn't tried to restart the SystemVMs. Reading through the docs i had the
impression that "live-patching" would be enough for this....
Anyway - I tried my luck with the console proxy.
Which failed utterly. Not starting as it isn't finding any useable storage
pool. - logs at the end of the mail.
I tried deploying a VM using the same storage pool which worked wonderfully.
There i noticed that all my custome storage-offerings are unuseable at the
moment. They are not accepting any custome storage sizes anymore...
Throwing failures "VM Creation failed. Volume size: -1GB is out of allowed
tange. Max:3072 Min:1"...
So this is also currently broken....
2022-08-29 10:51:56,189 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539) (logid:ba10fec5) Run
VM work job: com.cloud.vm.VmWorkStart for VM 83, job origin: 3445
2022-08-29 10:51:56,191 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Execute VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":83,"handlerName":"VirtualMachineManagerImpl"}
2022-08-29 10:51:56,192 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) orchestrating VM start for 'v-83-VM'
com.cloud.vm.VirtualMachineProfile$Param@b66cdd7d set to null
2022-08-29 10:51:56,198 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) VM instance {id: "83", name: "v-83-VM", uuid:
"b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} state
transited from [Stopped] to [Starting] with event [StartRequested]. VM's
original host: null, new host: null, host before state transition: null
2022-08-29 10:51:56,198 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Successfully transitioned to start state for VM instance
{id: "83", name: "v-83-VM", uuid: "b1d1f9c6-eafe-4527-9335-e649c646aab0",
type="ConsoleProxy"} reservation id = 6641ccc0-bcef-4b1a-a1f7-8dd8000626ad
2022-08-29 10:51:56,203 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Deploy avoids pods: null, clusters: null, hosts: null
2022-08-29 10:51:56,206 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) VM start attempt #1
2022-08-29 10:51:56,207 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) DeploymentPlanner allocation algorithm: null
2022-08-29 10:51:56,207 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Trying to allocate a host and storage pools from dc:1,
pod:null,cluster:null, requested cpu: 500, requested ram: (1.00 GB)
1073741824
2022-08-29 10:51:56,207 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Is ROOT volume READY (pool already allocated)?: No
2022-08-29 10:51:56,214 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Deploy avoids pods: [], clusters: [], hosts: null
2022-08-29 10:51:56,215 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Searching all possible resources under this Zone: 1
2022-08-29 10:51:56,216 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Listing clusters in order of aggregate capacity, that have
(at least one host with) enough CPU and RAM capacity under this Zone: 1
2022-08-29 10:51:56,219 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Removing from the clusterId list these clusters from avoid
set: []
2022-08-29 10:51:56,224 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Checking resources in Cluster: 1 under Pod: 1
2022-08-29 10:51:56,225 INFO [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Guest VM is requested with
Custom[UEFI] Boot Type false
2022-08-29 10:51:56,225 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Looking for hosts in dc: 1
pod:1 cluster:1
2022-08-29 10:51:56,227 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) FirstFitAllocator has 4 hosts to
check for allocation: [Host {"id": "7", "name": "srv-5-comp.blabla.blub",
"uuid": "8c753ce1-0253-49c9-9c7f-1f971a719ce6", "type"="Routing"}, Host
{"id": "5", "name": "srv-4-comp.blabla.blub", "uuid":
"134c625d-15ab-461a-a38d-36fb0b979b5d", "type"="Routing"}, Host {"id": "6",
"name": "srv-3-comp.blabla.blub", "uuid":
"6858713b-7e17-46d8-854d-24a13f9c7001", "type"="Routing"}, Host {"id": "1",
"name": "srv-2-comp.blabla.blub", "uuid":
"883f7f62-55f2-42f4-8242-16b6847c4204", "type"="Routing"}]
2022-08-29 10:51:56,233 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Found 4 hosts for allocation
after prioritization: [Host {"id": "7", "name": "srv-5-comp.blabla.blub",
"uuid": "8c753ce1-0253-49c9-9c7f-1f971a719ce6", "type"="Routing"}, Host
{"id": "5", "name": "srv-4-comp.blabla.blub", "uuid":
"134c625d-15ab-461a-a38d-36fb0b979b5d", "type"="Routing"}, Host {"id": "6",
"name": "srv-3-comp.blabla.blub", "uuid":
"6858713b-7e17-46d8-854d-24a13f9c7001", "type"="Routing"}, Host {"id": "1",
"name": "srv-2-comp.blabla.blub", "uuid":
"883f7f62-55f2-42f4-8242-16b6847c4204", "type"="Routing"}]
2022-08-29 10:51:56,233 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Looking for speed=500Mhz,
Ram=1024 MB
2022-08-29 10:51:56,233 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host {id: 7, name:
srv-5-comp.blabla.blub, uuid: 8c753ce1-0253-49c9-9c7f-1f971a719ce6} is KVM
hypervisor type, no max guest limit check needed
2022-08-29 10:51:56,235 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host: 7 has cpu capability
(cpu:48, speed:2650) to support requested CPU: 1 and requested speed: 500
2022-08-29 10:51:56,235 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Checking if host: 7 has enough
capacity for requested CPU: 500 and requested RAM: (1.00 GB) 1073741824 ,
cpuOverprovisioningFactor: 1.0
2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Hosts's actual total CPU: 127200
and CPU after applying overprovisioning: 127200
2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Free CPU: 37100 , Requested CPU:
500
2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Free RAM: (176.37 GB)
189373743104 , Requested RAM: (1.00 GB) 1073741824
2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host has enough CPU and RAM
available
2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc CPU from host:
7, used: 90100, reserved: 0, actual total: 127200, total with
overprovisioning: 127200; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc MEM from host:
7, used: (73.90 GB) 79352037376, reserved: (0 bytes) 0, total: (250.27 GB)
268725780480; requested mem: (1.00 GB) 1073741824, alloc_from_last_host?:
false , considerReservedCapacity?: true
2022-08-29 10:51:56,237 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Found a suitable host, adding to
list: 7
2022-08-29 10:51:56,237 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host {id: 5, name:
srv-4-comp.blabla.blub, uuid: 134c625d-15ab-461a-a38d-36fb0b979b5d} is KVM
hypervisor type, no max guest limit check needed
2022-08-29 10:51:56,239 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host: 5 has cpu capability
(cpu:48, speed:2650) to support requested CPU: 1 and requested speed: 500
2022-08-29 10:51:56,239 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Checking if host: 5 has enough
capacity for requested CPU: 500 and requested RAM: (1.00 GB) 1073741824 ,
cpuOverprovisioningFactor: 1.0
2022-08-29 10:51:56,240 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Hosts's actual total CPU: 127200
and CPU after applying overprovisioning: 127200
2022-08-29 10:51:56,240 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Free CPU: 116100 , Requested
CPU: 500
2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Free RAM: (233.77 GB)
251009040384 , Requested RAM: (1.00 GB) 1073741824
2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host has enough CPU and RAM
available
2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc CPU from host:
5, used: 11100, reserved: 0, actual total: 127200, total with
overprovisioning: 127200; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc MEM from host:
5, used: (16.50 GB) 17716740096, reserved: (0 bytes) 0, total: (250.27 GB)
268725780480; requested mem: (1.00 GB) 1073741824, alloc_from_last_host?:
false , considerReservedCapacity?: true
2022-08-29 10:51:56,241 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Found a suitable host, adding to
list: 5
2022-08-29 10:51:56,241 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host {id: 6, name:
srv-3-comp.blabla.blub, uuid: 6858713b-7e17-46d8-854d-24a13f9c7001} is KVM
hypervisor type, no max guest limit check needed
2022-08-29 10:51:56,243 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host: 6 has cpu capability
(cpu:48, speed:2650) to support requested CPU: 1 and requested speed: 500
2022-08-29 10:51:56,243 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Checking if host: 6 has enough
capacity for requested CPU: 500 and requested RAM: (1.00 GB) 1073741824 ,
cpuOverprovisioningFactor: 1.0
2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Hosts's actual total CPU: 127200
and CPU after applying overprovisioning: 127200
2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Free CPU: 84300 , Requested CPU:
500
2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Free RAM: (190.02 GB)
204032835584 , Requested RAM: (1.00 GB) 1073741824
2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host has enough CPU and RAM
available
2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc CPU from host:
6, used: 42900, reserved: 0, actual total: 127200, total with
overprovisioning: 127200; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc MEM from host:
6, used: (60.25 GB) 64692944896, reserved: (0 bytes) 0, total: (250.27 GB)
268725780480; requested mem: (1.00 GB) 1073741824, alloc_from_last_host?:
false , considerReservedCapacity?: true
2022-08-29 10:51:56,244 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Found a suitable host, adding to
list: 6
2022-08-29 10:51:56,244 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host {id: 1, name:
srv-2-comp.blabla.blub, uuid: 883f7f62-55f2-42f4-8242-16b6847c4204} is KVM
hypervisor type, no max guest limit check needed
2022-08-29 10:51:56,247 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host: 1 has cpu capability
(cpu:48, speed:2650) to support requested CPU: 1 and requested speed: 500
2022-08-29 10:51:56,247 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Checking if host: 1 has enough
capacity for requested CPU: 500 and requested RAM: (1.00 GB) 1073741824 ,
cpuOverprovisioningFactor: 1.0
2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Hosts's actual total CPU: 127200
and CPU after applying overprovisioning: 127200
2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Free CPU: 79000 , Requested CPU:
500
2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Free RAM: (214.02 GB)
229802639360 , Requested RAM: (1.00 GB) 1073741824
2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host has enough CPU and RAM
available
2022-08-29 10:51:56,248 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc CPU from host:
1, used: 48200, reserved: 0, actual total: 127200, total with
overprovisioning: 127200; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2022-08-29 10:51:56,249 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) STATS: Can alloc MEM from host:
1, used: (36.25 GB) 38923141120, reserved: (0 bytes) 0, total: (250.27 GB)
268725780480; requested mem: (1.00 GB) 1073741824, alloc_from_last_host?:
false , considerReservedCapacity?: true
2022-08-29 10:51:56,249 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Found a suitable host, adding to
list: 1
2022-08-29 10:51:56,249 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751
FirstFitRoutingAllocator) (logid:ba10fec5) Host Allocator returning 4
suitable hosts
2022-08-29 10:51:56,260 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) VM instance {id: "83", name: "v-83-VM", uuid:
"b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} state
transited from [Starting] to [Stopped] with event [OperationFailed]. VM's
original host: null, new host: null, host before state transition: null
2022-08-29 10:51:56,260 WARN [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Unable to orchestrate start VM instance {id: "83", name:
"v-83-VM", uuid: "b1d1f9c6-eafe-4527-9335-e649c646aab0",
type="ConsoleProxy"} due to [Unable to create deployment, no usable volumes
found for the VM: 83].
com.cloud.utils.exception.CloudRuntimeException: Unable to create
deployment, no usable volumes found for the VM: 83
at
com.cloud.deploy.DeploymentPlanningManagerImpl.findSuitablePoolsForVolumes(DeploymentPlanningManagerImpl.java:1569)
at
com.cloud.deploy.DeploymentPlanningManagerImpl.checkClustersforDestination(DeploymentPlanningManagerImpl.java:1217)
at
com.cloud.deploy.DeploymentPlanningManagerImpl.planDeployment(DeploymentPlanningManagerImpl.java:527)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1126)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
at jdk.internal.reflect.GeneratedMethodAccessor945.invoke(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-08-29 10:51:56,262 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Done executing VM work job:
com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":83,"handlerName":"VirtualMachineManagerImpl"}
2022-08-29 10:51:56,263 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Complete async job-3539, jobStatus: FAILED, resultCode: 0,
result:
rO0ABXNyAC9jb20uY2xvdWQudXRpbHMuZXhjZXB0aW9uLkNsb3VkUnVudGltZUV4Y2VwdGlvbgAAAABWTU9yAwABSQALY3NFcnJvckNvZGV4cgAaamF2YS5sYW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0fPho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAKc3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRFeGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0ANRVbmFibGUgdG8gb3JjaGVzdHJhdGUgc3RhcnQgVk0gaW5zdGFuY2Uge2lkOiAiODMiLCBuYW1lOiAidi04My1WTSIsIHV1aWQ6ICJiMWQxZjljNi1lYWZlLTQ1MjctOTMzNS1lNjQ5YzY0NmFhYjAiLCB0eXBlPSJDb25zb2xlUHJveHkifSBkdWUgdG8gW1VuYWJsZSB0byBjcmVhdGUgZGVwbG95bWVudCwgbm8gdXNhYmxlIHZvbHVtZXMgZm91bmQgZm9yIHRoZSBWTTogODNdLnVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAE3NyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVxAH4ABUwADmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfgAFTAAKbW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAAUx3QAA2FwcHQAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAP____9wdAAvamRrLmludGVybmFsLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3I5NDVwdAAGaW52b2tlcHBzcQB-AAwCAAAAK3B0ADFqZGsuaW50ZXJuYWwucmVmbGVjdC5EZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsdAAhRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbC5qYXZhcQB-ABR0AAlqYXZhLmJhc2V0AAcxMS4wLjE2c3EAfgAMAgAAAjZwdAAYamF2YS5sYW5nLnJlZmxlY3QuTWV0aG9kdAALTWV0aG9kLmphdmFxAH4AFHEAfgAYcQB-ABlzcQB-AAwBAAAAa3EAfgAOdAAiY29tLmNsb3VkLnZtLlZtV29ya0pvYkhhbmRsZXJQcm94eXQAGlZtV29ya0pvYkhhbmRsZXJQcm94eS5qYXZhdAAPaGFuZGxlVm1Xb3JrSm9icHBzcQB-AAwBAAAVP3EAfgAOcQB-AA9xAH4AEHEAfgAgcHBzcQB-AAwBAAAAZnEAfgAOdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JwcHNxAH4ADAEAAAJscQB-AA50AD9vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZnJhbWV3b3JrLmpvYnMuaW1wbC5Bc3luY0pvYk1hbmFnZXJJbXBsJDV0ABhBc3luY0pvYk1hbmFnZXJJbXBsLmphdmF0AAxydW5JbkNvbnRleHRwcHNxAH4ADAEAAAAwcQB-AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGUkMXQAG01hbmFnZWRDb250ZXh0UnVubmFibGUuamF2YXQAA3J1bnBwc3EAfgAMAQAAADdxAH4ADnQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHBwc3EAfgAMAQAAAGZxAH4ADnQAQG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHRxAH4AMHQAD2NhbGxXaXRoQ29udGV4dHBwc3EAfgAMAQAAADRxAH4ADnEAfgAzcQB-ADB0AA5ydW5XaXRoQ29udGV4dHBwc3EAfgAMAQAAAC1xAH4ADnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAscQB-AC1wcHNxAH4ADAEAAAI4cQB-AA5xAH4AJ3EAfgAocQB-AC1wcHNxAH4ADAIAAAIDcHQALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgAxcQB-ABhxAH4AGXNxAH4ADAIAAAEIcHQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4ALXEAfgAYcQB-ABlzcQB-AAwCAAAEaHB0ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnEAfgAYcQB-ABlzcQB-AAwCAAACdHB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEJxAH4ALXEAfgAYcQB-ABlzcQB-AAwCAAADPXB0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4ALXEAfgAYcQB-ABlzcgAfamF2YS51dGlsLkNvbGxlY3Rpb25zJEVtcHR5TGlzdHq4F7Q8p57eAgAAeHB4AAAQmncIAAAAAAAAAAB4
2022-08-29 10:51:56,264 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Publish async job-3539 complete on message bus
2022-08-29 10:51:56,264 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Wake up jobs related to job-3539
2022-08-29 10:51:56,264 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Update db status for job-3539
2022-08-29 10:51:56,266 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539 ctx-073f2751)
(logid:ba10fec5) Wake up jobs joined with job-3539 and disjoin all subjobs
created from job- 3539
2022-08-29 10:51:56,276 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539) (logid:ba10fec5) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 83, job origin:
3445
2022-08-29 10:51:56,276 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539) (logid:ba10fec5) Done
executing com.cloud.vm.VmWorkStart for job-3539
2022-08-29 10:51:56,278 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-58:ctx-dd8fe695 job-3445/job-3539) (logid:ba10fec5)
Remove job-3539 from job monitoring
2022-08-29 10:51:56,285 WARN [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-88c25b3a) (logid:d39faaa7) Unable to allocate console
proxy standby capacity for zone [1] due to [Unable to orchestrate start VM
instance {id: "83", name: "v-83-VM", uuid:
"b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} due to [Unable
to create deployment, no usable volumes found for the VM: 83].].
com.cloud.utils.exception.CloudRuntimeException: Unable to orchestrate
start VM instance {id: "83", name: "v-83-VM", uuid:
"b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} due to [Unable
to create deployment, no usable volumes found for the VM: 83].
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5319)
at jdk.internal.reflect.GeneratedMethodAccessor945.invoke(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-08-29 10:51:56,287 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-88c25b3a) (logid:d39faaa7) received console proxy alert
2022-08-29 10:51:56,288 DEBUG [c.c.a.ConsoleProxyAlertAdapter]
(consoleproxy-1:ctx-88c25b3a) (logid:d39faaa7) Console proxy creation
failure, zone: blabla
2022-08-29 10:51:56,289 WARN [c.c.a.AlertManagerImpl]
(consoleproxy-1:ctx-88c25b3a) (logid:d39faaa7) alertType=[10]
dataCenterId=[1] podId=[null] clusterId=[null] message=[Console proxy
creation failure. zone: blabla, error details: Unable to orchestrate start
VM instance {id: "83", name: "v-83-VM", uuid:
"b1d1f9c6-eafe-4527-9335-e649c646aab0", type="ConsoleProxy"} due to [Unable
to create deployment, no usable volumes found for the VM: 83].].
Am Mo., 29. Aug. 2022 um 09:40 Uhr schrieb Slavka Peleva
<slav...@storpool.com.invalid>:
> Hi Chris,
>
> Did you recreate the system VMs? In the 4.17 version, the `systemvm.iso` is
> deprecated. In its place, you should have the file on agents
> `/usr/share/cloudstack-common/vms/agent.zip`.
> Can you share the complete log if the system VMs are newly created after
> the upgrade?
>
> Best regards,
> Slavka
>
> On Sun, Aug 28, 2022 at 2:16 PM vas...@gmx.de <vas...@gmx.de> wrote:
>
> > Hi everyone,
> >
> > faceing some challanges again after upgradeing to 4.17.
> > Did as explained in the docs.
> >
> > Afterwards I am now not able to migrate system VMs to different hosts.
> > Getting the error
> > "Exception during migrate: org.libvirt.LibvirtException: Cannot access
> > storage file '/usr/share/cloudstack-common/vms/systemvm.iso': No such
> file
> > or directory"
> >
> > Checked the exitstence of the file on the servers:
> > Mangement-Server has a "systemvm.iso.bak"
> > Hosts - no "systemvm.iso" at all present.
> >
> > Maybe someone has an idea on how to move on with this?
> >
> > Regards,
> > Chris
> >
>
