I think `this QEMU binary` is the key phrase here, Piotr. On Tue, Nov 23, 2021 at 9:51 AM Piotr Pisz <pi...@piszki.pl> wrote:
> Hi Wei, > > Unfortunately, neither systemvm, nor any other vm, is starting due to the > message: libvirt.LibvirtException: unsupported configuration: io uring is > not supported by this QEMU binary > > In one LAB I have a custom kernel: 5.4.129-1.el8.elrepo.x86_64 and > libvirtd (libvirt) 7.6.0 with qemu 6.0.0 (CentOS 8.5 with advanced > virtualization repo) > > But in the second, it comes with the system: 4.18.0-240.22.1.el8_3.x86_64 > and libvirtd (libvirt) 7.0.0 with qemu-kvm-5.2.0-16.el8 (CentOS 8.2), error > is in both > > IO uring works on host: > > root@psc1 /h/piotr# grep io_uring_setup /proc/kallsyms > ffffffffa274a090 t io_uring_setup > ffffffffa274a150 T __x64_sys_io_uring_setup > ffffffffa274a170 T __ia32_sys_io_uring_setup > ffffffffa38ff720 d event_exit__io_uring_setup > ffffffffa38ff7c0 d event_enter__io_uring_setup > ffffffffa38ff860 d __syscall_meta__io_uring_setup > ffffffffa38ff8a0 d args__io_uring_setup > ffffffffa38ff8b0 d types__io_uring_setup > ffffffffa3fd8658 t __event_exit__io_uring_setup > ffffffffa3fd8660 t __event_enter__io_uring_setup > ffffffffa3fda4b0 t __p_syscall_meta__io_uring_setup > ffffffffa3fddd20 t _eil_addr___ia32_sys_io_uring_setup > ffffffffa3fddd30 t _eil_addr___x64_sys_io_uring_setup > > I don't know what to do with it yet. > > Regards, > Piotr > > > > -----Original Message----- > From: Wei ZHOU <ustcweiz...@gmail.com> > Sent: Tuesday, November 23, 2021 9:13 AM > To: users <users@cloudstack.apache.org>; pi...@piszki.pl > Subject: Re: configdrive not working in 4.16 > > Hi Piotr, > > Could you please check if all your management servers, cloudstack agents, > and system vms (SSVM/CPVM) are running with 4.16.0.0 ? > It's mostly like your SSVM is not 4.16.0.0. > When processing HandleConfigDriveIsoCommand, SSVM with the old version > returns Answer (not HandleConfigDriveIsoAnswer). > > -Wei > > > On Mon, 22 Nov 2021 at 10:25, Piotr Pisz <pi...@piszki.pl> wrote: > > > Hi Daan, > > > > Unfortunately, I couldn't deal with this problem and withdrawn the > changes. > > I have two LAB environments, both behaved the same. In one, MGMT is > > installed on Ubuntu 20, in the other, on CentOS 8, in both cases the > > KVM is CentOS 8.4/8.5 > > > > On CentOS 8: > > openjdk version "11.0.13" 2021-10-19 LTS OpenJDK Runtime Environment > > 18.9 (build 11.0.13+8-LTS) OpenJDK 64-Bit Server VM 18.9 (build > > 11.0.13+8-LTS, mixed mode, sharing) > > > > Packages downloaded from official page. > > > > I also found a problem with tags, after the upgrade I was not able to > > add new tags with UI, old tags (added with legacy client) in primary / > > secondary storage were not displayed correctly. > > Right now I am building a LAB with version 4.16, I need a clean DB to > > compare. > > > > Regards, > > Piotr > > > > > > -----Original Message----- > > From: Daan Hoogland <daan.hoogl...@gmail.com> > > Sent: Monday, November 22, 2021 9:17 AM > > To: users <users@cloudstack.apache.org>; pi...@piszki.pl > > Subject: Re: configdrive not working in 4.16 > > > > Piotr, > > have you gotten anywhere with this? > > the IPMI exception seems unrelated, > > and the class cast exceptions seem to me to be a java version issue? > > can you give the versions of relevant platform version? (at least os, > > jvm) Have you built yourself or downloaded any package? > > > > On Wed, Nov 17, 2021 at 3:27 PM Piotr Pisz <pi...@piszki.pl> wrote: > > > > > > > > > > > Hi, > > > > > > I have upgraded to 4.16 and am having trouble booting my VM on L2 > > > network with ConfigDrive. I would have a request for help. > > > > > > > > > > > > Regards, > > > > > > Piotr > > > > > > > > > > > > > > > > > > > > > > > > 2021-11-17 14:47:27,229 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after > > > applying > > > overprovisioning: 31200 > > > > > > 2021-11-17 14:47:27,230 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) We are allocating VM, increasing the used capacity > > > of this > > > host:1 > > > > > > 2021-11-17 14:47:27,231 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Current Used CPU: 0 , Free CPU:27300 ,Requested CPU: > > > 3900 > > > > > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Current Used RAM: (0 bytes) 0 , Free RAM:(57,66 GB) > > > 61916983296 ,Requested RAM: (4,00 GB) 4294967296 > > > > > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) We are allocating VM to the last host again, so > > > adjusting the reserved capacity if it is not less than required > > > > > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900 > > > > > > 2021-11-17 14:47:27,232 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM: > > > (4,00 > > > GB) 4294967296 > > > > > > 2021-11-17 14:47:27,236 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) CPU STATS after allocation: for host: 1, old used: > > > 0, old > > > reserved: 3900, actual total: 31200, total with overprovisioning: > > > 31200; new used:3900, reserved:0; requested > > > cpu:3900,alloc_from_last:true > > > > > > 2021-11-17 14:47:27,241 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) RAM STATS after allocation: for host: 1, old used: > > > (0 > > > bytes) 0, old reserved: (4,00 GB) 4294967296, total: (61,66 GB) > > > 66211950592; new used: (4,00 GB) 4294967296, reserved: (0 bytes) 0; > > > requested mem: (4,00 > > > GB) 4294967296,alloc_from_last:true > > > > > > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Host: 1 has cpu capability (cpu:8, speed:3900) to > > > support requested CPU: 1 and requested speed: 3900 > > > > > > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Checking if host: 1 has enough capacity for > > > requested > > CPU: > > > 3900 and requested RAM: (4,00 GB) 4294967296 , > cpuOverprovisioningFactor: > > > 1.0 > > > > > > 2021-11-17 14:47:27,243 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after > > > applying > > > overprovisioning: 31200 > > > > > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) We need to allocate to the last host again, so > > > checking if there is enough reserved capacity > > > > > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Reserved CPU: 3900 , Requested CPU: 3900 > > > > > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Reserved RAM: (4,00 GB) 4294967296 , Requested RAM: > > > (4,00 > > > GB) 4294967296 > > > > > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Host has enough CPU and RAM available > > > > > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) STATS: Can alloc CPU from host: 1, used: 0, reserved: > > > 3900, > > > actual total: 31200, total with overprovisioning: 31200; requested > > > cpu:3900,alloc_from_last_host?:true ,considerReservedCapacity?: true > > > > > > 2021-11-17 14:47:27,244 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) STATS: Can alloc MEM from host: 1, used: (0 bytes) > > > 0, > > > reserved: (4,00 GB) 4294967296, total: (61,66 GB) 66211950592; > > > requested > > > mem: (4,00 GB) 4294967296, alloc_from_last_host?: true , > > > considerReservedCapacity?: true > > > > > > 2021-11-17 14:47:27,258 DEBUG [o.a.c.e.o.NetworkOrchestrator] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Network id=212 is already implemented > > > > > > 2021-11-17 14:47:27,275 DEBUG [c.c.n.NetworkModelImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Service SecurityGroup is not supported in the > > > network > > > id=212 > > > > > > 2021-11-17 14:47:27,279 DEBUG [o.a.c.e.o.NetworkOrchestrator] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Changing active number of nics for network id=212 > > > on > > > 1 > > > > > > 2021-11-17 14:47:27,294 DEBUG [o.a.c.e.o.NetworkOrchestrator] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Asking ConfigDrive to prepare for > > > Nic[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.243] > > > > > > 2021-11-17 14:47:27,303 DEBUG [c.c.s.StatsCollector] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 > > > TB], > > > used=[1 TB], threshold=[89.99999761581421%]. > > > > > > 2021-11-17 14:47:27,304 DEBUG [c.c.s.StatsCollector] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 > > > TB], > > > used=[1 TB], threshold=[89.99999761581421%]. > > > > > > 2021-11-17 14:47:27,306 DEBUG [c.c.n.e.ConfigDriveNetworkElement] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Creating config drive ISO for vm: i-6-83-VM > > > > > > 2021-11-17 14:47:27,309 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=service-offering, content=KVM-RBD-Custom > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=availability-zone, content=Piszki Lab > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=local-hostname, content=Brooklyn > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=local-ipv4, content=192.168.40.243 > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=public-ipv4, content=192.168.40.243 > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=public-hostname, content=null > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=instance-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=vm-id, content=6b236c6c-4cae-465b-b1e7-dfeef70d34cb > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=public-keys, content=null > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=cloud-identifier, > > > content=CloudStack-{f9bacf5f-af20-4116-8068-7d6a7fc92e26} > > > > > > 2021-11-17 14:47:27,310 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) [createConfigDriveIsoForVM] dataType=metadata, > > > filename=hypervisor-host-name, content=psc1.piszki.lab > > > > > > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Executing config drive creation command: > > > /usr/bin/genisoimage -o > > > /tmp/configdrive12981467783887013707/i-6-83-VM.iso > > > -ldots -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J > > > -r -V > > > config-2 /tmp/configdrive12981467783887013707 > > > > > > 2021-11-17 14:47:27,311 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Executing: /usr/bin/genisoimage -o > > > /tmp/configdrive12981467783887013707/i-6-83-VM.iso -ldots > > > -allow-lowercase -allow-multidot -cache-inodes -l -quiet -J -r -V > > > config-2 > > > /tmp/configdrive12981467783887013707 > > > > > > 2021-11-17 14:47:27,312 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Executing while with timeout : 300000 > > > > > > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Execution is successful. > > > > > > 2021-11-17 14:47:27,314 DEBUG [o.a.c.s.c.ConfigDriveBuilder] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Warning: creating filesystem that does not conform > > > to ISO-9660. > > > > > > > > > > > > 2021-11-17 14:47:27,334 DEBUG [c.c.a.t.Request] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Seq 11-7692992588478939146: Sending { Cmd , MgmtId: > > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011, > > > > > > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"conf > > > ig > > > drive/ > > > > > > i-6-83-VM.iso","create":"true","destStore":{"com.cloud.agent.api.to. > > > Nf > > > sTO":{ > > > > > > "_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOnU > > > ns > > > upport > > > > > > > > > edPool":"true","preferHostCache":"false","wait":"0","bypassHostMaintenance": > > > "false"}}] } > > > > > > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request] > > > (AgentManager-Handler-5:null) (logid:) Seq 11-7692992588478939146: > > > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1, > > > Flags: 10, > > > [{"com.cloud.agent.api.Answer":{"result":"true","details":"Successfu > > > ll > > > y > > > saved config drive at secondary > > > storage","wait":"0","bypassHostMaintenance":"false"}}] } > > > > > > 2021-11-17 14:47:27,450 DEBUG [c.c.a.t.Request] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Seq 11-7692992588478939146: Received: { Ans: , > MgmtId: > > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } } > > > > > > 2021-11-17 14:47:27,450 DEBUG [c.c.a.m.AgentManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Details from executing class > > > com.cloud.agent.api.HandleConfigDriveIsoCommand: Successfully saved > > > config drive at secondary storage > > > > > > 2021-11-17 14:47:27,460 DEBUG [c.c.v.VirtualMachineManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Cleaning up resources for the vm VM instance {id: > > > "83", > > > name: "i-6-83-VM", uuid: "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", > > > type="User"} in Starting state > > > > > > 2021-11-17 14:47:27,467 DEBUG [c.c.a.t.Request] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Seq 1-8430738502437568545: Sending { Cmd , MgmtId: > > > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 100011, > > > > > > > > > [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup": > > > > > > "false","forceStop":"false","vlanToPersistenceMap":{"40":"true"},"vo > > > lu > > > mesToD > > > > > > isconnect":[],"vmName":"i-6-83-VM","executeInSequence":"false","wait": > > > "0","b > > > ypassHostMaintenance":"false"}}] } > > > > > > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request] > > > (AgentManager-Handler-6:null) (logid:) Seq 1-843073850c2437568545: > > > Processing: { Ans: , MgmtId: 260497154630684, via: 1, Ver: v1, Flags: > > > 10, > > > > > > [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypa > > > ss > > > HostMa > > > intenance":"false"}}] } > > > > > > 2021-11-17 14:47:27,650 DEBUG [c.c.a.t.Request] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Seq 1-8430738502437568545: Received: { Ans: , MgmtId: > > > 260497154630684, via: 1(psc1.piszki.lab), Ver: v1, Flags: 10, { > > > StopAnswer } } > > > > > > 2021-11-17 14:47:27,655 DEBUG [c.c.n.NetworkModelImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Service SecurityGroup is not supported in the > > > network > > > id=212 > > > > > > 2021-11-17 14:47:27,658 DEBUG [o.a.c.e.o.NetworkOrchestrator] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Changing active number of nics for network id=212 > > > on > > > -1 > > > > > > 2021-11-17 14:47:27,671 DEBUG [o.a.c.e.o.NetworkOrchestrator] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Asking ConfigDrive to release > > > NicProfile[136-83-cd11d70e-92a1-4e93-be1a-be2bb6f308c6-192.168.40.24 > > > 3- > > > null] > > > > > > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 > > > TB], > > > used=[1 TB], threshold=[89.99999761581421%]. > > > > > > 2021-11-17 14:47:27,673 DEBUG [c.c.s.StatsCollector] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Verifying image storage [1]. Capacity: total=[9 > > > TB], > > > used=[1 TB], threshold=[89.99999761581421%]. > > > > > > 2021-11-17 14:47:27,675 DEBUG [c.c.n.e.ConfigDriveNetworkElement] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Deleting config drive ISO for vm: i-6-83-VM > > > > > > 2021-11-17 14:47:27,676 DEBUG [c.c.a.t.Request] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Seq 11-7692992588478939147: Sending { Cmd , MgmtId: > > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 100011, > > > > > > [{"com.cloud.agent.api.HandleConfigDriveIsoCommand":{"isoFile":"conf > > > ig > > > drive/ > > > > > > > > > i-6-83-VM.iso","create":"false","destStore":{"com.cloud.agent.api.to.NfsTO": > > > > > > {"_url":"nfs://nfs.piszki.lab/nfs","_role":"Image"}},"useHostCacheOn > > > Un > > > suppor > > > > > > tedPool":"false","preferHostCache":"false","wait":"0","bypassHostMai > > > nt > > > enance > > > ":"false"}}] } > > > > > > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request] > > > (AgentManager-Handler-1:null) (logid:) Seq 11-7692992588478939147: > > > Processing: { Ans: , MgmtId: 260497154630684, via: 11, Ver: v1, > > > Flags: 10, > > > > > > [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHo > > > st > > > Mainte > > > nance":"false"}}] } > > > > > > 2021-11-17 14:47:27,687 DEBUG [c.c.a.t.Request] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Seq 11-7692992588478939147: Received: { Ans: , > MgmtId: > > > 260497154630684, via: 11(s-163-VM), Ver: v1, Flags: 10, { Answer } } > > > > > > 2021-11-17 14:47:27,687 WARN [c.c.v.VirtualMachineManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Unable to release some network resources for the VM > > > VM instance {id: "83", name: "i-6-83-VM", uuid: > > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting > > > state > > > > > > java.lang.ClassCastException: class com.cloud.agent.api.Answer > > > cannot be cast to class > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer > > > (com.cloud.agent.api.Answer and > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module > > > of loader 'app') > > > > > > at > > > > > > com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriv > > > eI > > > so(Con > > > figDriveNetworkElement.java:661) > > > > > > at > > > > > > com.cloud.network.element.ConfigDriveNetworkElement.release(ConfigDr > > > iv > > > eNetwo > > > rkElement.java:167) > > > > > > at > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.relea > > > se > > > Nic(Ne > > > tworkOrchestrator.java:2238) > > > > > > at > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.relea > > > se > > > (Netwo > > > rkOrchestrator.java:2164) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.releaseVmResources(VirtualMac > > > hi > > > neMana > > > gerImpl.java:1836) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManager > > > Im > > > pl.jav > > > a:1826) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi > > > ne > > > Manage > > > rImpl.java:1373) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi > > > ne > > > Manage > > > rImpl.java:5802) > > > > > > at > > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Nati > > > ve > > > Method) > > > > > > at > > > > > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Nativ > > > eM > > > ethodA > > > ccessorImpl.java:62) > > > > > > at > > > > > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(D > > > el > > > egatin > > > gMethodAccessorImpl.java:43) > > > > > > at > > > java.base/java.lang.reflect.Method.invoke(Method.java:566) > > > > > > at > > > > > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerP > > > ro > > > xy.jav > > > a:107) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachin > > > eM > > > anager > > > Impl.java:5967) > > > > > > at > > > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102 > > > ) > > > > > > at > > > > > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runI > > > nC > > > ontext > > > (AsyncJobManagerImpl.java:620) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(M > > > an > > > agedCo > > > ntextRunnable.java:48) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.c > > > al > > > l(Defa > > > ultManagedContext.java:55) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.cal > > > lW > > > ithCon > > > text(DefaultManagedContext.java:102) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.run > > > Wi > > > thCont > > > ext(DefaultManagedContext.java:52) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Man > > > ag > > > edCont > > > extRunnable.java:45) > > > > > > at > > > > > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run( > > > As > > > yncJob > > > ManagerImpl.java:568) > > > > > > at > > > > > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Execut > > > or > > > s.java > > > :515) > > > > > > at > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > > > > at > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPo > > > ol > > > Execut > > > or.java:1128) > > > > > > at > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadP > > > oo > > > lExecu > > > tor.java:628) > > > > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Successfully released storage resources for the VM > > > VM instance {id: "83", name: "i-6-83-VM", uuid: > > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} in Starting > > > state > > > > > > 2021-11-17 14:47:27,692 DEBUG [c.c.v.VirtualMachineManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Successfully cleaned up resources for the VM VM > > > instance > > > {id: "83", name: "i-6-83-VM", uuid: > > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", > > > type="User"} in Starting state > > > > > > 2021-11-17 14:47:27,715 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) VM instance {id: "83", name: "i-6-83-VM", uuid: > > > "6b236c6c-4cae-465b-b1e7-dfeef70d34cb", type="User"} state transited > > > from [Starting] to [Stopped] with event [OperationFailed]. VM's > > > original > > host: > > > Host {"id": "1", "name": "psc1.piszki.lab", "uuid": > > > "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"}, new host: > > > null, host before state transition: Host {"id": "1", "name": > > > "psc1.piszki.lab", > > > "uuid": "173b2490-2c07-426c-abef-022cd45c3066", "type"="Routing"} > > > > > > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Hosts's actual total CPU: 31200 and CPU after > > > applying > > > overprovisioning: 31200 > > > > > > 2021-11-17 14:47:27,719 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Hosts's actual total RAM: (61,66 GB) 66211950592 > > > and RAM after applying overprovisioning: (61,66 GB) 66211950592 > > > > > > 2021-11-17 14:47:27,723 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) release cpu from host: 1, old used: 3900,reserved: > > > 0, actual total: 31200, total with overprovisioning: 31200; new used: > > > 0,reserved:0; movedfromreserved: false,moveToReserveredfalse > > > > > > 2021-11-17 14:47:27,727 DEBUG [c.c.c.CapacityManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) release mem from host: 1, old used: (4,00 GB) > > > 4294967296,reserved: (0 bytes) 0, total: (61,66 GB) 66211950592; new > > used: > > > (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved: > > > false,moveToReserveredfalse > > > > > > 2021-11-17 14:47:27,760 ERROR [c.c.v.VmWorkJobHandlerProxy] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Invocation exception, caused by: > > > java.lang.ClassCastException: class com.cloud.agent.api.Answer > > > cannot be cast to class > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer > > > (com.cloud.agent.api.Answer and > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module > > > of loader 'app') > > > > > > 2021-11-17 14:47:27,760 INFO [c.c.v.VmWorkJobHandlerProxy] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915 ctx-e883b663) > > > (logid:b59e4ee2) Rethrow exception java.lang.ClassCastException: > > > class com.cloud.agent.api.Answer cannot be cast to class > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer > > > (com.cloud.agent.api.Answer and > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module > > > of loader 'app') > > > > > > 2021-11-17 14:47:27,761 DEBUG [c.c.v.VmWorkJobDispatcher] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) > > > (logid:b59e4ee2) Done with run of VM work job: > > > com.cloud.vm.VmWorkStart for VM 83, job > > origin: > > > 4912 > > > > > > 2021-11-17 14:47:27,761 ERROR [c.c.v.VmWorkJobDispatcher] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) > > > (logid:b59e4ee2) Unable to complete AsyncJobVO {id:4915, userId: 4, > > > accountId: 4, > > > instanceType: > > > null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: > > > > > > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRM > > > AA > > > Zhdm9p > > > > > > ZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlz > > > dD > > > tMAAlj > > > > > > bHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFs > > > Tm > > > FtZXQA > > > > > > EkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFu > > > bm > > > VycQB- > > > > > > AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91 > > > dG > > > lsL01h > > > > > > cDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-Ztlbw > > > JW > > > drAgAE > > > > > > SgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAA > > > AA > > > AAAAAE > > > > > > AAAAAAAAAAQAAAAAAAAAU3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAA > > > AX > > > BzcgAO > > > > > > amF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1i > > > ZX > > > KGrJUd > > > > > > C5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1h > > > cA > > > UH2sHD > > > > > > FmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAAB > > > dA AKVm1Q YXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA, > > > cmdVersion: 0, > > > status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, > > > initMsid: 260497154630684, completeMsid: null, lastUpdated: null, > > > lastPolled: null, created: Wed Nov 17 14:47:26 CET 2021, removed: > > > null}, job > > > origin:4912 > > > > > > java.lang.ClassCastException: class com.cloud.agent.api.Answer > > > cannot be cast to class > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer > > > (com.cloud.agent.api.Answer and > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module > > > of loader 'app') > > > > > > at > > > > > > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriv > > > eI > > > so(Con > > > figDriveNetworkElement.java:597) > > > > > > at > > > > > > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUs > > > er > > > data(C > > > onfigDriveNetworkElement.java:225) > > > > > > at > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa > > > re > > > Elemen > > > t(NetworkOrchestrator.java:1610) > > > > > > at > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa > > > re > > > Nic(Ne > > > tworkOrchestrator.java:1936) > > > > > > at > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa > > > re > > > (Netwo > > > rkOrchestrator.java:1870) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi > > > ne > > > Manage > > > rImpl.java:1201) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi > > > ne > > > Manage > > > rImpl.java:5802) > > > > > > at > > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Nati > > > ve > > > Method) > > > > > > at > > > > > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Nativ > > > eM > > > ethodA > > > ccessorImpl.java:62) > > > > > > at > > > > > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(D > > > el > > > egatin > > > gMethodAccessorImpl.java:43) > > > > > > at > > > java.base/java.lang.reflect.Method.invoke(Method.java:566) > > > > > > at > > > > > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerP > > > ro > > > xy.jav > > > a:107) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachin > > > eM > > > anager > > > Impl.java:5967) > > > > > > at > > > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102 > > > ) > > > > > > at > > > > > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runI > > > nC > > > ontext > > > (AsyncJobManagerImpl.java:620) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(M > > > an > > > agedCo > > > ntextRunnable.java:48) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.c > > > al > > > l(Defa > > > ultManagedContext.java:55) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.cal > > > lW > > > ithCon > > > text(DefaultManagedContext.java:102) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.run > > > Wi > > > thCont > > > ext(DefaultManagedContext.java:52) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Man > > > ag > > > edCont > > > extRunnable.java:45) > > > > > > at > > > > > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run( > > > As > > > yncJob > > > ManagerImpl.java:568) > > > > > > at > > > > > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Execut > > > or > > > s.java > > > :515) > > > > > > at > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > > > > at > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPo > > > ol > > > Execut > > > or.java:1128) > > > > > > at > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadP > > > oo > > > lExecu > > > tor.java:628) > > > > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > 2021-11-17 14:47:27,770 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) > > > (logid:b59e4ee2) Complete async job-4915, jobStatus: FAILED, > resultCode: 0, result: > > > > > > rO0ABXNyABxqYXZhLmxhbmcuQ2xhc3NDYXN0RXhjZXB0aW9ugAAFzs5n5VwCAAB4cgAa > > > am > > > F2YS5s > > > > > > YW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0 > > > aW > > > 9u0P0f > > > > > > Pho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0 > > > AB > > > VMamF2 > > > > > > YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJp > > > bm > > > c7WwAK > > > > > > c3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHBy > > > ZX > > > NzZWRF > > > > > > eGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AN1jbGFzcyBjb20uY2xv > > > dW > > > QuYWdl > > > > > > bnQuYXBpLkFuc3dlciBjYW5ub3QgYmUgY2FzdCB0byBjbGFzcyBjb20uY2xvdWQuYWdl > > > bn > > > QuYXBp > > > > > > LkhhbmRsZUNvbmZpZ0RyaXZlSXNvQW5zd2VyIChjb20uY2xvdWQuYWdlbnQuYXBpLkFu > > > c3 > > > dlciBh > > > > > > bmQgY29tLmNsb3VkLmFnZW50LmFwaS5IYW5kbGVDb25maWdEcml2ZUlzb0Fuc3dlciBh > > > cm > > > UgaW4g > > > > > > dW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKXVyAB5bTGphdmEubGFuZy5TdGFj > > > a1 > > > RyYWNl > > > > > > RWxlbWVudDsCRio8PP0iOQIAAHhwAAAAGnNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVs > > > ZW > > > 1lbnRh > > > > > > CcWaJjbdhQIACEIABmZvcm1hdEkACmxpbmVOdW1iZXJMAA9jbGFzc0xvYWRlck5hbWVx > > > AH > > > 4ABUwA > > > > > > DmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEA > > > fg > > > AFTAAK > > > > > > bW9kdWxlTmFtZXEAfgAFTAANbW9kdWxlVmVyc2lvbnEAfgAFeHABAAACVXQAA2FwcHQA > > > M2 > > > NvbS5j > > > > > > bG91ZC5uZXR3b3JrLmVsZW1lbnQuQ29uZmlnRHJpdmVOZXR3b3JrRWxlbWVudHQAHkNv > > > bm > > > ZpZ0Ry > > > > > > aXZlTmV0d29ya0VsZW1lbnQuamF2YXQAFGNyZWF0ZUNvbmZpZ0RyaXZlSXNvcHBzcQB- > > > AA > > > wBAAAA > > > > > > 4XEAfgAOcQB-AA9xAH4AEHQAFmFkZFBhc3N3b3JkQW5kVXNlcmRhdGFwcHNxAH4ADAEA > > > AA > > > ZKcQB- > > > > > > AA50AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uTmV0 > > > d2 > > > 9ya09y > > > > > > Y2hlc3RyYXRvcnQAGE5ldHdvcmtPcmNoZXN0cmF0b3IuamF2YXQADnByZXBhcmVFbGVt > > > ZW > > > 50cHBz > > > > > > cQB-AAwBAAAHkHEAfgAOcQB-ABVxAH4AFnQACnByZXBhcmVOaWNwcHNxAH4ADAEAAAdO > > > cQ > > > B-AA5x > > > > > > AH4AFXEAfgAWdAAHcHJlcGFyZXBwc3EAfgAMAQAABLFxAH4ADnQAJmNvbS5jbG91ZC52 > > > bS > > > 5WaXJ0 > > > > > > dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5q > > > YX > > > ZhdAAQ > > > > > > b3JjaGVzdHJhdGVTdGFydHBwc3EAfgAMAQAAFqpxAH4ADnEAfgAdcQB-AB5xAH4AH3Bw > > > c3 > > > EAfgAM > > > > > > Av____5wdAAtamRrLmludGVybmFsLnJlZmxlY3QuTmF0aXZlTWV0aG9kQWNjZXNzb3JJ > > > bX > > > BsdAAd > > > > > > TmF0aXZlTWV0aG9kQWNjZXNzb3JJbXBsLmphdmF0AAdpbnZva2UwdAAJamF2YS5iYXNl > > > dA > > > AHMTEu > > > > > > MC4xM3NxAH4ADAIAAAA-cHEAfgAicQB-ACN0AAZpbnZva2VxAH4AJXEAfgAmc3EAfgAM > > > Ag > > > AAACtw > > > > > > dAAxamRrLmludGVybmFsLnJlZmxlY3QuRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1w > > > bH > > > QAIURl > > > > > > bGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGwuamF2YXEAfgAocQB-ACVxAH4AJnNxAH4A > > > DA > > > IAAAI2 > > > > > > cHQAGGphdmEubGFuZy5yZWZsZWN0Lk1ldGhvZHQAC01ldGhvZC5qYXZhcQB-AChxAH4A > > > JX > > > EAfgAm > > > > > > c3EAfgAMAQAAAGtxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJv > > > eH > > > l0ABpW > > > > > > bVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAM > > > AQ > > > AAF09x > > > > > > AH4ADnEAfgAdcQB-AB5xAH4AMnBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52 > > > bS > > > 5WbVdv > > > > > > cmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9i > > > cH > > > BzcQB- > > > > > > AAwBAAACbHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2Jz > > > Lm > > > ltcGwu > > > > > > QXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAM > > > cn > > > VuSW5D > > > > > > b250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1h > > > bm > > > FnZWQu > > > > > > Y29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1 > > > bm > > > 5hYmxl > > > > > > LmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3Rh > > > Y2 > > > subWFu > > > > > > YWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0 > > > TW > > > FuYWdl > > > > > > ZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hl > > > Lm > > > Nsb3Vk > > > > > > c3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB- > > > AE > > > J0AA9j > > > > > > YWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4ARXEAfgBCdAAOcnVuV2l0 > > > aE > > > NvbnRl > > > > > > eHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdl > > > ZC > > > 5jb250 > > > > > > ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4APnEAfgA_cHBzcQB-AAwBAAACOHEA > > > fg > > > AOcQB- > > > > > > ADlxAH4AOnEAfgA_cHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5F > > > eG > > > VjdXRv > > > > > > cnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AQ3EAfgAlcQB-ACZz > > > cQ > > > B-AAwC > > > > > > AAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFz > > > ay > > > 5qYXZh > > > > > > cQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQu > > > VG > > > hyZWFk > > > > > > UG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJx > > > AH > > > 4AJXEA > > > > > > fgAmc3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4 > > > ZW > > > N1dG9y > > > > > > JFdvcmtlcnEAfgBUcQB-AD9xAH4AJXEAfgAmc3EAfgAMAgAAAz1wdAAQamF2YS5sYW5n > > > Ll > > > RocmVh > > > > > > ZHQAC1RocmVhZC5qYXZhcQB-AD9xAH4AJXEAfgAmc3IAH2phdmEudXRpbC5Db2xsZWN0 > > > aW > > > 9ucyRF > > > bXB0eUxpc3R6uBe0PKee3gIAAHhweA > > > > > > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) > > > (logid:b59e4ee2) Publish async job-4915 complete on message bus > > > > > > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) > > > (logid:b59e4ee2) Wake up jobs related to job-4915 > > > > > > 2021-11-17 14:47:27,771 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) > > > (logid:b59e4ee2) Update db status for job-4915 > > > > > > 2021-11-17 14:47:27,772 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) > > > (logid:b59e4ee2) Wake up jobs joined with job-4915 and disjoin all > > > subjobs created from > > > job- 4915 > > > > > > 2021-11-17 14:47:27,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) > > > (logid:b59e4ee2) Done executing com.cloud.vm.VmWorkStart for > > > job-4915 > > > > > > 2021-11-17 14:47:27,787 INFO [o.a.c.f.j.i.AsyncJobMonitor] > > > (Work-Job-Executor-1:ctx-16dd5098 job-4912/job-4915) > > > (logid:b59e4ee2) Remove > > > job-4915 from job monitoring > > > > > > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner] > > > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard output for > > > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.182 -p > > > 623 -U piotr -P > > > Seneka22 chassis power status]: [Chassis Power is on > > > > > > ]. > > > > > > 2021-11-17 14:47:27,805 DEBUG [o.a.c.u.p.ProcessRunner] > > > (pool-2-thread-10:null) (logid:da9a8ffa) Process standard error > > > output command [/usr/bin/ipmitool -I lanplus -R 1 -v -H > > > 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status]: > > > [Unable to Get Channel Cipher Suites > > > > > > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20 > > > > > > Error response 0xc1 from Get PICMG Properities > > > > > > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20 > > > > > > Invalid completion code received: Invalid command > > > > > > Discovered IPMB address 0x0 > > > > > > ]. > > > > > > 2021-11-17 14:47:27,805 DEBUG > > > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver] > > > (pool-2-thread-10:null) > > > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v > > > -H > > > 192.168.0.182 -p 623 -U piotr -P Seneka22 chassis power status] was > > > successful and got the result [Chassis Power is on]. > > > > > > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner] > > > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard output for > > > command [/usr/bin/ipmitool -I lanplus -R 1 -v -H 192.168.0.181 -p > > > 623 -U piotr -P > > > Seneka22 chassis power status]: [Chassis Power is on > > > > > > ]. > > > > > > 2021-11-17 14:47:27,806 DEBUG [o.a.c.u.p.ProcessRunner] > > > (pool-2-thread-11:null) (logid:da9a8ffa) Process standard error > > > output command [/usr/bin/ipmitool -I lanplus -R 1 -v -H > > > 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status]: > > > [Unable to Get Channel Cipher Suites > > > > > > Running Get PICMG Properties my_addr 0x20, transit 0, target 0x20 > > > > > > Error response 0xc1 from Get PICMG Properities > > > > > > Running Get VSO Capabilities my_addr 0x20, transit 0, target 0x20 > > > > > > Invalid completion code received: Invalid command > > > > > > Discovered IPMB address 0x0 > > > > > > ]. > > > > > > 2021-11-17 14:47:27,806 DEBUG > > > [o.a.c.o.d.i.IpmitoolOutOfBandManagementDriver] > > > (pool-2-thread-11:null) > > > (logid:da9a8ffa) The command [/usr/bin/ipmitool -I lanplus -R 1 -v > > > -H > > > 192.168.0.181 -p 623 -U piotr -P Seneka22 chassis power status] was > > > successful and got the result [Chassis Power is on]. > > > > > > 2021-11-17 14:47:27,811 ERROR [c.c.a.ApiAsyncJobDispatcher] > > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) > > > Unexpected exception while executing > > > org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin > > > > > > java.lang.ClassCastException: class com.cloud.agent.api.Answer > > > cannot be cast to class > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer > > > (com.cloud.agent.api.Answer and > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module > > > of loader 'app') > > > > > > at > > > > > > com.cloud.network.element.ConfigDriveNetworkElement.createConfigDriv > > > eI > > > so(Con > > > figDriveNetworkElement.java:597) > > > > > > at > > > > > > com.cloud.network.element.ConfigDriveNetworkElement.addPasswordAndUs > > > er > > > data(C > > > onfigDriveNetworkElement.java:225) > > > > > > at > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa > > > re > > > Elemen > > > t(NetworkOrchestrator.java:1610) > > > > > > at > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa > > > re > > > Nic(Ne > > > tworkOrchestrator.java:1936) > > > > > > at > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepa > > > re > > > (Netwo > > > rkOrchestrator.java:1870) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi > > > ne > > > Manage > > > rImpl.java:1201) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachi > > > ne > > > Manage > > > rImpl.java:5802) > > > > > > at > > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Nati > > > ve > > > Method) > > > > > > at > > > > > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Nativ > > > eM > > > ethodA > > > ccessorImpl.java:62) > > > > > > at > > > > > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(D > > > el > > > egatin > > > gMethodAccessorImpl.java:43) > > > > > > at > > > java.base/java.lang.reflect.Method.invoke(Method.java:566) > > > > > > at > > > > > > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerP > > > ro > > > xy.jav > > > a:107) > > > > > > at > > > > > > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachin > > > eM > > > anager > > > Impl.java:5967) > > > > > > at > > > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102 > > > ) > > > > > > at > > > > > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runI > > > nC > > > ontext > > > (AsyncJobManagerImpl.java:620) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(M > > > an > > > agedCo > > > ntextRunnable.java:48) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.c > > > al > > > l(Defa > > > ultManagedContext.java:55) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.cal > > > lW > > > ithCon > > > text(DefaultManagedContext.java:102) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.run > > > Wi > > > thCont > > > ext(DefaultManagedContext.java:52) > > > > > > at > > > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Man > > > ag > > > edCont > > > extRunnable.java:45) > > > > > > at > > > > > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run( > > > As > > > yncJob > > > ManagerImpl.java:568) > > > > > > at > > > > > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Execut > > > or > > > s.java > > > :515) > > > > > > at > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > > > > at > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPo > > > ol > > > Execut > > > or.java:1128) > > > > > > at > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadP > > > oo > > > lExecu > > > tor.java:628) > > > > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > 2021-11-17 14:47:27,819 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Complete > > > async job-4912, jobStatus: FAILED, resultCode: 530, result: > > > > > > org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList": > > > [],"er rorcode":"530","errortext":"class com.cloud.agent.api.Answer > > > cannot be cast to class > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer > > > (com.cloud.agent.api.Answer and > > > com.cloud.agent.api.HandleConfigDriveIsoAnswer are in unnamed module > > > of loader 'app')"} > > > > > > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Publish > > > async > > > job-4912 complete on message bus > > > > > > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up > > > jobs related to job-4912 > > > > > > 2021-11-17 14:47:27,820 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Update > > > db status for job-4912 > > > > > > 2021-11-17 14:47:27,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (API-Job-Executor-1:ctx-a3c415d6 job-4912) (logid:b59e4ee2) Wake up > > > jobs joined with job-4912 and disjoin all subjobs created > > > > > > > > > > -- > > Daan > > > > > > > > > -- Daan