Hi CloudStack Community,
We have a critical production issue where VMs cannot start due to expired VNC certificates. Due to local storage constraints, we CANNOT remove and re-add the host. Need urgent guidance on certificate renewal without host removal. ENVIRONMENT (Production): - CloudStack Version: [,4.18.0.0] - Hypervisor: KVM - Host OS: [ Ubuntu 22.04 LTS] - Storage: LOCAL STORAGE (VM migration NOT possible) ERROR DETAILS: From /var/log/cloudstack/agent/agent.log: org.libvirt.LibvirtException: internal error: process exited while connecting to monitor: 2026-03-31 01:31:11.350+0000: Domain id=13 is tainted: high-privileges 2026-03-31 01:31:11.350+0000: Domain id=13 is tainted: host-cpu 2026-03-31T01:31:11.413970Z qemu-system-x86_64: -drive file=/var/lib/libvirt/images/5c12f1be-3788-40c8-a019-bb82ea42fb61,format=qcow2,if=none,id=drive-virtio-disk0,serial=5c12f1be378840c8a019,cache=none: 'serial' is deprecated, please use the corresponding option of '-device' instead 2026-03-31T01:31:11.788215Z qemu-system-x86_64: -vnc 172.17.0.2:0,password,tls,x509verify=/etc/pki/libvirt-vnc: Failed to start VNC server: The server certificate /etc/pki/libvirt-vnc/server-cert.pem has expired at org.libvirt.ErrorHandler.processError(Unknown Source) at org.libvirt.ErrorHandler.processError(Unknown Source) at org.libvirt.Connect.domainCreateXML(Unknown Source) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startVM(LibvirtComputingResource.java:1821) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:104) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:49) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1853) at com.cloud.agent.Agent.processRequest(Agent.java:662) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Current Global Settings: - ca.framework.cert.automatic.renewal: [true/false](currently [true]) - ca.plugin.root.allow.expired.cert: [true/false] (currently [true]) - ca.framework.cert.validity.period: [365] - ca.framework.cert.expiry.alert.period: [15] - ca.plugin.root.auth.strictness: [true/false] (currently [true]) CONSTRAINTS (Critical): 1. CANNOT remove and re-add host - business critical VMs on local storage 2. CANNOT migrate VMs - local storage limitation 3. VMs MUST remain running if possible - production workload QUESTIONS: 1. Is there a way to manually trigger cert renewal on the host side? 2. Are there any manual certificate replacement procedures? 3. Is there a way to disable VNC TLS temporarily to start VMs? This is affecting production business operations. Any urgent guidance or workaround would be greatly appreciated! 流云逝水 [email protected]
