A bit more digging revealed that the host server does receive the xml data with the virtual router details:
2015-10-21 12:04:58,457 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) starting r-1407-VM: <domain type='kvm'> <name>r-1407-VM</name> <uuid>815d2860-cc7f-475d-bf63-02814c720fe4</uuid> <description>Debian GNU/Linux 7(64-bit)</description> <clock offset='utc'> ..... <cpu></cpu><on_reboot>restart</on_reboot> <on_poweroff>destroy</on_poweroff> <on_crash>destroy</on_crash> </domain> but there is no instance being created on the host server and the agent logs do not reveal anything else apart from what i've provided in my previous email. Thanks for the help Andrei ----- Original Message ----- From: "Andrei Mikhailovsky" <[email protected]> To: [email protected] Sent: Wednesday, 21 October, 2015 11:36:28 AM Subject: KVM - No longer able to start virtual routers Hello guys, I have recently upgraded from ACS 4.5.1 to 4.5.2. The upgrade went well as far as I can tell, no error messages. As there was no need to update the systemvm templates, i've not bothered to reboot the system vms and virtual routers. I am running Ubuntu 14.04 with KVM hosts and using nfs for secondary and ceph rbd for primary storage. Today i've tried to restart one of the networks with the clean up option and noticed that the restart has failed. After a bit of poking around i've identified that ACS is no longer able to create virtual routers. ACS just showing them with Starting status, but they are not starting on the host server. I've looked at the host server, which is suppose to be starting that virtual router and there is no sign of the domain being created. The host server has the following entries in the agent.log file: 2015-10-21 10:39:02,140 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Executing: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.pl -n r-1405-VM -p %template=domP%name=r-1405-VM%eth2ip=XXX.XXX.XXX.53%eth2mask=255.255.255.128%gateway=XXX.XXX.XXX.1%eth0ip=10.1.1.1%eth0mask=255.255.255.0%domain=kaspersky.local%cidrsize=24%dhcprange=10.1.1.1%eth1ip=169.254.3.232%eth1mask=255.255.0.0%type=router%disable_rp_filter=true%dns1=178.248.108.130%dns2=91.224.1.152 2015-10-21 10:39:02,172 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Exit value is 111 2015-10-21 10:39:02,173 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) ERROR: unable to connect to /var/lib/libvirt/qemu/r-1405-VM.agent - Connection refused 2015-10-21 10:39:02,174 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) passcmd failed:ERROR: unable to connect to /var/lib/libvirt/qemu/r-1405-VM.agent - Connection refused I've checked ssvm and it seems to be running perfectly well. The /usr/local/cloud/systemvm/ssvm-check.sh script produces no warnings or errors, the nfs mount point is mounted and writable. I've also tried to create a new vm with a new network and that vm is not starting because ACS is unable to start the virtual router for the network. Any idea how to get this issue resolved? Thanks
