Did you check the 'Routing' checkbox when you added the template?
-- Erik On Wed, May 6, 2015 at 1:25 PM, Andrei Mikhailovsky <[email protected]> wrote: > > Hello guys, > > I've recently upgraded from 4.4.2 (shapeblue repo) to 4.4.3 (shapeblue > repo). I am running Ubuntu 14.04 LTS on both management and hosts. > > > The upgrade went okay and i've not noticed any issues with the upgrade > process. I've also upgraded the systemvm template as per the ShapeBlue > instructions. The system vm was downloaded from: > > > http://packages.shapeblue.com/systemvmtemplate/4.4/systemvm64template-4.4-kvm.qcow2.bz2 > > As during my previous upgrades, i've first downloaded the system vm > template and made sure that the status is shown as READY. I've first > created a new Template called systemvm-kvm-4.4.3 (as i've already had the > systemvm-kvm-4.4). Following the successfull download and install of the > template, i've changed the router.template.kvm global setting to > systemvm-kvm-4.4.3. Following this I've upgraded the management server > followed by the agents. > > > After the management server restart I've manually deleted the SSVM and > CPVM and they were successfully recreated and shown as UP. > > However, I've tried to manually restart one of the existing networks and > got an error message. Also, I can't start a newly created VM with a new > network. I can successfully create a new VM with an existing network, which > has a running VR. > > Further investigating the issue, i've discovered that during the VR start > up, the management server shows the following error: > > > > 2015-05-06 11:32:23,994 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) KVM > won't support system vm, skip it > > > > I have downloaded the right template for the KVM hypervisor, which can be > indicated by successfully starting both CPVM and SSVM. The full log from > the management server: > > > > --------------------------------------------- > > 2015-05-06 11:32:23,545 DEBUG [c.c.a.t.Request] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Seq > 8-6236922533954715673: Sending { Cmd , MgmtId: 115129173025118, via: > 8(arh-cloud2-ib), Ver: v1, Flags: 100111, > [{"com.cloud.agent.api.routing.SavePasswordCommand":{,"vmIpAddress":"10.1.1.67","vmName":"Win7-Pentest","executeInSequence":true,"accessDetails":{"zone.network.type":"Advanced"," > router.name > ":"r-1200-VM","router.ip":"169.254.3.212","router.guest.ip":"10.1.1.1"},"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"10.1.1.67","vmName":"Win7-Pentest","executeInSequence":true,"accessDetails":{"zone.network.type":"Advanced"," > router.name":"r-1200-VM","router.ip":"169.254.3.212","router.guest.ip":"10.1.1.1"},"wait":0}}] > } > 2015-05-06 11:32:23,943 DEBUG [c.c.a.t.Request] > (AgentManager-Handler-15:null) Seq 8-6236922533954715673: Processing: { > Ans: , MgmtId: 115129173025118, via: 8, Ver: v1, Flags: 110, > [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}},{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] > } > 2015-05-06 11:32:23,944 DEBUG [c.c.a.m.AgentAttache] > (AgentManager-Handler-15:null) Seq 8-6236922533954715673: No more commands > found > 2015-05-06 11:32:23,944 DEBUG [c.c.a.t.Request] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Seq > 8-6236922533954715673: Received: { Ans: , MgmtId: 115129173025118, via: 8, > Ver: v1, Flags: 110, { Answer, Answer } } > 2015-05-06 11:32:23,945 DEBUG [c.c.n.NetworkModelImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Service > SecurityGroup is not supported in the network id=280 > 2015-05-06 11:32:23,949 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Lock is > acquired for network id 311 as a part of network implement > 2015-05-06 11:32:23,949 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Asking > ExternalGuestNetworkGuru to implement > Ntwk[c9d0ba49-5eeb-481a-89fa-436e80fb61c0|Guest|8] > 2015-05-06 11:32:23,969 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Asking > VirtualRouter to implemenet > Ntwk[c9d0ba49-5eeb-481a-89fa-436e80fb61c0|Guest|8] > 2015-05-06 11:32:23,972 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Lock is > acquired for network id 311 as a part of router startup in > Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] > : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(48)-Storage()] > 2015-05-06 11:32:23,981 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Adding > nic for Virtual Router in Guest network > Ntwk[c9d0ba49-5eeb-481a-89fa-436e80fb61c0|Guest|8] > 2015-05-06 11:32:23,981 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Adding > nic for Virtual Router in Control network > 2015-05-06 11:32:23,983 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Found > existing network configuration for offering [Network Offering > [3-Control-System-Control-Network]: > Ntwk[0eaa3a90-c909-424f-8ea2-1665742d89b5|Control|3] > 2015-05-06 11:32:23,983 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Releasing lock for Acct[06ee8d45-65f2-11e3-9bd1-d8d38559b2d0-system] > 2015-05-06 11:32:23,984 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Adding > nic for Virtual Router in Public network > 2015-05-06 11:32:23,987 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Found > existing network configuration for offering [Network Offering > [1-Public-System-Public-Network]: > Ntwk[06594454-14b0-4fe5-b0cd-641240896ec6|Public|1] > 2015-05-06 11:32:23,987 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Releasing lock for Acct[06ee8d45-65f2-11e3-9bd1-d8d38559b2d0-system] > 2015-05-06 11:32:23,992 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Allocating the VR i=1222 in datacenter > com.cloud.dc.DataCenterVO$$EnhancerByCGLIB$$c79d907a@1with the hypervisor > type KVM > 2015-05-06 11:32:23,994 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) KVM > won't support system vm, skip it > 2015-05-06 11:32:23,995 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Lock is > released for network id 311 as a part of router startup in > Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] > : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(48)-Storage()] > 2015-05-06 11:32:23,995 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Cleaning up because we're unable to implement the network > Ntwk[c9d0ba49-5eeb-481a-89fa-436e80fb61c0|Guest|8] > 2015-05-06 11:32:24,000 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Lock is > acquired for network Ntwk[c9d0ba49-5eeb-481a-89fa-436e80fb61c0|Guest|8] as > a part of network shutdown > 2015-05-06 11:32:24,003 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Sending > network shutdown to VirtualRouter > 2015-05-06 11:32:24,004 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Network > id=311 is shutdown successfully, cleaning up corresponding resources now. > 2015-05-06 11:32:24,006 DEBUG [c.c.n.g.GuestNetworkGuru] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Releasing vnet for the network id=311 > 2015-05-06 11:32:24,013 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Lock is > released for network Ntwk[c9d0ba49-5eeb-481a-89fa-436e80fb61c0|Guest|8] as > a part of network shutdown > 2015-05-06 11:32:24,014 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Lock is > released for network id 311 as a part of network implement > 2015-05-06 11:32:24,014 INFO [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Unable > to contact resource. > com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] > is unreachable: Can't find all necessary running routers! > at > com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:199) > at > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1080) > at > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetwork(NetworkOrchestrator.java:992) > at > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(NetworkOrchestrator.java:1272) > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:986) > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5201) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) > at > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5346) > at > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:502) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:459) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-05-06 11:32:24,016 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Cleaning up resources for the vm VM[User|i-2-468-VM] in Starting state > 2015-05-06 11:32:24,018 DEBUG [c.c.a.t.Request] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Seq > 48-3897021052559032342: Sending { Cmd , MgmtId: 115129173025118, via: > 48(arh-clou > d4-ib), Ver: v1, Flags: 100011, > [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-2-468-VM","wait":0}}] > } > 2015-05-06 11:32:24,231 DEBUG [c.c.a.t.Request] > (AgentManager-Handler-13:null) Seq 48-3897021052559032342: Processing: { > Ans: , MgmtId: 115129173025118, via: 48, Ver: v1, Flags: 10, > [{"com.cloud.agent. > api.StopAnswer":{"result":true,"wait":0}}] } > 2015-05-06 11:32:24,231 DEBUG [c.c.a.t.Request] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Seq > 48-3897021052559032342: Received: { Ans: , MgmtId: 115129173025118, via: > 48, Ver: v1, Flags: 10, { StopAnswer } } > 2015-05-06 11:32:24,239 DEBUG [c.c.n.NetworkModelImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Service > SecurityGroup is not supported in the network id=280 > 2015-05-06 11:32:24,241 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Changing active number of nics for network id=280 on -1 > 2015-05-06 11:32:24,245 DEBUG [o.a.c.e.o.NetworkOrchestrator] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Asking > VirtualRouter to release > NicProfile[2804-468-cb3675f8-dc3c-47b7-9e44-037c83202ab0-10.1.1.67-null > 2015-05-06 11:32:24,247 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Successfully released network resources for the vm VM[User|i-2-468-VM] > 2015-05-06 11:32:24,247 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Successfully cleanued up resources for the vm VM[User|i-2-468-VM] in > Starting state > 2015-05-06 11:32:24,249 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Root > volume is ready, need to place VM in volume's cluster > 2015-05-06 11:32:24,249 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Vol[477|vm=468|ROOT] is READY, changing deployment plan to use this pool's > dcId: 1 , podId: 1 , and clusterId: 1 > 2015-05-06 11:32:24,255 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Deploy > avoids pods: [], clusters: [], hosts: [48] > 2015-05-06 11:32:24,255 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > DataCenter id = '1' provided is in avoid set, DeploymentPlanner cannot > allocate the VM, returning. > 2015-05-06 11:32:24,261 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Deploy > avoids pods: [], clusters: [], hosts: [48] > 2015-05-06 11:32:24,261 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > DataCenter id = '1' provided is in avoid set, DeploymentPlanner cannot > allocate the VM, returning. > 2015-05-06 11:32:24,267 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) VM > state transitted from :Starting to Stopped with event: OperationFailedvm's > original host id: 48 new host id: null host id before state transition: 48 > 2015-05-06 11:32:24,272 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Hosts's > actual total CPU: 50400 and CPU after applying overprovisioning: 1008000 > 2015-05-06 11:32:24,272 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Hosts's > actual total RAM: 33701453824 and RAM after applying overprovisioning: > 33701453824 > 2015-05-06 11:32:24,272 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) release > cpu from host: 48, old used: 80000,reserved: 0, actual total: 50400, total > with overprovisioning: 1008000; new used: 64000,reserved:0; > movedfromreserved: false,moveToReserveredfalse > 2015-05-06 11:32:24,272 DEBUG [c.c.c.CapacityManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) release > mem from host: 48, old used: 32212254720,reserved: 0, total: 33701453824; > new used: 27917287424,reserved:0; movedfromreserved: > false,moveToReserveredfalse > 2015-05-06 11:32:24,279 ERROR [c.c.v.VmWorkJobHandlerProxy] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) > Invocation exception, caused by: > com.cloud.exception.InsufficientServerCapacityException: Unable to create a > deployment for VM[User|i-2-468-VM]Scope=interface com.cloud.dc.DataCenter; > id=1 > 2015-05-06 11:32:24,279 INFO [c.c.v.VmWorkJobHandlerProxy] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435 ctx-d02e2a98) Rethrow > exception com.cloud.exception.InsufficientServerCapacityException: Unable > to create a deployment for VM[User|i-2-468-VM]Scope=interface > com.cloud.dc.DataCenter; id=1 > 2015-05-06 11:32:24,280 DEBUG [c.c.v.VmWorkJobDispatcher] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435) Done with run of VM > work job: com.cloud.vm.VmWorkStart for VM 468, job origin: 14433 > 2015-05-06 11:32:24,280 ERROR [c.c.v.VmWorkJobDispatcher] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435) Unable to complete > AsyncJobVO {id:14435, userId: 3, accountId: 2, instanceType: null, > instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAMAAAAAAAAB1HQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwc3IAEWphdmEudXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2FkRmFjdG9ySQAJdGhyZXNob2xkeHA_QAAAAAAADHcIAAAAEAAAAAF0AApWbVBhc3N3b3JkdAAcck8wQUJYUUFEbk5oZG1Wa1gzQmhjM04zYjNKa3hw, > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, > result: null, initMsid: 115129173025118, completeMsid: null, lastUpdated: > null, lastPolled: null, created: Wed May 06 11:32:22 BST 2015}, job > origin:14433 > com.cloud.exception.InsufficientServerCapacityException: Unable to create > a deployment for VM[User|i-2-468-VM]Scope=interface > com.cloud.dc.DataCenter; id=1 > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:947) > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5201) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) > at > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5346) > at > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:502) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:459) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-05-06 11:32:24,282 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435) Complete async > job-14435, jobStatus: FAILED, resultCode: 0, result: > rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBxAH4AB3QAUUpvYiBmYWlsZWQgZHVlIHRvIGV4Y2VwdGlvbiBVbmFibGUgdG8gY3JlYXRlIGEgZGVwbG95bWVudCBmb3IgVk1bVXNlcnxpLTItNDY4LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAADXNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAETAAIZmlsZU5hbWVxAH4ABEwACm1ldGhvZE5hbWVxAH4ABHhwAAAAcnQAIGNvbS5jbG91ZC52bS5WbVdvcmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9ic3EAfgALAAAB9nQAP29yZy5hcGFjaGUuY2xvdWRzdGFjay5mcmFtZXdvcmsuam9icy5pbXBsLkFzeW5jSm9iTWFuYWdlckltcGwkNXQAGEFzeW5jSm9iTWFuYWdlckltcGwuamF2YXQADHJ1bkluQ29udGV4dHNxAH4ACwAAADF0AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGUkMXQAG01hbmFnZWRDb250ZXh0UnVubmFibGUuamF2YXQAA3J1bnNxAH4ACwAAADh0AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TWFuYWdlZENvbnRleHQuamF2YXQABGNhbGxzcQB-AAsAAABndABAb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dHEAfgAadAAPY2FsbFdpdGhDb250ZXh0c3EAfgALAAAANXEAfgAdcQB-ABp0AA5ydW5XaXRoQ29udGV4dHNxAH4ACwAAAC50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4AFnEAfgAXc3EAfgALAAABy3EAfgARcQB-ABJxAH4AF3NxAH4ACwAAAdd0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeGVjdXRvcnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AG3NxAH4ACwAAAQZ0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay5qYXZhcQB-ABdzcQB-AAsAAAR5dAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJzcQB-AAsAAAJndAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9yJFdvcmtlcnEAfgAscQB-ABdzcQB-AAsAAALpdAAQamF2YS5sYW5nLlRocmVhZHQAC1RocmVhZC5qYXZhcQB-ABdzcgAmamF2YS51dGlsLkNvbGxlY3Rpb25zJFVubW9kaWZpYWJsZUxpc3T8DyUxteyOEAIAAUwABGxpc3RxAH4ABnhyACxqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlQ29sbGVjdGlvbhlCAIDLXvceAgABTAABY3QAFkxqYXZhL3V0aWwvQ29sbGVjdGlvbjt4cHNyABNqYXZhLnV0aWwuQXJyYXlMaXN0eIHSHZnHYZ0DAAFJAARzaXpleHAAAAAAdwQAAAAAeHEAfgA4eA > 2015-05-06 11:32:24,288 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435) Done executing > com.cloud.vm.VmWorkStart for job-14435 > 2015-05-06 11:32:24,300 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435) Sync queue (3383) is > currently empty > 2015-05-06 11:32:24,300 INFO [o.a.c.f.j.i.AsyncJobMonitor] > (Work-Job-Executor-7:ctx-222e5de7 job-14433/job-14435) Remove job-14435 > from job monitoring > 2015-05-06 11:32:24,301 ERROR [c.c.a.ApiAsyncJobDispatcher] > (API-Job-Executor-3:ctx-01eebe9f job-14433) Unexpected exception while > executing org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin > java.lang.RuntimeException: Job failed due to exception Unable to create a > deployment for VM[User|i-2-468-VM] > at > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:502) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:459) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-05-06 11:32:24,302 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (API-Job-Executor-3:ctx-01eebe9f job-14433) Complete async job-14433, > jobStatus: FAILED, resultCode: 530, result: > org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Job > failed due to exception Unable to create a deployment for > VM[User|i-2-468-VM]"} > 2015-05-06 11:32:24,307 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (API-Job-Executor-3:ctx-01eebe9f job-14433) Done executing > org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin for job-14433 > 2015-05-06 11:32:24,309 INFO [o.a.c.f.j.i.AsyncJobMonitor] > (API-Job-Executor-3:ctx-01eebe9f job-14433) Remove job-14433 from job > monitoring > > > > ------------------------------------------- > > > I can't find any errors on the host servers, so I have no idea why ACS can > create SSVm/CPVM but can't create new virtual routers. > > Could someone please help me fix this problem. I've got a half broken > cloud at the moment as I can't create or restart networks. > > Many thanks > > Andrei > > > >
