Sangeetha Hariharan created CLOUDSTACK-1016: -----------------------------------------------
Summary: Not able to deploy a VM , Key: CLOUDSTACK-1016 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1016 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Management Server Affects Versions: 4.1.0 Environment: latest "nextwork-refactor" build. Reporter: Sangeetha Hariharan Priority: Blocker Fix For: 4.1.0 Set up: KVM host in advanced zone: I upgraded the management server and KVM agent to the latest "nextwork-refactor" build. I already had 1 existing "isolated" network with 2 vms. I destroyed these Vms and waited for them to get "Destroyed". Then I deleted the network. Network deletion reported success. Now I tried to deploy 2 Vms in a new isolated network. Vm deployment started failing. I see the router getting started. But seems like it is not able to communicate using the lini local ip address. I have attempted to deploy Vms in the same network and tried to deploy Vms in different network. They are all failing now. "StartCommand" for the Router fails to get any anwer back: 2013-01-18 16:09:09,266 DEBUG [agent.transport.Request] (Job-Executor-16:job-69) Seq 1-386073116: Sending { Cmd , MgmtId: 206915885081428, via: 1, Ver: v1, Flags: 100111, [{"StartCommand":{"vm":{"id":33,"name":"r-33-VM","type":"DomainRouter","cpus":1,"speed":500,"minRam":134217728,"maxRam":134217728,"a rch":"x86_64","os":"Debian GNU/Linux 5.0 (32-bit)","bootArgs":" template=domP name=r-33-VM eth2ip=10.223.139.32 eth2mask=255.255.255.192 gateway=10.223. 139.1 eth0ip=10.1.1.1 eth0mask=255.255.255.0 domain=cs5cloud.internal dhcprange=10.1.1.1 eth1ip=169.254.3.125 eth1mask=255.255.0.0 type=router disable_r p_filter=true dns1=72.52.126.11 dns2=72.52.126.12","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"vncPassword":"7b62e0c457bbfc02","params":{ },"uuid":"d577f4b2-6e7c-4c63-b621-2af3b8637416","disks":[{"id":32,"name":"/export/home/sangeetha/asf/primary","mountPoint":"c80a61af-c459-437e-9621-e671 34d0b589","path":"c80a61af-c459-437e-9621-e67134d0b589","size":725811200,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"afcf9136 -8f08-3361-8328-fac0f471b3e4","deviceId":0}],"nics":[{"deviceId":2,"networkRateMbps":200,"defaultNic":true,"uuid":"770340ea-77ac-4acc-9e5c-8fd14fa7b7cc" ,"ip":"10.223.139.32","netmask":"255.255.255.192","gateway":"10.223.139.1","mac":"06:4d:92:00:00:0c","dns1":"72.52.126.11","dns2":"72.52.126.12","broadc astType":"Vlan","type":"Public","broadcastUri":"vlan://1390","isolationUri":"vlan://1390","isSecurityGroupEnabled":false},{"deviceId":0,"networkRateMbps ":200,"defaultNic":false,"uuid":"01d11c44-6783-4c36-821c-2a9691425dc5","ip":"10.1.1.1","netmask":"255.255.255.0","mac":"02:00:18:cd:00:02","dns1":"72.52 .126.11","dns2":"72.52.126.12","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2484","isolationUri":"vlan://2484","isSecurityGroupEnabled": false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"06a4d3e3-d83e-428f-989a-191836cee980","ip":"169.254.3.125","netmask":"255.255.0.0", "gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:7d","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"wait":0}},{"check.Chec kSshCommand":{"ip":"169.254.3.125","port":3922,"interval":6,"retries":100,"name":"r-33-VM","wait":0}},{"GetDomRVersionCmd":{"accessDetails":{"router.ip" :"169.254.3.125","router.name":"r-33-VM"},"wait":0}},{},{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":5,"publicIp":"10.223.139.32","sourceNat": true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"1390","vlanGateway":"10.223.139.1","vlanNetmask":"255.255.255.192","vifMacAddress":"06:26:b c:00:00:0c","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254 .3.125","router.name":"r-33-VM"},"wait":0}}] } 2013-01-18 16:20:53,281 DEBUG [agent.transport.Request] (Job-Executor-16:job-69) Seq 1-386073116: Received: { Ans: , MgmtId: 206915885081428, via: 1, Ver: v1, Flags: 110, { StartAnswer, CheckSshAnswer, Answer, Answer, Answer } } 2013-01-18 16:20:53,333 WARN [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-16:job-69) Unable to ssh to the VM: Can not ping System vm r-33-VMdue to:Unable to connect 2013-01-18 16:20:53,333 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-16:job-69) The guru did not like the answers so stopping VM[DomainRouter|r-33-VM] 2013-01-18 16:20:53,337 DEBUG [agent.transport.Request] (Job-Executor-16:job-69) Seq 1-386073152: Sending { Cmd , MgmtId: 206915885081428, via: 1, Ver: v1, Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"r-33-VM","wait":0}}] } 2013-01-18 16:20:54,400 DEBUG [cloud.api.ApiServlet] (catalina-exec-19:null) ===START=== 10.217.252.59 -- GET command=queryAsyncJobResult&jobId=2459861f-0a4f-4af3-8501-6fcb29b2ff5a&response=json&sessionkey=IcQpAjjwsSzMvMQHV3Y%2F95K6IqE%3D&_=1358544279605 2013-01-18 16:20:54,431 DEBUG [cloud.api.ApiServlet] (catalina-exec-19:null) ===END=== 10.217.252.59 -- GET command=queryAsyncJobResult&jobId=2459861f-0a4f-4af3-8501-6fcb29b2ff5a&response=json&sessionkey=IcQpAjjwsSzMvMQHV3Y%2F95K6IqE%3D&_=1358544279605 2013-01-18 16:20:57,794 DEBUG [agent.transport.Request] (AgentManager-Handler-6:null) Seq 1-386073152: Processing: { Ans: , MgmtId: 206915885081428, via: 1, Ver: v1, Flags: 110, [{"StopAnswer":{"vncPort":0,"result":true,"wait":0}}] } 2013-01-18 16:20:57,794 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-6:null) Seq 1-386073152: No more commands found 2013-01-18 16:20:57,794 DEBUG [agent.transport.Request] (Job-Executor-16:job-69) Seq 1-386073152: Received: { Ans: , MgmtId: 206915885081428, via: 1, Ver: v1, Flags: 110, { StopAnswer } } 2013-01-18 16:20:57,794 ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-16:job-69) Failed to start instance VM[DomainRouter|r-33-VM] com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRouter|r-33-VM] due to error in finalizeStart, not retrying at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:817) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:475) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2449) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1693) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1793) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1771) at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:173) at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1544) at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1466) at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34) at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:1622) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:755) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:475) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2876) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2544) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2532) at com.cloud.event.ActionEventCallback.intercept(ActionEventCallback.java:36) at com.cloud.api.commands.DeployVMCmd.execute(DeployVMCmd.java:330) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:168) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:433) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2013-01-18 16:20:57,808 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-16:job-69) Cleaning up resources for the vm VM[DomainRouter|r-33-VM] in Starting state -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira