[
https://issues.apache.org/jira/browse/CLOUDSTACK-120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463770#comment-13463770
]
Sowmya Krishnan commented on CLOUDSTACK-120:
--------------------------------------------
I am able to consistently see the issue in both Basic and Advanced zone. I am
trying on fresh install of hypervisor OS and MS.
Needed to manually restart cloud-agent on host to get the router VM started.
Here are the steps:
Install Agent on host
Restart host (due to bug: https://issues.apache.org/jira/browse/CLOUDSTACK-205)
Add host to MS
System VMs are up
Deploy instance-> fails due to router VM not coming up
Restart cloud-agent on host
Start router (succeeds)
Here's the agent log:
2012-09-26 13:06:09,391 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null)
Processing command: com.cloud.agent.api.GetDomRVersionCmd
2012-09-26 13:06:09,393 DEBUG [resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-4:null) Executing:
/usr/lib64/cloud/common/scripts/network/domr/router_proxy.sh
get_template_version.sh 169.254.3.249
2012-09-26 13:06:10,010 DEBUG [resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-4:null) Exit value is 255
2012-09-26 13:06:10,011 DEBUG [resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-4:null) Warning: Identity file /root/.ssh/id_rsa.cloud
not accessible: No such file or directory.
2012-09-26 13:06:10,020 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null)
Seq 1-235405401: { Ans: , MgmtId: 257562663661698, via: 1, Ver: v1, Flags:
110,
[{"StartAnswer":{"vm":{"id":4,"name":"r-4-VM","type":"DomainRouter","cpus":1,"speed":500,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"Debian
GNU/Linux 5.0 (32-bit)","bootArgs":" template=domP name=r-4-VM
eth0ip=10.102.125.195 eth0mask=255.255.255.0 gateway=10.102.125.1
domain=cs1cloud.internal dhcprange=10.102.125.1 eth1ip=169.254.3.249
eth1mask=255.255.0.0 type=dhcpsrvr disable_rp_filter=true
dns1=10.103.128.15","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"vncPassword":"18ec3a45dbff2e59","params":{},"uuid":"3aacb8f2-1c63-4d72-b71b-53abb5b89b8d","disks":[{"id":5,"name":"/cloudstack/sowmya/primary1","mountPoint":"b1647294-be68-4453-b0e5-6b710adf44d2","path":"b1647294-be68-4453-b0e5-6b710adf44d2","size":725811200,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"d1d82ec9-237a-345b-a269-21d08f66bb48","deviceId":0}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"90bc9073-baed-4ec7-972d-2713c8e6033b","ip":"10.102.125.195","netmask":"255.255.255.0","gateway":"10.102.125.1","mac":"06:f9:06:00:00:10","dns1":"10.103.128.15","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://untagged","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"97e6c35f-5f01-4450-ab5e-eae2d8d08c61","ip":"169.254.3.249","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:f9","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"result":true,"wait":0}},{"check.CheckSshAnswer":{"result":true,"wait":0}},{"GetDomRVersionAnswer":{"result":false,"details":"GetDomRVersionCmd
failed","wait":0}},{"Answer":{"result":false,"details":"Stopped by previous
failure","wait":0}}] }
2012-09-26 13:06:10,066 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Request:Seq 1-235405412: { Cmd , MgmtId: 257562663661698, via: 1, Ver: v1,
Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"r-4-VM","wait":0}}] }
2012-09-26 13:06:10,066 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Processing command: com.cloud.agent.api.StopCommand
2012-09-26 13:06:10,200 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Executing:
/usr/lib64/cloud/common/scripts/vm/network/security_group.py
destroy_network_rules_for_vm --vmname r-4-VM --vif vnet7
2012-09-26 13:06:10,392 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Execution is successful.
2012-09-26 13:06:10,393 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Try to stop the vm at first
2012-09-26 13:06:25,490 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-2:null) requested delete disk
/mnt/d1d82ec9-237a-345b-a269-21d08f66bb48/r-4-VM-patchdisk
2012-09-26 13:06:25,555 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Failed to get dom xml:
org.libvirt.LibvirtException: Domain not found: no domain with matching uuid
'f257919b-0e2e-3d85-823e-c36d79e10727'
2012-09-26 13:06:25,556 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Seq 1-235405412: { Ans: , MgmtId: 257562663661698, via: 1, Ver: v1, Flags:
110, [{"StopAnswer":{"vncPort":0,"result":true,"wait":0}}] }
2012-09-26 13:06:25,612 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null)
Request:Seq 1-235405413: { Cmd , MgmtId: 257562663661698, via: 1, Ver: v1,
Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"r-4-VM","wait":0}}] }
2012-09-26 13:06:25,613 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null)
Processing command: com.cloud.agent.api.StopCommand
2012-09-26 13:06:25,616 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Failed to get dom xml:
org.libvirt.LibvirtException: Domain not found: no domain with matching uuid
'f257919b-0e2e-3d85-823e-c36d79e10727'
2012-09-26 13:06:25,617 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Failed to get dom xml:
org.libvirt.LibvirtException: Domain not found: no domain with matching uuid
'f257919b-0e2e-3d85-823e-c36d79e10727'
2012-09-26 13:06:25,617 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Executing:
/usr/lib64/cloud/common/scripts/vm/network/security_group.py
destroy_network_rules_for_vm --vmname r-4-VM
2012-09-26 13:06:25,783 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Execution is successful.
2012-09-26 13:06:25,784 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Try to stop the vm at first
2012-09-26 13:06:25,786 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Failed to stop VM :r-4-VM :
org.libvirt.LibvirtException: Domain not found: no domain with matching uuid
'f257919b-0e2e-3d85-823e-c36d79e10727'
at org.libvirt.ErrorHandler.processError(Unknown Source)
at org.libvirt.Connect.processError(Unknown Source)
at org.libvirt.Connect.domainLookupByUUIDString(Unknown Source)
at org.libvirt.Connect.domainLookupByUUID(Unknown Source)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3721)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3653)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2586)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:965)
at com.cloud.agent.Agent.processRequest(Agent.java:518)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:831)
at com.cloud.utils.nio.Task.run(Task.java:83)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2012-09-26 13:06:25,790 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Failed to get vm status:Domain not found: no
domain with matching uuid 'f257919b-0e2e-3d85-823e-c36d79e10727'
2012-09-26 13:06:25,791 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Failed to get vm status:Domain not found: no
domain with matching uuid 'f257919b-0e2e-3d85-823e-c36d79e10727'
2012-09-26 13:06:25,793 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Failed to get vm status:Domain not found: no
domain with matching uuid 'f257919b-0e2e-3d85-823e-c36d79e10727'
> Failed to deploy Router with KVM hyperviosor [ Can't find the mapping of
> guest os: Debian GNU/Linux 5.0 (32-bit)]
> ------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-120
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-120
> Project: CloudStack
> Issue Type: Bug
> Components: KVM, Management Server
> Affects Versions: pre-4.0.0
> Reporter: Sailaja Mada
> Assignee: edison su
> Priority: Blocker
> Fix For: pre-4.0.0
>
> Attachments: agent.log, agentlog_reopen120, api-server.log,
> management-server.log, mslog_reopen120.log
>
>
> Steps:
> 1. Setup Management server
> 2. Setup KVM host with Cloudstack Agent installed
> 3. Setup Basic Zone with SG service offering
> 4. After having all the system VM's running , tried to deploy the first
> instance
> Observation :
> First Instance deployment failed as the router failed start .
> MS Error log :
> 2012-09-17 15:10:27,255 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:null) Found 0 routers.
> 2012-09-17 15:10:31,882 DEBUG [agent.transport.Request]
> (AgentManager-Handler-14:null) Seq 3-1151271005: Processing: { Ans: ,
> MgmtId: 55487956346259, via: 3, Ver: v1, Flags: 110,
> [{"StartAnswer":{"vm":{"id":4,"name":"r-4-VM","type":"DomainRouter","cpus":1,"speed":500,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"Debian
> GNU/Linux 5.0 (32-bit)","bootArgs":" template=domP name=r-4-VM
> eth0ip=10.102.125.188 eth0mask=255.255.255.0 gateway=10.102.125.1
> domain=cs1cloud.internal dhcprange=10.102.125.1 eth1ip=169.254.1.202
> eth1mask=255.255.0.0 type=dhcpsrvr disable_rp_filter=true
> dns1=10.103.128.15","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"vncPassword":"fa3d755b18ba1297","params":{},"uuid":"6e5680fa-14b1-447d-9024-1bed9cb59268","disks":[{"id":4,"name":"/cloudstack/sailaja/pri1/","mountPoint":"0614e5e3-77ce-46da-b356-4e528752aa9d","path":"0614e5e3-77ce-46da-b356-4e528752aa9d","size":725811200,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"e95f2598-5038-3ceb-a326-5309b3295012","deviceId":0}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"a6b006af-039e-4a00-bdeb-97ac623fb56e","ip":"10.102.125.188","netmask":"255.255.255.0","gateway":"10.102.125.1","mac":"06:bb:c4:00:00:09","dns1":"10.103.128.15","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://untagged","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"e694c359-0609-42b6-872b-f2191ee20241","ip":"169.254.1.202","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:ca","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"result":true,"wait":0}},{"check.CheckSshAnswer":{"result":true,"wait":0}},{"GetDomRVersionAnswer":{"result":false,"details":"GetDomRVersionCmd
> failed","wait":0}},{"Answer":{"result":false,"details":"Stopped by previous
> failure","wait":0}}] }
> 2012-09-17 15:10:31,882 DEBUG [agent.transport.Request]
> (Job-Executor-8:job-8) Seq 3-1151271005: Received: { Ans: , MgmtId:
> 55487956346259, via: 3, Ver: v1, Flags: 110, { StartAnswer, CheckSshAnswer,
> GetDomRVersionAnswer, Answer } }
> 2012-09-17 15:10:31,884 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-14:null) Seq 3-1151271005: No more commands found
> 2012-09-17 15:10:31,888 WARN
> [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-8:job-8)
> Unable to get the template/scripts version of router r-4-VM due to:
> GetDomRVersionCmd failed
> 2012-09-17 15:10:31,888 INFO [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-8:job-8) The guru did not like the answers so stopping
> VM[DomainRouter|r-4-VM]
> 2012-09-17 15:10:31,890 DEBUG [agent.transport.Request]
> (Job-Executor-8:job-8) Seq 3-1151271010: Sending { Cmd , MgmtId:
> 55487956346259, via: 3, Ver: v1, Flags: 100111,
> [{"StopCommand":{"isProxy":false,"vmName":"r-4-VM","wait":0}}] }
> Cloud Agent Error log :
> 2012-09-17 15:44:06,178 DEBUG [kvm.resource.KVMGuestOsMapper]
> (agentRequest-Handler-5:null) Can't find the mapping of guest os: Debian
> GNU/Linux 5.0 (32-bit)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
