prashant kumar mishra created CLOUDSTACK-4137:
-------------------------------------------------
Summary: KVM: After unmanaging cluster, manage cluster will not
bring KVM hosts to UP state. cloud-agent on KVM hosts has to be restarted
manually
Key: CLOUDSTACK-4137
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4137
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: KVM
Affects Versions: 4.2.0
Environment: hypervisor : KVM
Reporter: prashant kumar mishra
Fix For: 4.2.0
Steps to reproduce
---------------------------
1-prepare a CS ->one cluster-->one kvm(rhel6.3)
2-unmanage cluster
3-manage cluster
Expected
--------------
Host should come up , in running state
Actual
----------
Host remain in Disconnect state
My observation
-----------------------
1-after host went in disconnect state i performed host maintenance mode then
cancel maintenance mode and host came up
2-restart cloud-agent on the kvm hosts bring the hosts to UP state
Logs:
----------
2013-08-07 20:14:16,999 DEBUG [cloud.api.ApiServlet] (catalina-exec-23:null)
===START=== 10.252.192.53 -- GET
command=updateCluster&id=854b547a-fbee-4eed-895d-b8ea96d1cc23&managedstate=Unmanaged&response=json&sessionkey=vOfsgLOiksyXg%2B23XuFp6maNm1I%3D&_=1375867173396
2013-08-07 20:14:17,031 DEBUG [agent.transport.Request] (catalina-exec-23:null)
Seq 1-2042691611: Sending { Cmd , MgmtId: 6703101771911, via: 1, Ver: v1,
Flags: 100111, [{"com.cloud.agent.api.MaintainCommand":{"wait":0}}] }
2013-08-07 20:14:17,038 DEBUG [agent.transport.Request]
(AgentManager-Handler-7:null) Seq 1-2042691611: Processing: { Ans: , MgmtId:
6703101771911, via: 1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.MaintainAnswer":{"willMigrate":true,"result":true,"wait":0}}]
}
2013-08-07 20:14:17,038 DEBUG [agent.manager.AgentAttache]
(AgentManager-Handler-7:null) Seq 1-2042691611: No more commands found
2013-08-07 20:14:17,038 DEBUG [agent.transport.Request] (catalina-exec-23:null)
Seq 1-2042691611: Received: { Ans: , MgmtId: 6703101771911, via: 1, Ver: v1,
Flags: 110, { MaintainAnswer } }
2013-08-07 20:14:17,039 INFO [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Host 1 is disconnecting with event ShutdownRequested
2013-08-07 20:14:17,051 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) The next status of agent 1is Disconnected, current
status is Up
2013-08-07 20:14:17,051 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Deregistering link for 1 with state Disconnected
2013-08-07 20:14:17,051 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Remove Agent : 1
2013-08-07 20:14:17,052 DEBUG [agent.manager.ConnectedAgentAttache]
(AgentTaskPool-4:null) Processing Disconnect.
2013-08-07 20:14:17,052 DEBUG [agent.manager.AgentAttache]
(AgentTaskPool-4:null) Seq 1-2042691586: Sending disconnect to class
com.cloud.network.security.SecurityGroupListener
2013-08-07 20:14:17,052 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer_EnhancerByCloudStack_eccb8bca
2013-08-07 20:14:17,053 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.deploy.DeploymentPlanningManagerImpl_EnhancerByCloudStack_b3901640
2013-08-07 20:14:17,053 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.network.NetworkManagerImpl_EnhancerByCloudStack_c52127d3
2013-08-07 20:14:17,053 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.storage.secondary.SecondaryStorageListener
2013-08-07 20:14:17,053 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.hypervisor.vmware.manager.VmwareManagerImpl_EnhancerByCloudStack_5c9626cd
2013-08-07 20:14:17,053 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.network.security.SecurityGroupListener
2013-08-07 20:14:17,053 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.storage.listener.StoragePoolMonitor
2013-08-07 20:14:17,054 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.vm.ClusteredVirtualMachineManagerImpl_EnhancerByCloudStack_f1e1d8d7
2013-08-07 20:14:17,054 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.storage.LocalStoragePoolListener
2013-08-07 20:14:17,054 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.network.SshKeysDistriMonitor
2013-08-07 20:14:17,054 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.network.router.VirtualNetworkApplianceManagerImpl_EnhancerByCloudStack_8b534578
2013-08-07 20:14:17,054 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.network.SshKeysDistriMonitor
2013-08-07 20:14:17,056 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl_EnhancerByCloudStack_6370d9b
2013-08-07 20:14:17,056 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.storage.upload.UploadListener
2013-08-07 20:14:17,056 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.storage.download.DownloadListener
2013-08-07 20:14:17,056 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.agent.manager.AgentMonitor
2013-08-07 20:14:17,057 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.capacity.StorageCapacityListener
2013-08-07 20:14:17,057 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.capacity.ComputeCapacityListener
2013-08-07 20:14:17,058 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
2013-08-07 20:14:17,058 DEBUG [cloud.network.NetworkUsageManagerImpl]
(AgentTaskPool-4:null) Disconnected called on 1 with status Disconnected
2013-08-07 20:14:17,058 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-4:null) Sending Disconnect to listener:
com.cloud.consoleproxy.ConsoleProxyListener
2013-08-07 20:14:17,061 DEBUG [cloud.host.Status] (AgentTaskPool-4:null)
Transition:[Resource state = Enabled, Agent event = ShutdownRequested, Host id
= 1, name = Rack1Pod1Host18]
2013-08-07 20:14:17,081 DEBUG [cloud.host.Status] (AgentTaskPool-4:null) Agent
status update: [id = 1; name = Rack1Pod1Host18; old status = Up; event =
ShutdownRequested; new status = Disconnected; old update count = 7; new update
count = 8]
2013-08-07 20:14:17,082 DEBUG [agent.manager.ClusteredAgentManagerImpl]
(AgentTaskPool-4:null) Notifying other nodes of to disconnect
2013-08-07 20:14:21,077 DEBUG [storage.secondary.SecondaryStorageManagerImpl]
(secstorage-1:null) Zone 1 is not ready to launch secondary storage VM yet
2013-08-07 20:14:21,588 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl]
(consoleproxy-1:null) Zone 1 is not ready to launch console proxy yet
2013-08-07 20:15:31,485 INFO [agent.manager.AgentMonitor] (Thread-6:null)
Found the following agents behind on ping: [2]
2013-08-07 20:15:31,489 DEBUG [cloud.host.Status] (Thread-6:null) Ping timeout
for host 2, do invstigation
2013-08-07 20:15:31,494 INFO [agent.manager.AgentManagerImpl]
(AgentTaskPool-5:null) Investigating why host 2 has disconnected with event
PingTimeout
2013-08-07 20:15:31,496 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-5:null) checking if agent (2) is alive
2013-08-07 20:15:31,501 DEBUG [agent.transport.Request] (AgentTaskPool-5:null)
Seq 2-55508996: Sending { Cmd , MgmtId: 6703101771911, via: 2, Ver: v1, Flags:
100011, [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
2013-08-07 20:15:31,549 DEBUG [agent.transport.Request]
(AgentManager-Handler-8:null) Seq 2-55508996: Processing: { Ans: , MgmtId:
6703101771911, via: 2, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource is
alive","wait":0}}] }
2013-08-07 20:15:31,549 DEBUG [agent.transport.Request] (AgentTaskPool-5:null)
Seq 2-55508996: Received: { Ans: , MgmtId: 6703101771911, via: 2, Ver: v1,
Flags: 10, { CheckHealthAnswer } }
2013-08-07 20:15:31,549 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-5:null) Details from executing class
com.cloud.agent.api.CheckHealthCommand: resource is alive
2013-08-07 20:15:31,550 DEBUG [agent.manager.AgentManagerImpl]
(AgentTaskPool-5:null) agent (2) responded to checkHeathCommand, reporting that
agent is Up
2013-08-07 20:15:31,550 INFO [agent.manager.AgentManagerImpl]
(AgentTaskPool-5:null) The state determined is Up
2013-08-07 20:15:31,550 INFO [agent.manager.AgentManagerImpl]
(AgentTaskPool-5:null) Agent is determined to be up and running
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
