Sowmya Krishnan created CLOUDSTACK-3590:
-------------------------------------------
Summary: Adding new Xen server host to cluster fails in first
attempt - Need to add the host again
Key: CLOUDSTACK-3590
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3590
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Hypervisor Controller
Affects Versions: 4.2.0
Environment: 4.2 branch, Xen Server 6.1
Reporter: Sowmya Krishnan
Priority: Critical
Fix For: 4.2.0
Adding a new host to a cluster fails at first attempt. Need to add again and
then it succeeds.
Although the hosts are of same CPU type, speed, count and flags. Hosts have all
patches installed as well.
Steps:
=====
1. Create CS setup with one Xen server cluster and add a host (XS version 6.1)
2. Prepare another XS 6.1 host of same CPU type, speed, count and flags.
Install all required patches
3. Add the host to the cluster
Result
======
The new host gets added to the existing cluster pool and the first host becomes
master of the pool. Cloudstack fails to add the host and throws the following
exception:
2013-07-17 17:36:51,519 DEBUG [xen.resource.XenServerConnectionPool]
(catalina-exec-19:null) Logging on as the slave to 10.102.192.9
2013-07-17 17:36:53,001 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-10:null) SeqA 2-45687: Processing Seq 2-45687: { Cmd ,
MgmtId: -1, via:
2, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2013-07-17 17:36:53,006 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-10:null) SeqA 2-45687: Sending Seq 2-45687: { Ans: ,
MgmtId: 2227715236
17711, via: 2, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2013-07-17 17:36:53,545 DEBUG [xen.resource.XenServerConnectionPool]
(catalina-exec-19:null) Logging on as the slave to 10.102.192.9
2013-07-17 17:36:55,571 DEBUG [xen.resource.XenServerConnectionPool]
(catalina-exec-19:null) Logging on as the slave to 10.102.192.9
2013-07-17 17:36:57,597 DEBUG [xen.resource.XenServerConnectionPool]
(catalina-exec-19:null) Logging on as the slave to 10.102.192.9
2013-07-17 17:36:59,623 DEBUG [xen.resource.XenServerConnectionPool]
(catalina-exec-19:null) Logging on as the slave to 10.102.192.9
2013-07-17 17:37:01,648 DEBUG [xen.resource.XenServerConnectionPool]
(catalina-exec-19:null) Logging on as the slave to 10.102.192.9
2013-07-17 17:37:03,001 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-12:null) SeqA 2-45688: Processing Seq 2-45688: { Cmd ,
MgmtId: -1, via: 2, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2013-07-17 17:37:03,006 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-12:null) SeqA 2-45688: Sending Seq 2-45688: { Ans: ,
MgmtId: 222771523617711, via: 2, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2013-07-17 17:37:03,679 DEBUG [xen.resource.XenServerConnectionPool]
(catalina-exec-19:null) Logging on as the slave to 10.102.192.9
2013-07-17 17:37:05,712 DEBUG [xen.resource.XenServerConnectionPool]
(catalina-exec-19:null) Host(10.102.192.9) unable to Join the pool at
10.102.192.14
2013-07-17 17:37:05,712 WARN [xen.discoverer.XcpServerDiscoverer]
(catalina-exec-19:null) Unable to join the pool
2013-07-17 17:37:05,716 INFO [utils.exception.CSExceptionErrorCode]
(catalina-exec-19:null) Could not find exception:
com.cloud.exception.DiscoveryException in error code list for exceptions
2013-07-17 17:37:05,716 DEBUG [xen.discoverer.XcpServerDiscoverer]
(catalina-exec-19:null) other exceptions:
com.cloud.exception.DiscoveryException: Unable to join the pool
com.cloud.exception.DiscoveryException: Unable to join the pool
at
com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer.addHostsToPool(XcpServerDiscoverer.java:409)
at
com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer.find(XcpServerDiscoverer.java:205)
at
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:785)
at
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:612)
at
org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:143)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:529)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:371)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:304)
at com.cloud.api.ApiServlet.doPost(ApiServlet.java:71)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-07-17 17:37:05,717 WARN [cloud.resource.ResourceManagerImpl]
(catalina-exec-19:null) Unable to find the server resources at
http://10.102.192.9
2013-07-17 17:37:05,717 INFO [utils.exception.CSExceptionErrorCode]
(catalina-exec-19:null) Could not find exception:
com.cloud.exception.DiscoveryException in error code list for exceptions
2013-07-17 17:37:05,717 WARN [admin.host.AddHostCmd] (catalina-exec-19:null)
Exception:
com.cloud.exception.DiscoveryException: Unable to add the host
at
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:842)
at
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:612)
at
org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:143)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:529)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:371)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:304)
at com.cloud.api.ApiServlet.doPost(ApiServlet.java:71)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-07-17 17:37:05,723 INFO [cloud.api.ApiServer] (catalina-exec-19:null)
Unable to add the host
2013-07-17 17:37:05,724 DEBUG [cloud.api.ApiServlet] (catalina-exec-19:null)
===END=== 10.151.138.80 -- POST
command=addHost&response=json&sessionkey=o3Ipoq0Ez7n4a7lDTZARwvEasrU%3D
Later, when i add the same host again, it succeeds:
2013-07-17 17:39:07,543 INFO [xen.discoverer.XcpServerDiscoverer]
(catalina-exec-5:null) Host: sowmya.xs61.9 connected with hypervisor type:
XenServer. Checking CIDR...
2013-07-17 17:39:07,584 DEBUG [cloud.resource.ResourceState]
(catalina-exec-5:null) Resource state update: [id = 21; name = sowmya.xs61.9;
old state = Creating; event = InternalCreated; new state = Enabled]
2013-07-17 17:39:07,584 DEBUG [cloud.host.Status] (catalina-exec-5:null)
Transition:[Resource state = Enabled, Agent event = AgentConnected, Host id =
21, name = sowmya.xs61.9]
2013-07-17 17:39:07,593 DEBUG [cloud.host.Status] (catalina-exec-5:null) Agent
status update: [id = 21; name = sowmya.xs61.9; old status = Creating; event =
AgentConnected; new status = Connecting; old update count = 0; new update count
= 1]
2013-07-17 17:39:07,598 DEBUG [cloud.host.Status] (catalina-exec-5:null)
Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host id
= 21, name = sowmya.xs61.9]
2013-07-17 17:39:07,604 DEBUG [cloud.host.Status] (catalina-exec-5:null) Agent
status update: [id = 21; name = sowmya.xs61.9; old status = Connecting; event =
AgentDisconnected; new status = Alert; old update count = 1; new update count =
2]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
