Did you check the storage configuration to see why the new host cannot
access it? There may be some additional details on the error in
/var/log/xensource.log and SMlog.
Regards,
Kirk
On 10/08/2012 06:58 PM, Hieu Le wrote:
> Hi everyone,
>
> I have some troubles while adding more XCP host in cluster. I have 2
> running hosts which are homogeneous and another host that is inhomogeneous.
> Now I want to add this inhomogeneous host to CS in the same cluster with 2
> running hosts but it is always fail. The step I follow is:
>
> - Force the inhomogeneous to join existing pool with xe command (xe
> pool-join --force=true --master-address=....)
> - Add new host in the same cluster with 2 existing hosts via CS UI.
> - Getting the error.
>
> The error log I got here is:
>
> 2012-10-09 09:19:04,334 DEBUG [cloud.storage.StorageManagerImpl]
> (catalina-exec-2:null) Adding pool XenPrimary to host 14
> 2012-10-09 09:19:04,336 DEBUG [agent.transport.Request]
> (catalina-exec-2:null) Seq 14-976879620: Sending { Cmd , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePoolC$
> 2012-10-09 09:19:04,337 DEBUG [agent.transport.Request]
> (catalina-exec-2:null) Seq 14-976879620: Executing: { Cmd , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePo$
> 2012-10-09 09:19:04,337 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-20:null) Seq 14-976879620: Executing request
> 2012-10-09 09:19:04,381 WARN [xen.resource.CitrixResourceBase]
> (DirectAgent-20:null) ModifyStoragePoolCommand add XenAPIException:Can not
> see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732$
> com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool:
> b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
> host:1949af01-e15e-467c-8d27-42678a5f826c
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
> at
> com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-10-09 09:19:04,382 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-20:null) Seq 14-976879620: Response Received:
> 2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
> (DirectAgent-20:null) Seq 14-976879620: Processing: { Ans: , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 10, [{"Answer":{"result":$
> 2012-10-09 09:19:04,382 WARN [cloud.vm.VirtualMachineManagerImpl]
> (DirectAgent-20:null) Cleanup failed due to ModifyStoragePoolCommand add
> XenAPIException:Can not see storage pool: b43e6f96-d0$
> 2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
> (catalina-exec-2:null) Seq 14-976879620: Received: { Ans: , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 10, { Answer } }
> 2012-10-09 09:19:04,383 WARN [cloud.vm.VirtualMachineManagerImpl]
> (catalina-exec-2:null) Cleanup failed due to ModifyStoragePoolCommand add
> XenAPIException:Can not see storage pool: b43e6f96-d$
> 2012-10-09 09:19:04,383 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Details from executing class
> com.cloud.agent.api.ModifyStoragePoolCommand: ModifyStoragePoolCommand add
> Xen$
> 2012-10-09 09:19:04,461 WARN [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Monitor StoragePoolMonitor says there is an error in
> the connect process for 14 due to Unable to connect to$
> 2012-10-09 09:19:04,461 INFO [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Host 14 is disconnecting with event AgentDisconnected
> 2012-10-09 09:19:04,462 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) The next status of agent 14is Alert, current status
> is Connecting
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Deregistering link for 14 with state Alert
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Remove Agent : 14
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.DirectAgentAttache]
> (catalina-exec-2:null) Processing disconnect 14
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer$$EnhancerByCGLIB$$4$
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.hypervisor.vmware.VmwareManagerImpl$$EnhancerByCGLIB$$8d05fd47
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.vm.ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$4b7c62a0
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.security.SecurityGroupListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.listener.StoragePoolMonitor
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.secondary.SecondaryStorageListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.NetworkManagerImpl$$EnhancerByCGLIB$$f53c41bc
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.agent.manager.AgentMonitor$$EnhancerByCGLIB$$65dcc41c
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.download.DownloadListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.upload.UploadListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.SshKeysDistriMonitor
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl$$EnhancerByCGLI$
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.capacity.StorageCapacityListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.capacity.ComputeCapacityListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.consoleproxy.ConsoleProxyListener
> 2012-10-09 09:19:04,464 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
> 2012-10-09 09:19:04,464 DEBUG [cloud.network.NetworkUsageManagerImpl]
> (catalina-exec-2:null) Disconnected called on 14 with status Alert
> 2012-10-09 09:19:04,464 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
> id = 14, name = node1.viettelrd.vn]
> 2012-10-09 09:19:04,532 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Agent status update: [id = 14; name = node1.viettelrd.vn; old status =
> Connecting; event = AgentDisconnected; new status$
> 2012-10-09 09:19:04,532 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (catalina-exec-2:null) Notifying other nodes of to disconnect
> 2012-10-09 09:19:04,534 WARN [cloud.resource.ResourceManagerImpl]
> (catalina-exec-2:null) Unable to connect due to
> com.cloud.exception.ConnectionException: Unable to connect to pool
> Pool[200|NetworkFilesystem]
> at
> com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:91)
> at
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:601)
> at
> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1496)
> at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1574)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
> at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> Caused by: com.cloud.exception.StorageUnavailableException: Resource
> [StoragePool:200] is unreachable: Unable establish connection from storage
> head to storage pool 200 due to ModifyStoragePool$
> at
> com.cloud.storage.StorageManagerImpl.connectHostToSharedPool(StorageManagerImpl.java:1567)
> at
> com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:88)
> ... 28 more
> 2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
> id = 14, name = node1.viettelrd.vn]
> 2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Cannot transit agent status with event AgentDisconnected for host 14, name=
> node1.viettelrd.vn, mangement server id is 14$
> 2012-10-09 09:19:04,535 ERROR [cloud.api.ApiDispatcher]
> (catalina-exec-2:null) Exception while executing AddHostCmd:
> com.cloud.utils.exception.CloudRuntimeException: Cannot transit agent
> status with event AgentDisconnected for host 14, mangement server id is
> 148045887322015,Unable to transition to a new state$
> at
> com.cloud.agent.manager.AgentManagerImpl.agentStatusTransitTo(AgentManagerImpl.java:1428)
> at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1588)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
> at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-10-09 09:19:08,143 WARN [xen.resource.CitrixResourceBase]
> (DirectAgent-9:null) ModifyStoragePoolCommand add XenAPIException:Can not
> see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732 $
> com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool:
> b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
> host:1949af01-e15e-467c-8d27-42678a5f826c
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
> at
> com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
>
> - When force joining the inhomogeneous host, Xen Center throws an error
> that it can not see the SR (primary storage) and can not connect to host
> via PBD.
>
> Please support !
>
> Best regards !
>
