https://cwiki.apache.org/CLOUDSTACK/ports-used-by-cloudstack.html
Port 443 is not listed there for Management server On Wed, Jul 17, 2013 at 10:59 AM, Leeno Jose.P.A <leeno...@gmail.com> wrote: > Hi Ahmad, > > Thanks for the mail. > Telnet failed on port 443 to CS from XS hosts. I did not see any service > listening on 443 on CS management server. Something wrong with management > server? > > Thanks > Leeno > > > On Tue, Jul 16, 2013 at 11:50 PM, Ahmad Emneina <aemne...@gmail.com>wrote: > >> can you check if your hosts can connect back to the management server >> (ping, telnet 22,443)? there might be some firewall rules in place, or >> routing issues, preventing this. >> >> >> On Tue, Jul 16, 2013 at 9:01 AM, Leeno Jose.P.A <leeno...@gmail.com> >> wrote: >> >> > CS startup logs, >> > >> > 2013-07-16 11:25:30,702 INFO [utils.component.ComponentContext] >> > (Timer-1:null) Starting >> > >> com.cloud.network.guru.NiciraNvpGuestNetworkGuru_EnhancerByCloudStack_1f6b4bb6 >> > 2013-07-16 11:25:30,702 INFO [utils.component.ComponentContext] >> > (Timer-1:null) Starting >> > com.cloud.server.ManagementServerImpl_EnhancerByCloudStack_d54e1bb1 >> > 2013-07-16 11:25:30,702 INFO [cloud.server.ManagementServerImpl] >> > (Timer-1:null) Startup CloudStack management server... >> > 2013-07-16 11:25:30,707 INFO >> > [cloud.cluster.ClusterServiceServletContainer] (Thread-18:null) Cluster >> > service servlet container listening on port 9090 >> > 2013-07-16 11:25:31,832 DEBUG [utils.db.ConnectionConcierge] >> > (Cluster-Heartbeat-1:null) Registering a database connection for >> > ClusterManagerHeartBeat2 >> > 2013-07-16 11:25:31,845 INFO [cloud.cluster.ClusterManagerImpl] >> > (Cluster-Heartbeat-1:null) We are good, no orphan management server >> msid in >> > host table is found >> > 2013-07-16 11:25:31,845 INFO [cloud.cluster.ClusterManagerImpl] >> > (Cluster-Heartbeat-1:null) Found 1 inactive management server node >> based on >> > timestamp >> > 2013-07-16 11:25:31,846 INFO [cloud.cluster.ClusterManagerImpl] >> > (Cluster-Heartbeat-1:null) management server node msid: 130602634328, >> name: >> > cstagcms, service ip: 192.168.10.251, version: 4.1.0 >> > 2013-07-16 11:25:31,846 INFO [cloud.cluster.ClusterManagerImpl] >> > (Cluster-Heartbeat-1:null) Trying to connect to 192.168.10.251 >> > 2013-07-16 11:25:31,860 DEBUG [cloud.cluster.ClusterManagerImpl] >> > (Cluster-Heartbeat-1:null) Detected management node joined, id:2, >> > nodeIP:192.168.10.251 >> > 2013-07-16 11:25:33,348 DEBUG [cloud.cluster.ClusterManagerImpl] >> > (Cluster-Notification-1:null) Notify management server node join to >> > listeners. >> > 2013-07-16 11:25:33,349 DEBUG [cloud.cluster.ClusterManagerImpl] >> > (Cluster-Notification-1:null) Joining node, IP: 192.168.10.251, msid: >> > 81375086018793 >> > 2013-07-16 11:25:33,350 DEBUG [cloud.alert.ClusterAlertAdapter] >> > (Cluster-Notification-1:null) Receive cluster alert, EventArgs: >> > com.cloud.cluster.ClusterNodeJoinEventArgs >> > 2013-07-16 11:25:33,350 DEBUG [cloud.alert.ClusterAlertAdapter] >> > (Cluster-Notification-1:null) Handle cluster node join alert, joined >> node: >> > 192.168.10.251, msidL: 81375086018793 >> > 2013-07-16 11:25:33,350 DEBUG [cloud.alert.ClusterAlertAdapter] >> > (Cluster-Notification-1:null) Management server node 192.168.10.251 is >> up, >> > send alert >> > 2013-07-16 11:25:33,361 WARN [cloud.cluster.ClusterManagerImpl] >> > (Cluster-Notification-1:null) Notifying management server join event >> took >> > 12 ms >> > 2013-07-16 11:25:45,450 DEBUG [cloud.server.StatsCollector] >> > (StatsCollector-2:null) HostStatsCollector is running... >> > 2013-07-16 11:25:45,452 DEBUG [cloud.server.StatsCollector] >> > (StatsCollector-1:null) VmStatsCollector is running... >> > 2013-07-16 11:25:45,467 DEBUG [cloud.server.StatsCollector] >> > (StatsCollector-3:null) StorageCollector is running... >> > 2013-07-16 11:25:45,498 DEBUG [agent.manager.ClusteredAgentManagerImpl] >> > (StatsCollector-2:null) create forwarding ClusteredAgentAttache for 39 >> > 2013-07-16 11:25:45,491 DEBUG [agent.manager.ClusteredAgentManagerImpl] >> > (StatsCollector-3:null) create forwarding ClusteredAgentAttache for 50 >> > 2013-07-16 11:25:45,751 INFO [agent.manager.ClusteredAgentManagerImpl] >> > (StatsCollector-3:null) SSL: Handshake done >> > 2013-07-16 11:25:45,752 DEBUG [agent.manager.ClusteredAgentManagerImpl] >> > (StatsCollector-3:null) Connection to peer opened: 130602634328, ip: >> > 192.168.10.251 >> > 2013-07-16 11:25:45,757 DEBUG [agent.manager.ClusteredAgentAttache] >> > (StatsCollector-2:null) Seq 39-282525697: Forwarding null to >> 130602634328 >> > 2013-07-16 11:25:45,758 DEBUG [agent.manager.ClusteredAgentAttache] >> > (StatsCollector-3:null) Seq 50-1962541057: Forwarding null to >> 130602634328 >> > 2013-07-16 11:25:45,804 DEBUG [agent.manager.ClusteredAgentAttache] >> > (AgentManager-Handler-2:null) Seq 39-282525697: Routing from >> 81375086018793 >> > 2013-07-16 11:25:45,804 DEBUG [agent.manager.ClusteredAgentAttache] >> > (AgentManager-Handler-2:null) Seq 39-282525697: Link is closed >> > 2013-07-16 11:25:45,806 DEBUG [agent.manager.ClusteredAgentManagerImpl] >> > (AgentManager-Handler-2:null) Seq 39-282525697: MgmtId 81375086018793: >> Req: >> > Resource [Host:39] is unreachable: Host 39: Link is closed >> > >> > >> > Thanks >> > Leeno >> > >> > >> > On Tue, Jul 16, 2013 at 6:10 PM, Leeno Jose.P.A <leeno...@gmail.com >> >wrote: >> > >> >> Hi Todd, >> >> >> >> Thanks for the help. >> >> >> >> I executed the steps as you mentioned above but that did not help. >> Still >> >> I get same error message. But I can do ping, telnet ports 22, 80 and >> 443 on >> >> XS hosts from CS. >> >> >> >> Thanks >> >> Leeno >> >> >> >> >> >> On Tue, Jul 16, 2013 at 5:12 PM, Todd Pigram <t...@toddpigram.com> >> wrote: >> >> >> >>> Did you remove the Tags on each XenServer host prior to starting? >> >>> >> >>> Management Controller Failure and Replacement >> >>> >> >>> < >> https://cwiki.apache.org/confluence/pages/editpage.action?pageId=30755366 >> > >> >>> Edit Page< >> https://cwiki.apache.org/confluence/pages/editpage.action?pageId=30755366 >> > >> >>> < >> https://cwiki.apache.org/confluence/pages/listpages.action?key=CLOUDSTACK >> > >> >>> Browse Space< >> https://cwiki.apache.org/confluence/pages/listpages.action?key=CLOUDSTACK >> > >> >>> < >> https://cwiki.apache.org/confluence/pages/createpage.action?spaceKey=CLOUDSTACK&fromPageId=30755366 >> > >> >>> Add Page< >> https://cwiki.apache.org/confluence/pages/createpage.action?spaceKey=CLOUDSTACK&fromPageId=30755366 >> > >> >>> < >> https://cwiki.apache.org/confluence/pages/createblogpost.action?spaceKey=CLOUDSTACK&fromPageId=30755366 >> > >> >>> Add News< >> https://cwiki.apache.org/confluence/pages/createblogpost.action?spaceKey=CLOUDSTACK&fromPageId=30755366 >> > >> >>> >> >>> In setting up your cloud, you should have a backup routine for your >> >>> controller. The most important item to back up is the MySQL databases >> that >> >>> Cloudstack uses. A suitable backup script is attached to this page. >> In the >> >>> even of a cloud management controller failure, the steps to replace >> the >> >>> controller with a new one are: >> >>> >> >>> These instructions assume your cluster is Xenserver - Contributors >> >>> using other Hypervisor OSs, please contribute. >> >>> >> >>> 1. Setup new management server hardware >> >>> 2. Install your OS >> >>> 3. Install Cloudstack, up to and including the "Install Database >> >>> step" >> >>> 4. Import your database backup >> >>> 5. In Xencenter, connect to your Cloudstack host pool. >> >>> 6. On each host, remove the tags on Host > General Tab > Tags by >> >>> editing the tags and un-checking each one. >> >>> 7. On the management controller, start Cloudstack >> >>> 1. service cloud-management start >> >>> 8. the new cloud management controller will connect to each host in >> >>> the database and push out new tags and keys to each host in the >> pool. >> >>> >> >>> >> >>> On Jul 16, 2013, at 1:13 AM, Leeno Jose.P.A <leeno...@gmail.com> >> wrote: >> >>> >> >>> After restoring the old database dump to new installation. CS is >> unable >> >>> to >> >>> contact Xenserver hosts. I getting following errors in >> >>> mamangement-server.log, >> >>> >> >>> >> >>> 2013-07-15 11:57:49,646 DEBUG >> [agent.manager.ClusteredAgentManagerImpl] >> >>> (StatsCollector-1:null) Connection to peer opened: 130602634328, ip: >> >>> 192.168.10.251 >> >>> 2013-07-15 11:57:49,652 DEBUG [agent.manager.ClusteredAgentAttache] >> >>> (StatsCollector-2:null) Seq 50-185008129: Forwarding null to >> 130602634328 >> >>> 2013-07-15 11:57:49,662 DEBUG [agent.manager.ClusteredAgentAttache] >> >>> (StatsCollector-1:null) Seq 39-1272840193: Forwarding null to >> >>> 130602634328 >> >>> 2013-07-15 11:57:49,699 DEBUG [agent.manager.ClusteredAgentAttache] >> >>> (AgentManager-Handler-2:null) Seq 50-185008129: Routing from >> >>> 81375086018793 >> >>> 2013-07-15 11:57:49,699 DEBUG [agent.manager.ClusteredAgentAttache] >> >>> (AgentManager-Handler-2:null) Seq 50-185008129: Link is closed >> >>> 2013-07-15 11:57:49,699 DEBUG [agent.manager.ClusteredAgentAttache] >> >>> (AgentManager-Handler-3:null) Seq 39-1272840193: Routing from >> >>> 81375086018793 >> >>> 2013-07-15 11:57:49,700 DEBUG [agent.manager.ClusteredAgentAttache] >> >>> (AgentManager-Handler-3:null) Seq 39-1272840193: Link is closed >> >>> 2013-07-15 11:57:49,700 DEBUG >> [agent.manager.ClusteredAgentManagerImpl] >> >>> (AgentManager-Handler-3:null) Seq 39-1272840193: MgmtId >> 81375086018793: >> >>> Req: Resource [Host:39] is unreachable: Host 39: Link is closed >> >>> >> >>> >> >>> 2013-07-15 11:57:49,861 DEBUG >> [agent.manager.ClusteredAgentManagerImpl] >> >>> (AgentManager-Handler-8:null) Seq 39--1: MgmtId 81375086018793: Req: >> >>> Cancel >> >>> request received >> >>> 2013-07-15 11:57:49,861 DEBUG [agent.manager.AgentAttache] >> >>> (AgentManager-Handler-8:null) Seq 39-1272840194: Cancelling. >> >>> 2013-07-15 11:57:49,861 DEBUG [agent.manager.AgentAttache] >> >>> (StatsCollector-2:null) Seq 39-1272840194: Waiting some more time >> because >> >>> this is the current command >> >>> 2013-07-15 11:57:49,862 DEBUG [agent.manager.AgentAttache] >> >>> (StatsCollector-2:null) Seq 39-1272840194: Waiting some more time >> because >> >>> this is the current command >> >>> 2013-07-15 11:57:49,862 INFO [utils.exception.CSExceptionErrorCode] >> >>> (StatsCollector-2:null) Could not find exception: >> >>> com.cloud.exception.OperationTimedoutException in error code list for >> >>> exceptions >> >>> 2013-07-15 11:57:49,862 WARN [agent.manager.AgentAttache] >> >>> (StatsCollector-2:null) Seq 39-1272840194: Timed out on null >> >>> 2013-07-15 11:57:49,862 DEBUG [agent.manager.AgentAttache] >> >>> (StatsCollector-2:null) Seq 39-1272840194: Cancelling. >> >>> 2013-07-15 11:57:49,863 DEBUG [cloud.storage.StorageManagerImpl] >> >>> (StatsCollector-2:null) Unable to send storage pool command to >> >>> Pool[210|NetworkFilesystem] via 39 >> >>> com.cloud.exception.OperationTimedoutException: Commands 1272840194 to >> >>> Host >> >>> 39 timed out after 3600 >> >>> at >> >>> com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:429) >> >>> at >> >>> >> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:511) >> >>> at >> >>> >> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:464) >> >>> at >> >>> >> >>> >> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:2347) >> >>> at >> >>> >> >>> >> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:422) >> >>> at >> >>> >> >>> >> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:436) >> >>> at >> >>> >> >>> >> com.cloud.server.StatsCollector$StorageCollector.run(StatsCollector.java:316) >> >>> at >> >>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> >>> at >> >>> >> >>> >> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) >> >>> at >> >>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) >> >>> at >> >>> >> >>> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) >> >>> at >> >>> >> >>> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) >> >>> at >> >>> >> >>> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >> >>> at >> >>> >> >>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> >>> at java.lang.Thread.run(Thread.java:679) >> >>> 2013-07-15 11:57:49,863 INFO [cloud.server.StatsCollector] >> >>> (StatsCollector-2:null) Unable to reach Pool[210|NetworkFilesystem] >> >>> com.cloud.exception.StorageUnavailableException: Resource >> >>> [StoragePool:210] >> >>> is unreachable: Unable to send command to the pool >> >>> at >> >>> >> >>> >> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:2357) >> >>> at >> >>> >> >>> >> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:422) >> >>> at >> >>> >> >>> >> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:436) >> >>> at >> >>> >> >>> >> com.cloud.server.StatsCollector$StorageCollector.run(StatsCollector.java:316) >> >>> at >> >>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> >>> at >> >>> >> >>> >> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) >> >>> at >> >>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) >> >>> at >> >>> >> >>> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) >> >>> at >> >>> >> >>> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) >> >>> at >> >>> >> >>> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >> >>> at >> >>> >> >>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> >>> at java.lang.Thread.run(Thread.java:679) >> >>> >> >>> >> >>> Thanks >> >>> Leeno >> >>> >> >>> >> >>> On Tue, Jul 16, 2013 at 10:21 AM, Leeno Jose.P.A <leeno...@gmail.com> >> >>> wrote: >> >>> >> >>> This is a dev box. We are planning a HA enabled environment for prod >> >>> setup. Thanks Geoff. >> >>> >> >>> >> >>> On Tue, Jul 16, 2013 at 12:11 AM, Geoff Higginbottom < >> >>> geoff.higginbot...@shapeblue.com> wrote: >> >>> >> >>> Hi Leeno, >> >>> >> >>> It theory that should work, but obviously you will lose all changes >> made >> >>> since the dump was taken. If any new VMs have been created, they will >> >>> get >> >>> purged by the system etc. >> >>> >> >>> I would highly recommend splitting the DB and the Management Server, >> and >> >>> if possible add a 2nd instance of each. >> >>> >> >>> Regards >> >>> >> >>> Geoff Higginbottom >> >>> >> >>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 >> >>> >> >>> geoff.higginbot...@shapeblue.com >> >>> >> >>> -----Original Message----- >> >>> From: Leeno Jose.P.A [mailto:leeno...@gmail.com] >> >>> Sent: 15 July 2013 18:46 >> >>> To: users@cloudstack.apache.org >> >>> Subject: Re: Rebuilding management server >> >>> >> >>> Hi Geoff, >> >>> >> >>> 1. I have only one management server. >> >>> 2. Management server is not functioning now but 'cloud' database dump >> is >> >>> available in backup. CS version was 4.1.0 Hosts were Xenserver 6.1.0 >> 3. >> >>> DB >> >>> server was on same machine where management server installed. >> >>> >> >>> Now I am planning to do a fresh install of CS 4.1.0 and restore cloud >> >>> database with old installation dump, which is available in backup. >> Will >> >>> it >> >>> work? >> >>> >> >>> Thanks >> >>> Leeno >> >>> >> >>> >> >>> On Mon, Jul 15, 2013 at 9:56 PM, Geoff Higginbottom < >> >>> geoff.higginbot...@shapeblue.com> wrote: >> >>> >> >>> The Management Servers are 'Stateless' so as Chip points out, it's the >> >>> DB that stores all the info. >> >>> >> >>> How you actually go about it depends on your current setup. >> >>> >> >>> 1. How many management servers do you currently have? >> >>> 2. Are the original Management Server(s) still functioning, or are >> >>> they down? >> >>> 3. Is DB on a separate server, or the same as the Management Server? >> >>> >> >>> Regards >> >>> >> >>> Geoff Higginbottom >> >>> >> >>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 >> >>> >> >>> geoff.higginbot...@shapeblue.com >> >>> >> >>> -----Original Message----- >> >>> From: Chip Childers [mailto:chip.child...@sungard.com] >> >>> Sent: 15 July 2013 15:50 >> >>> To: users@cloudstack.apache.org >> >>> Subject: Re: Rebuilding management server >> >>> >> >>> On Mon, Jul 15, 2013 at 03:19:42PM +0530, Leeno Jose.P.A wrote: >> >>> >> >>> Hi Users, >> >>> >> >>> Has anyone tried to rebuild management server with Xenserver hosts? >> >>> If yes, could you please share experience? >> >>> >> >>> >> >>> -- >> >>> Leeno Jose .P.A >> >>> >> >>> >> >>> I have not, but one of the most critical aspects of this is to ensure >> >>> that your database is retained. >> >>> >> >>> This email and any attachments to it may be confidential and are >> >>> intended solely for the use of the individual to whom it is addressed. >> >>> Any views or opinions expressed are solely those of the author and do >> >>> not necessarily represent those of Shape Blue Ltd or related >> >>> companies. If you are not the intended recipient of this email, you >> >>> must neither take any action based upon its contents, nor copy or show >> >>> it to anyone. Please contact the sender if you believe you have >> >>> received this email in error. Shape Blue Ltd is a company incorporated >> >>> in England & Wales. ShapeBlue Services India LLP is operated under >> >>> license from Shape Blue Ltd. ShapeBlue is a registered trademark. >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Leeno Jose .P.A >> >>> This email and any attachments to it may be confidential and are >> intended >> >>> solely for the use of the individual to whom it is addressed. Any >> views >> >>> or >> >>> opinions expressed are solely those of the author and do not >> necessarily >> >>> represent those of Shape Blue Ltd or related companies. If you are not >> >>> the >> >>> intended recipient of this email, you must neither take any action >> based >> >>> upon its contents, nor copy or show it to anyone. Please contact the >> >>> sender >> >>> if you believe you have received this email in error. Shape Blue Ltd >> is a >> >>> company incorporated in England & Wales. ShapeBlue Services India LLP >> is >> >>> operated under license from Shape Blue Ltd. ShapeBlue is a registered >> >>> trademark. >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Leeno Jose .P.A >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Leeno Jose .P.A >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> Todd Pigram >> >>> t...@toddpigram.com >> >>> >> >>> >> >>> >> >> >> >> >> >> -- >> >> Leeno Jose .P.A >> >> >> > >> > >> > >> > -- >> > Leeno Jose .P.A >> > >> > > > > -- > Leeno Jose .P.A > -- Leeno Jose .P.A
