RE: Management server down
Ok so I am back in the office and able to log into the servers. The management server UI is down with a tomcat 404 error the catalina.out has not rotated since 8.1 and has grown to 353MB while management-server.log is still from 8.1 and is 9MB in size. So now that I get my hands on my logs files. 2015-08-01 12:30:04,925 ERROR [c.c.u.d.Merovingian2] (main:null) Unable to get a new db connection com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up. Crap crap crap. I am able to ping my server farm but for some reason my management network cannot dig lookup from my management servers to the db servers. So I quickly fixed the nsservers and since I was able to ping but not hit named. Now my UI is back working. Jeremy -Original Message- From: Nick Brody [mailto:nickbrody2...@gmail.com] Sent: Sunday, August 2, 2015 2:51 AM To: users@cloudstack.apache.org Subject: Re: Management server down Superphones On Sat, Aug 1, 2015 at 11:47 AM, Ahmad Emneina aemne...@gmail.com wrote: Jeremy, can you paste logs to a service like pastebin... Typically the UI fails to load when the management server doesn't start properly... Management server logs and/Catalina.out should point to the problem. Ahmad E On Aug 1, 2015, at 11:34 AM, Milamber milam...@apache.org wrote: The screenshot of latest lines of catalina.out don't pass the mailing list (probably dropped) What says the cloudstack-management.log file when your restart the service ? (your CS version?) On 01/08/2015 19:06, Jeremy Peterson wrote: So out server farm is hosted in our corporate ESX environment. Our ESX hosts are over provisioned and we are waiting to get some UCS blades to bring our utilization down. But one of our hosts shutdown last night and HA was unable to keep all VM’s running. We fixed the issue our ESX host is back online but our cloudstack farm is unhappy. My DB servers are up and functional. My Management servers are online and can communicated to all servers My LB’s are good HA Proxy is working. My DNS servers are up and can resolve all servers and all servers can resolve to DNS propery. My hosts are online XenServer 6.5 SP1 and my vm’s are up. But when I restart cloudstack-management I end up with a tomcat 404 error. *HTTP Status 404 -* --- - *type* Status report *message* *description*_The requested resource () is not available._ --- - *Apache Tomcat/6.0.24* I am remote at this time and that is the last couple lines of catalina.out Anyone have any ideas? *Jeremy Peterson *
Re: Management server down
Superphones On Sat, Aug 1, 2015 at 11:47 AM, Ahmad Emneina aemne...@gmail.com wrote: Jeremy, can you paste logs to a service like pastebin... Typically the UI fails to load when the management server doesn't start properly... Management server logs and/Catalina.out should point to the problem. Ahmad E On Aug 1, 2015, at 11:34 AM, Milamber milam...@apache.org wrote: The screenshot of latest lines of catalina.out don't pass the mailing list (probably dropped) What says the cloudstack-management.log file when your restart the service ? (your CS version?) On 01/08/2015 19:06, Jeremy Peterson wrote: So out server farm is hosted in our corporate ESX environment. Our ESX hosts are over provisioned and we are waiting to get some UCS blades to bring our utilization down. But one of our hosts shutdown last night and HA was unable to keep all VM’s running. We fixed the issue our ESX host is back online but our cloudstack farm is unhappy. My DB servers are up and functional. My Management servers are online and can communicated to all servers My LB’s are good HA Proxy is working. My DNS servers are up and can resolve all servers and all servers can resolve to DNS propery. My hosts are online XenServer 6.5 SP1 and my vm’s are up. But when I restart cloudstack-management I end up with a tomcat 404 error. *HTTP Status 404 -* *type* Status report *message* *description*_The requested resource () is not available._ *Apache Tomcat/6.0.24* I am remote at this time and that is the last couple lines of catalina.out Anyone have any ideas? *Jeremy Peterson *
Re: Management server down
The screenshot of latest lines of catalina.out don't pass the mailing list (probably dropped) What says the cloudstack-management.log file when your restart the service ? (your CS version?) On 01/08/2015 19:06, Jeremy Peterson wrote: So out server farm is hosted in our corporate ESX environment. Our ESX hosts are over provisioned and we are waiting to get some UCS blades to bring our utilization down. But one of our hosts shutdown last night and HA was unable to keep all VM’s running. We fixed the issue our ESX host is back online but our cloudstack farm is unhappy. My DB servers are up and functional. My Management servers are online and can communicated to all servers My LB’s are good HA Proxy is working. My DNS servers are up and can resolve all servers and all servers can resolve to DNS propery. My hosts are online XenServer 6.5 SP1 and my vm’s are up. But when I restart cloudstack-management I end up with a tomcat 404 error. *HTTP Status 404 -* *type* Status report *message* *description*_The requested resource () is not available._ *Apache Tomcat/6.0.24* I am remote at this time and that is the last couple lines of catalina.out Anyone have any ideas? *Jeremy Peterson *
Re: Management server down
Jeremy, can you paste logs to a service like pastebin... Typically the UI fails to load when the management server doesn't start properly... Management server logs and/Catalina.out should point to the problem. Ahmad E On Aug 1, 2015, at 11:34 AM, Milamber milam...@apache.org wrote: The screenshot of latest lines of catalina.out don't pass the mailing list (probably dropped) What says the cloudstack-management.log file when your restart the service ? (your CS version?) On 01/08/2015 19:06, Jeremy Peterson wrote: So out server farm is hosted in our corporate ESX environment. Our ESX hosts are over provisioned and we are waiting to get some UCS blades to bring our utilization down. But one of our hosts shutdown last night and HA was unable to keep all VM’s running. We fixed the issue our ESX host is back online but our cloudstack farm is unhappy. My DB servers are up and functional. My Management servers are online and can communicated to all servers My LB’s are good HA Proxy is working. My DNS servers are up and can resolve all servers and all servers can resolve to DNS propery. My hosts are online XenServer 6.5 SP1 and my vm’s are up. But when I restart cloudstack-management I end up with a tomcat 404 error. *HTTP Status 404 -* *type* Status report *message* *description*_The requested resource () is not available._ *Apache Tomcat/6.0.24* I am remote at this time and that is the last couple lines of catalina.out Anyone have any ideas? *Jeremy Peterson *
