Hi Peter, The error from your usage.log says that the `db.cloud.maxActive` from your `db.properties` file is empty, and you have to provide a value for it. Also, if those properties are empty too - `db.cloud.maxIdle` and `db.cloud.maxWait` - the init of DB will fail again.
Best regards, Slavka On Wed, Dec 1, 2021 at 9:55 AM Peter Stine <peter.st...@granddial.com.invalid> wrote: > Wei, > > That got the Zone to recognize as up and ready. > To the database point, in the usage log, I now see that is is failing to > load the db configuration. As I am not using default logins, this means it > won't correctly connect to the database. I am not sure what is stopping > cloudstack from loading the database configuration. > > Here is the warning: > 2021-11-30 18:09:34,747 WARN [db.Transaction.Transaction] (main:null) > (logid:) Unable to load db configuration, using defaults with 5 > connections. Falling back on assumed datasource on localhost:3306 using > username:password=cloud:cloud. Please check your configuration > java.lang.NumberFormatException: null > at java.base/java.lang.Integer.parseInt(Integer.java:614) > at java.base/java.lang.Integer.parseInt(Integer.java:770) > at > com.cloud.utils.db.TransactionLegacy.initDataSource(TransactionLegacy.java:1030) > at > com.cloud.utils.db.TransactionLegacy.<clinit>(TransactionLegacy.java:1008) > at > com.cloud.utils.db.TransactionContextBuilder.interceptStart(TransactionContextBuilder.java:49) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:116) > at > com.cloud.usage.dao.UsagePortForwardingRuleDaoImpl_EnhancerByCloudStack_56282efc.createPartialSelectSql(<generated>) > at com.cloud.utils.db.GenericDaoBase.<init>(GenericDaoBase.java:238) > at > com.cloud.usage.dao.UsagePortForwardingRuleDaoImpl.<init>(UsagePortForwardingRuleDaoImpl.java:50) > at > com.cloud.usage.dao.UsagePortForwardingRuleDaoImpl_EnhancerByCloudStack_56282efc.<init>(<generated>) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at > java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) > at net.sf.cglib.core.ReflectUtils.newInstance(ReflectUtils.java:291) > at > net.sf.cglib.proxy.Enhancer$EnhancerFactoryData.newInstance(Enhancer.java:448) > at net.sf.cglib.proxy.Enhancer.nextInstance(Enhancer.java:700) > at > net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:298) > at net.sf.cglib.proxy.Enhancer.createHelper(Enhancer.java:480) > at net.sf.cglib.proxy.Enhancer.create(Enhancer.java:305) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor.postProcessBeforeInstantiation(ComponentInstantiationPostProcessor.java:92) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInstantiation(AbstractAutowireCapableBeanFactory.java:1144) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.resolveBeforeInstantiation(AbstractAutowireCapableBeanFactory.java:1119) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520) > at > org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) > at > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) > at > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) > at > org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:944) > at > org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:923) > at > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:588) > at > org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:144) > at > org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:85) > at com.cloud.usage.UsageServer.start(UsageServer.java:52) > at com.cloud.usage.UsageServer.main(UsageServer.java:41) > > ---------------------------------------- > From: "Wei ZHOU" <ustcweiz...@gmail.com> > Sent: 11/30/21 1:00 PM > To: users <users@cloudstack.apache.org>, peter.st...@granddial.com > Subject: Re: Zone not ready to launch system VMs though they are already > running... > > Hi Peter, > > This seems to be caused by database connection problems. Try to restart > cloudstack management service, and also mariadb if it still doesn't work. > > -Wei > > On Tue, 30 Nov 2021 at 18:37, Peter Stine <peter.st...@granddial.com.invalid> > wrote: > Hello everyone, > > I was getting a warning for high CPU usage on my management servers and > found that the spikes were in line with the cloudstack logs showing a > message that the Zone is not prepared to launch the consoleproxy and > secondary storage VMs. > > 2021-11-30 09:40:58,957 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] > (secstorage-1:ctx-ff6437c2) (logid:b30a1545) Zone 2 has no host available > which is enabled and in Up state > 2021-11-30 09:40:58,958 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] > (secstorage-1:ctx-ff6437c2) (logid:b30a1545) Zone 2 is not ready to launch > secondary storage VM yet > 2021-11-30 09:40:59,751 DEBUG [c.c.c.ConsoleProxyManagerImpl] > (consoleproxy-1:ctx-95609472) (logid:6b4db700) Zone 2 has no host available > which is enabled and in Up state > 2021-11-30 09:40:59,751 DEBUG [c.c.c.ConsoleProxyManagerImpl] > (consoleproxy-1:ctx-95609472) (logid:6b4db700) Zone 2 is not ready to > launch console proxy yet > > The odd thing is that both of those VMs are currently running, accessible, > and can be modified from the webUI. The zone also has 2 hosts, both of > which seem to be operating just fine. > However, there seems to be a Lockmaster1 java exception on the management > servers for the db connection. I think this might be part of the issue, but > I cannot find anything about it. > > This post seems relevant ( > https://blog.widodh.nl/2013/04/cloudstack-zone-x-is-is-not-ready-to-launch-console-proxy-yet/), > but seems only to refer to an issue when the VMs won't launch. > > Has anyone run across this before and found a solution? > > Cloudstack version: 4.15.2 > My systems are running Ubuntu 20.04 > Hypervisor is KVM. > My database (MariaDB) is running in a galera cluster (I have confirmed > that I can log into the database from each server) > My 3 management servers are in a corosync with a HA IP (That IP is where > the database connections are pointed) > > Management logs: > Management 2 (main at this point): > https://gist.github.com/PeterS-gd/10aff04f4204ebebf9066537dea06bac > Management 1 (in cluster): > https://gist.github.com/PeterS-gd/e9e29e030def574313a9168b581d71ed > Management 3 (in cluster): > https://gist.github.com/PeterS-gd/4effc4ddfe7d4a360cc518c121407ee3 > > On the server that has the HA IP, the sql error log shows this: > 2021-11-30 14:31:14 489 [Warning] Aborted connection 489 to db: 'cloud' > user: 'cloud' host: 'gd-cs-control-server' (Got an error reading > communication packets) > 2021-11-30 14:31:14 461 [Warning] Aborted connection 461 to db: 'cloud' > user: 'cloud' host: 'gd-cs-control-server' (Got an error reading > communication packets) > 2021-11-30 14:31:14 521 [Warning] Aborted connection 521 to db: 'cloud' > user: 'cloud' host: 'gd-cs-control-server' (Got an error reading > communication packets) > 2021-11-30 14:31:14 515 [Warning] Aborted connection 515 to db: 'cloud' > user: 'cloud' host: 'gd-cs-control-server' (Got an error reading > communication packets) > 2021-11-30 14:31:14 251 [Warning] Aborted connection 251 to db: > 'cloud_usage' user: 'cloud' host: 'gd-cs-control-server' (Got an error > reading communication packets) > 2021-11-30 14:31:14 490 [Warning] Aborted connection 490 to db: 'cloud' > user: 'cloud' host: 'gd-cs-control-server' (Got an error reading > communication packets) > 2021-11-30 14:31:14 452 [Warning] Aborted connection 452 to db: 'cloud' > user: 'cloud' host: 'gd-cs-control-server' (Got an error reading > communication packets) > 2021-11-30 14:31:14 445 [Warning] Aborted connection 445 to db: 'cloud' > user: 'cloud' host: 'gd-cs-control-server' (Got an error reading > communication packets) > 2021-11-30 14:31:14 514 [Warning] Aborted connection 514 to db: 'cloud' > user: 'cloud' host: 'gd-cs-control-server' (Got an error reading > communication packets) > 2021-11-30 14:31:14 502 [Warning] Aborted connection 502 to db: 'cloud' > user: 'cloud' host: 'gd-cs-control-server' (Got an error reading > communication packets) > > Thanks! > Peter >
