Just throwing in a pitch for my spring-modularization branch. The DB initialization stuff should all be fixed in the spring-modularization branch. If it isn't we can better address it there anyhow. The current initialization of spring in master is pure madness. When it works, its because it just happens to work at that time. Adding new beans and dependencies can easily break the initialization order as seen over and over again with the DB upgrades.
Darren On Tue, Oct 8, 2013 at 10:47 AM, Alena Prokharchyk <alena.prokharc...@citrix.com> wrote: > Ok, this is what going on - the DB upgrade procedure is different on > developer's setup and when deployed using cloudstack-setup-databases > > On developers setup: > > 1) you deploy the code > 2) Deploy the DB using 'mvn -P developer -pl developer -Ddeploydb'. As a > part of this step, the DataBaseUpgradeChecker: > > * first deploys the base DB version - 4.0.0 > * then checks the current version of the code, and performs the db upgrade > if needed. So on master, version table looks like this after the db is > deployed: > > mysql> select * from version; > +----+---------+---------------------+----------+ > | id | version | updated | step | > +----+---------+---------------------+----------+ > | 1 | 4.0.0 | 2013-10-08 10:34:47 | Complete | > | 2 | 4.1.0 | 2013-10-08 17:35:22 | Complete | > | 3 | 4.2.0 | 2013-10-08 17:35:22 | Complete | > | 4 | 4.3.0 | 2013-10-08 17:35:22 | Complete | > +----+---------+---------------------+----------+ > 4 rows in set (0.00 sec) > > > 3) Start management server. > > > When deployed from rpm: > > 1) you deploy the code > 2) run cloudstack-setup-databases. As the result of this step, 4.0.0 base > version of the DB is deployed. Thats why you see only 4.0.0 record in the > DB. > 3) Start management server. DataBaseUpgradeChecker is being invoked as a > part of it, and performs the db upgrade to the version of the current > code. Only after that all the managers get invoked + system caller context > get initialized. > > Looks like the load order for step 3) got broken recently, and system > context gets initialized before the db upgrade is finished. So we either > need to fix the order, or invoke DataBaseUpgradeChecker as a part of > cloudstack-setup-databases so at the point when management server starts > up, the DB already upgraded to the latest version. > > -Alena. > > > On 10/8/13 10:25 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com> > wrote: > >>Hmm... I just checked the DB version and it's 4.0??? It should be 4.3.0 >>no? >> >>mysql> select * from version; >>+----+---------+---------------------+----------+ >>| id | version | updated | step | >>+----+---------+---------------------+----------+ >>| 1 | 4.0.0 | 2013-10-08 10:58:49 | Complete | >>+----+---------+---------------------+----------+ >>1 row in set (0.00 sec) >> >>I installed cloudstack-management-4.3.0: >>[root@eng-testing-cstack_master ~]# rpm -qf >>/usr/bin/cloudstack-setup-databases >>cloudstack-management-4.3.0-SNAPSHOT.el6.x86_64 >> >>Francois >> >>On 10/8/2013, 1:04 PM, Francois Gaudreault wrote: >>> It's a fresh master RPM install. >>> >>> Francois >>> >>> On 10/8/2013, 12:40 PM, Alena Prokharchyk wrote: >>>> It is not a small issue. is_default filed was added to the table as a >>>> part >>>> of the 41-42 db upgrade. Looks like the code tries to retrieve system >>>> user >>>> before the db upgrade is completed. >>>> >>>> DB upgrade is a major part of system integrity check; no queries to >>>> the DB >>>> should be made before its completed. Francois, did you start seeing >>>>this >>>> problem just recently? >>>> >>>> -Alena. >>>> >>>> On 10/8/13 8:04 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I compiled Master this morning, and there is a small DB issue. One >>>>> field >>>>> is missing in the account table (field default). CS will not start >>>>> because of that. >>>>> >>>>> 2013-10-08 11:01:42,623 FATAL [o.a.c.c.CallContext] (Timer-2:null) >>>>> Exiting the system because we're unable to register the system call >>>>> context. >>>>> com.cloud.utils.exception.CloudRuntimeException: DB Exception on: >>>>> com.mysql.jdbc.JDBC4PreparedStatement@4c1aa2e9: SELECT account.id, >>>>> account.account_name, account.type, account.domain_id, account.state, >>>>> account.removed, account.cleanup_needed, account.network_domain, >>>>> account.uuid, account.default_zone_id, account.default FROM account >>>>> WHERE account.id = 1 AND account.removed IS NULL >>>>> at >>>>> com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:986) >>>>> at >>>>> >>>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercept >>>>>orD >>>>> >>>>> ispatcher.intercept(ComponentInstantiationPostProcessor.java:125) >>>>> at >>>>> com.cloud.utils.db.GenericDaoBase.lockRow(GenericDaoBase.java:963) >>>>> at >>>>> >>>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercept >>>>>orD >>>>> >>>>> ispatcher.intercept(ComponentInstantiationPostProcessor.java:125) >>>>> at >>>>> com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:926) >>>>> at >>>>> >>>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercept >>>>>orD >>>>> >>>>> ispatcher.intercept(ComponentInstantiationPostProcessor.java:125) >>>>> at >>>>> com.cloud.dao.EntityManagerImpl.findById(EntityManagerImpl.java:45) >>>>> at >>>>> >>>>>org.apache.cloudstack.context.CallContext.register(CallContext.java:166 >>>>>) >>>>> >>>>> at >>>>> >>>>>org.apache.cloudstack.context.CallContext.registerSystemCallContextOnce >>>>>Onl >>>>> >>>>> y(CallContext.java:141) >>>>> at >>>>> >>>>>org.apache.cloudstack.context.CallContextListener.onEnterContext(CallCo >>>>>nte >>>>> >>>>> xtListener.java:36) >>>>> at >>>>> >>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWi >>>>>thC >>>>> >>>>> ontext(DefaultManagedContext.java:83) >>>>> at >>>>> >>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWit >>>>>hCo >>>>> >>>>> ntext(DefaultManagedContext.java:53) >>>>> at >>>>> >>>>>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manage >>>>>dCo >>>>> >>>>> ntextRunnable.java:46) >>>>> at >>>>> >>>>>org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(Manag >>>>>edC >>>>> >>>>> ontextTimerTask.java:27) >>>>> at java.util.TimerThread.mainLoop(Timer.java:534) >>>>> at java.util.TimerThread.run(Timer.java:484) >>>>> Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: >>>>> Unknown column 'account.default' in 'field list' >>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>>> Method) >>>>> at >>>>> >>>>>sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructor >>>>>Acc >>>>> >>>>> essorImpl.java:57) >>>>> at >>>>> >>>>>sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCon >>>>>str >>>>> >>>>> uctorAccessorImpl.java:45) >>>>> at >>>>>java.lang.reflect.Constructor.newInstance(Constructor.java:532) >>>>> at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) >>>>> at com.mysql.jdbc.Util.getInstance(Util.java:386) >>>>> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053) >>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074) >>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006) >>>>> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468) >>>>> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2629) >>>>> at >>>>>com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2719) >>>>> at >>>>> >>>>>com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java >>>>>:21 >>>>> >>>>> 55) >>>>> at >>>>> >>>>>com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:23 >>>>>18) >>>>> >>>>> at >>>>> >>>>>org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delega >>>>>tin >>>>> >>>>> gPreparedStatement.java:96) >>>>> at >>>>> >>>>>org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delega >>>>>tin >>>>> >>>>> gPreparedStatement.java:96) >>>>> at >>>>> com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:983) >>>>> ... 27 more >>>>> >>>>> >>>>> -- >>>>> Francois Gaudreault >>>>> Architecte de Solution Cloud | Cloud Solutions Architect >>>>> fgaudrea...@cloudops.com >>>>> 514-629-6775 >>>>> - - - >>>>> CloudOps >>>>> 420 rue Guy >>>>> Montréal QC H3J 1S6 >>>>> www.cloudops.com >>>>> @CloudOps_ >>>>> >>>>> >>>> >>>> >>>> >>> >>> >> >> >>-- >>Francois Gaudreault >>Architecte de Solution Cloud | Cloud Solutions Architect >>fgaudrea...@cloudops.com >>514-629-6775 >>- - - >>CloudOps >>420 rue Guy >>Montréal QC H3J 1S6 >>www.cloudops.com >>@CloudOps_ >> >> > >
