I'm not familiar with RPM install, but I don't think that the workaround would be valid for everyone - QA folks will not have mvn install on their machines, so this issue is a blocker for them. You can try it on your setup though to unblock yourself.
It does have to be fixed in master, either by spring-modularization branch merge, or by applying some temporarily checkin till the merge is done if too many people are blocked by that. -Alena. On 10/8/13 10:55 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com> wrote: >Thanks Alena for the explaination. > >What is the better path to fix this on our setup? Should I wait for a >fix in master or should I manually run the deploydb with mvn? I guess >the second option won't work since I used RPM? > >Francois > >On 10/8/2013, 1:47 PM, Alena Prokharchyk wrote: >> Ok, this is what going on - the DB upgrade procedure is different on >> developer's setup and when deployed using cloudstack-setup-databases >> >> On developers setup: >> >> 1) you deploy the code >> 2) Deploy the DB using 'mvn -P developer -pl developer -Ddeploydb'. As a >> part of this step, the DataBaseUpgradeChecker: >> >> * first deploys the base DB version - 4.0.0 >> * then checks the current version of the code, and performs the db >>upgrade >> if needed. So on master, version table looks like this after the db is >> deployed: >> >> mysql> select * from version; >> +----+---------+---------------------+----------+ >> | id | version | updated | step | >> +----+---------+---------------------+----------+ >> | 1 | 4.0.0 | 2013-10-08 10:34:47 | Complete | >> | 2 | 4.1.0 | 2013-10-08 17:35:22 | Complete | >> | 3 | 4.2.0 | 2013-10-08 17:35:22 | Complete | >> | 4 | 4.3.0 | 2013-10-08 17:35:22 | Complete | >> +----+---------+---------------------+----------+ >> 4 rows in set (0.00 sec) >> >> >> 3) Start management server. >> >> >> When deployed from rpm: >> >> 1) you deploy the code >> 2) run cloudstack-setup-databases. As the result of this step, 4.0.0 >>base >> version of the DB is deployed. Thats why you see only 4.0.0 record in >>the >> DB. >> 3) Start management server. DataBaseUpgradeChecker is being invoked as a >> part of it, and performs the db upgrade to the version of the current >> code. Only after that all the managers get invoked + system caller >>context >> get initialized. >> >> Looks like the load order for step 3) got broken recently, and system >> context gets initialized before the db upgrade is finished. So we either >> need to fix the order, or invoke DataBaseUpgradeChecker as a part of >> cloudstack-setup-databases so at the point when management server starts >> up, the DB already upgraded to the latest version. >> >> -Alena. >> >> >> On 10/8/13 10:25 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com> >> wrote: >> >>> Hmm... I just checked the DB version and it's 4.0??? It should be 4.3.0 >>> no? >>> >>> mysql> select * from version; >>> +----+---------+---------------------+----------+ >>> | id | version | updated | step | >>> +----+---------+---------------------+----------+ >>> | 1 | 4.0.0 | 2013-10-08 10:58:49 | Complete | >>> +----+---------+---------------------+----------+ >>> 1 row in set (0.00 sec) >>> >>> I installed cloudstack-management-4.3.0: >>> [root@eng-testing-cstack_master ~]# rpm -qf >>> /usr/bin/cloudstack-setup-databases >>> cloudstack-management-4.3.0-SNAPSHOT.el6.x86_64 >>> >>> Francois >>> >>> On 10/8/2013, 1:04 PM, Francois Gaudreault wrote: >>>> It's a fresh master RPM install. >>>> >>>> Francois >>>> >>>> On 10/8/2013, 12:40 PM, Alena Prokharchyk wrote: >>>>> It is not a small issue. is_default filed was added to the table as a >>>>> part >>>>> of the 41-42 db upgrade. Looks like the code tries to retrieve system >>>>> user >>>>> before the db upgrade is completed. >>>>> >>>>> DB upgrade is a major part of system integrity check; no queries to >>>>> the DB >>>>> should be made before its completed. Francois, did you start seeing >>>>> this >>>>> problem just recently? >>>>> >>>>> -Alena. >>>>> >>>>> On 10/8/13 8:04 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I compiled Master this morning, and there is a small DB issue. One >>>>>> field >>>>>> is missing in the account table (field default). CS will not start >>>>>> because of that. >>>>>> >>>>>> 2013-10-08 11:01:42,623 FATAL [o.a.c.c.CallContext] (Timer-2:null) >>>>>> Exiting the system because we're unable to register the system call >>>>>> context. >>>>>> com.cloud.utils.exception.CloudRuntimeException: DB Exception on: >>>>>> com.mysql.jdbc.JDBC4PreparedStatement@4c1aa2e9: SELECT account.id, >>>>>> account.account_name, account.type, account.domain_id, >>>>>>account.state, >>>>>> account.removed, account.cleanup_needed, account.network_domain, >>>>>> account.uuid, account.default_zone_id, account.default FROM account >>>>>> WHERE account.id = 1 AND account.removed IS NULL >>>>>> at >>>>>> com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:986) >>>>>> at >>>>>> >>>>>> >>>>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Interce >>>>>>pt >>>>>> orD >>>>>> >>>>>> ispatcher.intercept(ComponentInstantiationPostProcessor.java:125) >>>>>> at >>>>>> com.cloud.utils.db.GenericDaoBase.lockRow(GenericDaoBase.java:963) >>>>>> at >>>>>> >>>>>> >>>>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Interce >>>>>>pt >>>>>> orD >>>>>> >>>>>> ispatcher.intercept(ComponentInstantiationPostProcessor.java:125) >>>>>> at >>>>>> com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:926) >>>>>> at >>>>>> >>>>>> >>>>>>com.cloud.utils.component.ComponentInstantiationPostProcessor$Interce >>>>>>pt >>>>>> orD >>>>>> >>>>>> ispatcher.intercept(ComponentInstantiationPostProcessor.java:125) >>>>>> at >>>>>> com.cloud.dao.EntityManagerImpl.findById(EntityManagerImpl.java:45) >>>>>> at >>>>>> >>>>>> >>>>>>org.apache.cloudstack.context.CallContext.register(CallContext.java:1 >>>>>>66 >>>>>> ) >>>>>> >>>>>> at >>>>>> >>>>>> >>>>>>org.apache.cloudstack.context.CallContext.registerSystemCallContextOn >>>>>>ce >>>>>> Onl >>>>>> >>>>>> y(CallContext.java:141) >>>>>> at >>>>>> >>>>>> >>>>>>org.apache.cloudstack.context.CallContextListener.onEnterContext(Call >>>>>>Co >>>>>> nte >>>>>> >>>>>> xtListener.java:36) >>>>>> at >>>>>> >>>>>> >>>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.call >>>>>>Wi >>>>>> thC >>>>>> >>>>>> ontext(DefaultManagedContext.java:83) >>>>>> at >>>>>> >>>>>> >>>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runW >>>>>>it >>>>>> hCo >>>>>> >>>>>> ntext(DefaultManagedContext.java:53) >>>>>> at >>>>>> >>>>>> >>>>>>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Mana >>>>>>ge >>>>>> dCo >>>>>> >>>>>> ntextRunnable.java:46) >>>>>> at >>>>>> >>>>>> >>>>>>org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(Man >>>>>>ag >>>>>> edC >>>>>> >>>>>> ontextTimerTask.java:27) >>>>>> at java.util.TimerThread.mainLoop(Timer.java:534) >>>>>> at java.util.TimerThread.run(Timer.java:484) >>>>>> Caused by: >>>>>>com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: >>>>>> Unknown column 'account.default' in 'field list' >>>>>> at >>>>>>sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>>>> Method) >>>>>> at >>>>>> >>>>>> >>>>>>sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct >>>>>>or >>>>>> Acc >>>>>> >>>>>> essorImpl.java:57) >>>>>> at >>>>>> >>>>>> >>>>>>sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC >>>>>>on >>>>>> str >>>>>> >>>>>> uctorAccessorImpl.java:45) >>>>>> at >>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java:532) >>>>>> at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) >>>>>> at com.mysql.jdbc.Util.getInstance(Util.java:386) >>>>>> at >>>>>>com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053) >>>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074) >>>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006) >>>>>> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468) >>>>>> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2629) >>>>>> at >>>>>> com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2719) >>>>>> at >>>>>> >>>>>> >>>>>>com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.ja >>>>>>va >>>>>> :21 >>>>>> >>>>>> 55) >>>>>> at >>>>>> >>>>>> >>>>>>com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java: >>>>>>23 >>>>>> 18) >>>>>> >>>>>> at >>>>>> >>>>>> >>>>>>org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Dele >>>>>>ga >>>>>> tin >>>>>> >>>>>> gPreparedStatement.java:96) >>>>>> at >>>>>> >>>>>> >>>>>>org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Dele >>>>>>ga >>>>>> tin >>>>>> >>>>>> gPreparedStatement.java:96) >>>>>> at >>>>>> com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:983) >>>>>> ... 27 more >>>>>> >>>>>> >>>>>> -- >>>>>> Francois Gaudreault >>>>>> Architecte de Solution Cloud | Cloud Solutions Architect >>>>>> fgaudrea...@cloudops.com >>>>>> 514-629-6775 >>>>>> - - - >>>>>> CloudOps >>>>>> 420 rue Guy >>>>>> Montréal QC H3J 1S6 >>>>>> www.cloudops.com >>>>>> @CloudOps_ >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >>> -- >>> Francois Gaudreault >>> Architecte de Solution Cloud | Cloud Solutions Architect >>> fgaudrea...@cloudops.com >>> 514-629-6775 >>> - - - >>> CloudOps >>> 420 rue Guy >>> Montréal QC H3J 1S6 >>> www.cloudops.com >>> @CloudOps_ >>> >>> >> >> >> > > >-- >Francois Gaudreault >Architecte de Solution Cloud | Cloud Solutions Architect >fgaudrea...@cloudops.com >514-629-6775 >- - - >CloudOps >420 rue Guy >Montréal QC H3J 1S6 >www.cloudops.com >@CloudOps_ > >
