Thanks Alena for the explaination.
What is the better path to fix this on our setup? Should I wait for a
fix in master or should I manually run the deploydb with mvn? I guess
the second option won't work since I used RPM?
Francois
On 10/8/2013, 1:47 PM, Alena Prokharchyk wrote:
Ok, this is what going on - the DB upgrade procedure is different on
developer's setup and when deployed using cloudstack-setup-databases
On developers setup:
1) you deploy the code
2) Deploy the DB using 'mvn -P developer -pl developer -Ddeploydb'. As a
part of this step, the DataBaseUpgradeChecker:
* first deploys the base DB version - 4.0.0
* then checks the current version of the code, and performs the db upgrade
if needed. So on master, version table looks like this after the db is
deployed:
mysql> select * from version;
+----+---------+---------------------+----------+
| id | version | updated | step |
+----+---------+---------------------+----------+
| 1 | 4.0.0 | 2013-10-08 10:34:47 | Complete |
| 2 | 4.1.0 | 2013-10-08 17:35:22 | Complete |
| 3 | 4.2.0 | 2013-10-08 17:35:22 | Complete |
| 4 | 4.3.0 | 2013-10-08 17:35:22 | Complete |
+----+---------+---------------------+----------+
4 rows in set (0.00 sec)
3) Start management server.
When deployed from rpm:
1) you deploy the code
2) run cloudstack-setup-databases. As the result of this step, 4.0.0 base
version of the DB is deployed. Thats why you see only 4.0.0 record in the
DB.
3) Start management server. DataBaseUpgradeChecker is being invoked as a
part of it, and performs the db upgrade to the version of the current
code. Only after that all the managers get invoked + system caller context
get initialized.
Looks like the load order for step 3) got broken recently, and system
context gets initialized before the db upgrade is finished. So we either
need to fix the order, or invoke DataBaseUpgradeChecker as a part of
cloudstack-setup-databases so at the point when management server starts
up, the DB already upgraded to the latest version.
-Alena.
On 10/8/13 10:25 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com>
wrote:
Hmm... I just checked the DB version and it's 4.0??? It should be 4.3.0
no?
mysql> select * from version;
+----+---------+---------------------+----------+
| id | version | updated | step |
+----+---------+---------------------+----------+
| 1 | 4.0.0 | 2013-10-08 10:58:49 | Complete |
+----+---------+---------------------+----------+
1 row in set (0.00 sec)
I installed cloudstack-management-4.3.0:
[root@eng-testing-cstack_master ~]# rpm -qf
/usr/bin/cloudstack-setup-databases
cloudstack-management-4.3.0-SNAPSHOT.el6.x86_64
Francois
On 10/8/2013, 1:04 PM, Francois Gaudreault wrote:
It's a fresh master RPM install.
Francois
On 10/8/2013, 12:40 PM, Alena Prokharchyk wrote:
It is not a small issue. is_default filed was added to the table as a
part
of the 41-42 db upgrade. Looks like the code tries to retrieve system
user
before the db upgrade is completed.
DB upgrade is a major part of system integrity check; no queries to
the DB
should be made before its completed. Francois, did you start seeing
this
problem just recently?
-Alena.
On 10/8/13 8:04 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com>
wrote:
Hi,
I compiled Master this morning, and there is a small DB issue. One
field
is missing in the account table (field default). CS will not start
because of that.
2013-10-08 11:01:42,623 FATAL [o.a.c.c.CallContext] (Timer-2:null)
Exiting the system because we're unable to register the system call
context.
com.cloud.utils.exception.CloudRuntimeException: DB Exception on:
com.mysql.jdbc.JDBC4PreparedStatement@4c1aa2e9: SELECT account.id,
account.account_name, account.type, account.domain_id, account.state,
account.removed, account.cleanup_needed, account.network_domain,
account.uuid, account.default_zone_id, account.default FROM account
WHERE account.id = 1 AND account.removed IS NULL
at
com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:986)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercept
orD
ispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
com.cloud.utils.db.GenericDaoBase.lockRow(GenericDaoBase.java:963)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercept
orD
ispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:926)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercept
orD
ispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
com.cloud.dao.EntityManagerImpl.findById(EntityManagerImpl.java:45)
at
org.apache.cloudstack.context.CallContext.register(CallContext.java:166
)
at
org.apache.cloudstack.context.CallContext.registerSystemCallContextOnce
Onl
y(CallContext.java:141)
at
org.apache.cloudstack.context.CallContextListener.onEnterContext(CallCo
nte
xtListener.java:36)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWi
thC
ontext(DefaultManagedContext.java:83)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWit
hCo
ntext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manage
dCo
ntextRunnable.java:46)
at
org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(Manag
edC
ontextTimerTask.java:27)
at java.util.TimerThread.mainLoop(Timer.java:534)
at java.util.TimerThread.run(Timer.java:484)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException:
Unknown column 'account.default' in 'field list'
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructor
Acc
essorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCon
str
uctorAccessorImpl.java:45)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2629)
at
com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2719)
at
com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java
:21
55)
at
com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:23
18)
at
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delega
tin
gPreparedStatement.java:96)
at
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delega
tin
gPreparedStatement.java:96)
at
com.cloud.utils.db.GenericDaoBase.findById(GenericDaoBase.java:983)
... 27 more
--
Francois Gaudreault
Architecte de Solution Cloud | Cloud Solutions Architect
fgaudrea...@cloudops.com
514-629-6775
- - -
CloudOps
420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
--
Francois Gaudreault
Architecte de Solution Cloud | Cloud Solutions Architect
fgaudrea...@cloudops.com
514-629-6775
- - -
CloudOps
420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
--
Francois Gaudreault
Architecte de Solution Cloud | Cloud Solutions Architect
fgaudrea...@cloudops.com
514-629-6775
- - -
CloudOps
420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
