[
https://issues.apache.org/jira/browse/CLOUDSTACK-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohit Yadav updated CLOUDSTACK-1019:
------------------------------------
Description:
A new tool/class called DatabaseCreator is introduced in javelin, the idea is
that DatabaseCreator can be used by mgmt server, cloud-setup-database script,
maven (developer/pom.xml) and possibly by plugins in future to:
1. Initialize database using db.properties file
2. Run sql scripts
3. Run database upgrades
Presently, this has been fixed in javelin, in cloud-server and is used by maven
to deploydb, the task is to fix that same for cloud-setup-databases.
Current help doc, usage and options:
DatabaseCreator creates the database schema by removing the
previous schema, creating the schema, and running
through the database updaters.
Usage: DatabaseCreator [options] [db.properties file] [schema.sql files]
[database upgrade class]
Options:
--database=a,b comma separate databases to initialize, use the db name in
db.properties defined as db.xyz.host, xyz should be passed
--rootpassword=password, by default it will try with an empty password
--dry or -d, this would not run any process, just does a dry run
--verbose or -v to print running sql commands, by default it won't print them
--help or -h for help
The issue now is to make cloudstack-setup-databases use DatabaseCreator and
remove db upgrading logic from mgmt server and move it to DatabaseCreator.
Let's talk about the present database deployment and the issues:
- We've two different routes to deploydb, for developers it's
DatabaseCreator now (used to be a maven plugin), for sysadmins it's
cloudstack-setup-database
- Upgrades are done by mgmt server, this has issues when there is a
clustered mgmt server, i.e. multiple mgmt servers sharing the same
database, we do upgrade by:
a. Turn down one mgmt server, upgrade it, start it. This mgmt
server checks in db the cloudstack version, upgrades the db
b. Sysadmins meanwhile upgrades all the other mgmt server
c. When all mgmt servers are upgraded, they ask the cluster
manager to run the cleanup. The whole process is automated, an error
can introduce whole sets of issues.
How will DatabaseCreator help:
- Would give a single tool used by both developer and sysadmins. Agony
of the sysadmins would be much better understood by developers.
Sysadmins would still use cloudstack-setup-database which would be
wrapper around DatabaseCreator
- Upgrade would be driven by sysadmin, giving them explicit power to
backup database, deploy etc. For example in case of multiple
(clustered) mgmt server, the upgrade would be like:
a. Upgrade database using DatabaseCreator, this step do a backup
and would just upgrade the db such that it's backward compatible to
the old database and old mgmt servers can still run on it. When this
succeeds, system can proceed. Transaction update would provide a way
to rollback.
b. Sysadmins would stop, upgrade and start mgmt server one by one.
c. Sysadmins would run the final cleanup operation, this would do a
final round of cleaning up. If everything goes well, they just did a
rolling CloudStack upgrade with little to no downtime (no downtime as
at least one mgmt server was running :)
TODOs for 4.1:
- We stick to major db version as 4.0, create-schema is not touched
unless it's a major version. So, next change is to be done only for
5.0.
- We move out any 4.1 changes to 4041 upgrade sql file and have a
upgrade path (a upgrading logic/class).
- Fix cloudstack-setup-databases, fix packaging.
For 4.2/future:
- Refactor out code that does upgrading in mgmt server to a separate
artifact which would also have DatabaseCreator.
- Implement tool so it supports the operations as discussed above
- Replace the current pythonic implementation with DatabaseCreator,
have same routes for both developers and sysadmins
NOTE TO DEVELOPERS:
- WE NEVER CHANGE create-schema.sql
- WE TEST AND IMPLEMENT UPGRADE PATHS from version a to version b
- WE CLEANUP PROPERLY via our upgrade sql files and upgrade classes
was:
A new tool/class called DatabaseCreator is introduced in javelin, the idea is
that DatabaseCreator can be used by mgmt server, cloud-setup-database script,
maven (developer/pom.xml) and possibly by plugins in future to:
1. Initialize database using db.properties file
2. Run sql scripts
3. Run database upgrades
Presently, this has been fixed in javelin, in cloud-server and is used by maven
to deploydb, the task is to fix that same for cloud-setup-databases.
Current help doc, usage and options:
DatabaseCreator creates the database schema by removing the
previous schema, creating the schema, and running
through the database updaters.
Usage: DatabaseCreator [options] [db.properties file] [schema.sql files]
[database upgrade class]
Options:
--database=a,b comma separate databases to initialize, use the db name in
db.properties defined as db.xyz.host, xyz should be passed
--rootpassword=password, by default it will try with an empty password
--dry or -d, this would not run any process, just does a dry run
--verbose or -v to print running sql commands, by default it won't print them
--help or -h for help
The issue now is to make cloudstack-setup-databases use DatabaseCreator and
remove db upgrading logic from mgmt server and move it to DatabaseCreator.
> Fix cloud-setup-database to use DatabaseCreator and make DatabaseCreator do
> the upgrades
> ----------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-1019
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1019
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Environment: branch: javelin (or master if merge happens)
> Reporter: Rohit Yadav
> Assignee: Rohit Yadav
> Fix For: 4.1.0
>
>
> A new tool/class called DatabaseCreator is introduced in javelin, the idea is
> that DatabaseCreator can be used by mgmt server, cloud-setup-database script,
> maven (developer/pom.xml) and possibly by plugins in future to:
> 1. Initialize database using db.properties file
> 2. Run sql scripts
> 3. Run database upgrades
> Presently, this has been fixed in javelin, in cloud-server and is used by
> maven to deploydb, the task is to fix that same for cloud-setup-databases.
> Current help doc, usage and options:
> DatabaseCreator creates the database schema by removing the
> previous schema, creating the schema, and running
> through the database updaters.
> Usage: DatabaseCreator [options] [db.properties file] [schema.sql files]
> [database upgrade class]
> Options:
> --database=a,b comma separate databases to initialize, use the db name in
> db.properties defined as db.xyz.host, xyz should be passed
> --rootpassword=password, by default it will try with an empty password
> --dry or -d, this would not run any process, just does a dry run
> --verbose or -v to print running sql commands, by default it won't print
> them
> --help or -h for help
> The issue now is to make cloudstack-setup-databases use DatabaseCreator and
> remove db upgrading logic from mgmt server and move it to DatabaseCreator.
> Let's talk about the present database deployment and the issues:
> - We've two different routes to deploydb, for developers it's
> DatabaseCreator now (used to be a maven plugin), for sysadmins it's
> cloudstack-setup-database
> - Upgrades are done by mgmt server, this has issues when there is a
> clustered mgmt server, i.e. multiple mgmt servers sharing the same
> database, we do upgrade by:
> a. Turn down one mgmt server, upgrade it, start it. This mgmt
> server checks in db the cloudstack version, upgrades the db
> b. Sysadmins meanwhile upgrades all the other mgmt server
> c. When all mgmt servers are upgraded, they ask the cluster
> manager to run the cleanup. The whole process is automated, an error
> can introduce whole sets of issues.
> How will DatabaseCreator help:
> - Would give a single tool used by both developer and sysadmins. Agony
> of the sysadmins would be much better understood by developers.
> Sysadmins would still use cloudstack-setup-database which would be
> wrapper around DatabaseCreator
> - Upgrade would be driven by sysadmin, giving them explicit power to
> backup database, deploy etc. For example in case of multiple
> (clustered) mgmt server, the upgrade would be like:
> a. Upgrade database using DatabaseCreator, this step do a backup
> and would just upgrade the db such that it's backward compatible to
> the old database and old mgmt servers can still run on it. When this
> succeeds, system can proceed. Transaction update would provide a way
> to rollback.
> b. Sysadmins would stop, upgrade and start mgmt server one by one.
> c. Sysadmins would run the final cleanup operation, this would do a
> final round of cleaning up. If everything goes well, they just did a
> rolling CloudStack upgrade with little to no downtime (no downtime as
> at least one mgmt server was running :)
> TODOs for 4.1:
> - We stick to major db version as 4.0, create-schema is not touched
> unless it's a major version. So, next change is to be done only for
> 5.0.
> - We move out any 4.1 changes to 4041 upgrade sql file and have a
> upgrade path (a upgrading logic/class).
> - Fix cloudstack-setup-databases, fix packaging.
> For 4.2/future:
> - Refactor out code that does upgrading in mgmt server to a separate
> artifact which would also have DatabaseCreator.
> - Implement tool so it supports the operations as discussed above
> - Replace the current pythonic implementation with DatabaseCreator,
> have same routes for both developers and sysadmins
> NOTE TO DEVELOPERS:
> - WE NEVER CHANGE create-schema.sql
> - WE TEST AND IMPLEMENT UPGRADE PATHS from version a to version b
> - WE CLEANUP PROPERLY via our upgrade sql files and upgrade classes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
