On 08/27/2013 11:22 AM, Sandy Walsh wrote:
On 08/27/2013 05:32 AM, Boris Pavlovic wrote:
Jay,
I should probably share to you about our work around DB.
Migrations should be run only in production and only for production
backends (e.g. psql and mysql)
In tests we should use Schemas created by Models
(BASE.metadata.create_all())
We are not able to use in this approach in moment because we don't have
any mechanism to check that MODELS and SCHEMAS are EQUAL.
And actually MODELS and SCHEMAS are DIFFERENT.
E.g. in Celiometer we have BP that syncs models and migration
https://blueprints.launchpad.net/ceilometer/+spec/ceilometer-db-sync-models-with-migrations
(in other projects we are doing the same)
And also we are working around (oslo) generic tests that checks that
models and migrations are equal:
https://review.openstack.org/#/c/42307/
So in our roadmap (in this case is):
1) Soft switch to alembic (with code that allows to have sqla-migrate
and alembic migration in the same time)
2) Sync Models and Migrations (fix DB schemas also)
3) Add from oslo generic test that checks all this stuff
4) Use BASE.create_all() for Schema creation instead of migrations.
But in OpenStack is not so simple to implement such huge changes, so it
take some time=)
Hmm, so I'm not sure how to proceed here?
Should we be using alembic or the older migrations right now?
Are there any examples of what you're proposing here?
Yeah, basically I am just confused in the case of Ceilometer,
specifically. There are sqlalchemy-migrate migrations [1] and then there
are Alembic migrations [2] all in the same project.
I have no idea how they are supposed to work together, but it seems like
the SQLalchemy migrations are supposed to be run before the Alembic
migrations [3]. However, I don't think the Alembic migrations have even
been tested, frankly. [4]
What I believe is an appropriate course of action would be to do a
single patch (hopefully BEFORE Havana is released) that would:
a) Remove all SQLAlchemy-migrate migrations
b) Create an initial Alembic migration script that would set the
database state to the model state that should have existed at the "end"
of the existing SQLAlchemy migrations (currently, migration 012)
c) Have the initial Alembic migration remove the sqlalchemy-migrate
versioning table and history, if it exists.
d) Remove the existing code in the base storage test case that calls
conn.upgrade() in the setUp() method (!)
e) Have the base storage test case for the sqlalchemy driver simply call
sqlalchemy.MetaData.create_all() instead of running any migrations
f) Have EVERY Alembic migration tested individually with a database
schema that has actual data in it (like is done in Glance with the
sqlalchemy-migrate migrations)
That said, Alembic has no support for common ALTER TABLE operations with
SQLite [5], and so I recommend having the migration tests in f) above
only tested on production database engines (MySQL, PostgreSQL, etc).
Thoughts?
-jay
[1]
https://github.com/openstack/ceilometer/tree/master/ceilometer/storage/sqlalchemy/migrate_repo/versions
[2]
https://github.com/openstack/ceilometer/tree/master/ceilometer/storage/sqlalchemy/alembic/versions
[3]
https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/sqlalchemy/migration.py#L68
[4] https://bugs.launchpad.net/ceilometer/+bug/1217156
[5]
https://bitbucket.org/zzzeek/alembic/issue/21/column-renames-not-supported-on-sqlite
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
