On 08/27/2013 05:32 AM, Boris Pavlovic wrote: > Jay, > > I should probably share to you about our work around DB. > > Migrations should be run only in production and only for production > backends (e.g. psql and mysql) > In tests we should use Schemas created by Models > (BASE.metadata.create_all()) > > We are not able to use in this approach in moment because we don't have > any mechanism to check that MODELS and SCHEMAS are EQUAL. > And actually MODELS and SCHEMAS are DIFFERENT. > > E.g. in Celiometer we have BP that syncs models and migration > https://blueprints.launchpad.net/ceilometer/+spec/ceilometer-db-sync-models-with-migrations > (in other projects we are doing the same) > > And also we are working around (oslo) generic tests that checks that > models and migrations are equal: > https://review.openstack.org/#/c/42307/ > > > So in our roadmap (in this case is): > 1) Soft switch to alembic (with code that allows to have sqla-migrate > and alembic migration in the same time) > 2) Sync Models and Migrations (fix DB schemas also) > 3) Add from oslo generic test that checks all this stuff > 4) Use BASE.create_all() for Schema creation instead of migrations. > > > But in OpenStack is not so simple to implement such huge changes, so it > take some time=)
Hmm, so I'm not sure how to proceed here? Should we be using alembic or the older migrations right now? Are there any examples of what you're proposing here? > > > Best regards, > Boris Pavlovic > --- > Mirantis Inc. > > > > > > > > > > > On Tue, Aug 27, 2013 at 12:02 AM, Jay Pipes <[email protected] > <mailto:[email protected]>> wrote: > > On 08/26/2013 03:40 PM, Herndon, John Luke (HPCS - Ft. Collins) wrote: > > Jay - > > It looks there is an error in the migration script that causes > it to abort: > > AttributeError: 'ForeignKeyConstraint' object has no attribute > 'drop' > > My guess is the migration runs on the first test, creates event > types > table fine, but exits with the above error, so migration is not > "complete". Thus every subsequent test tries to migrate the db, and > notices that event types already exists. > > > I'd corrected that particular mistake and pushed an updated > migration script. > > Best, > -jay > > > > -john > > On 8/26/13 1:15 PM, "Jay Pipes" <[email protected] > <mailto:[email protected]>> wrote: > > I just noticed that every single test case for SQL-driver > storage is > executing every single migration upgrade before every single > test case > run: > > > https://github.com/openstack/__ceilometer/blob/master/__ceilometer/tests/db.py > > <https://github.com/openstack/ceilometer/blob/master/ceilometer/tests/db.py> > #L46 > > > https://github.com/openstack/__ceilometer/blob/master/__ceilometer/storage/imp > > <https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/imp> > l_sqlalchemy.py#L153 > > instead of simply creating a new database schema from the > models in the > current source code base using a call to > sqlalchemy.MetaData.create___all(). > > This results in re-running migrations over and over again, > instead of > having dedicated migration tests that would test each migration > individually, as is the case in projects like Glance... > > Is this intentional? > > Best, > -jay > > On 08/26/2013 02:59 PM, Sandy Walsh wrote: > > I'm getting the same problem with a different migration > (mine is > complaining that a column already exists) > > http://paste.openstack.org/__show/44512/ > <http://paste.openstack.org/show/44512/> > > I've compared it to the other migrations and it seems fine. > > -S > > On 08/26/2013 02:34 PM, Jay Pipes wrote: > > Hey all, > > I'm trying to figure out what is going wrong with my > code for this > patch: > > https://review.openstack.org/__41316 > <https://review.openstack.org/41316> > > I had previously added a sqlalchemy-migrate > migration script to add an > event_type table, and had that working, but then was > asked to instead > use Alembic for migrations. So, I removed the > sqlalchemy-migrate > migration file and added an Alembic migration [1]. > > Unfortunately, I am getting the following error when > running tests: > > OperationalError: (OperationalError) table > event_type already exists > u'\nCREATE TABLE event_type (\n\tid INTEGER NOT > NULL, \n\t"desc" > VARCHAR(255), \n\tPRIMARY KEY (id), \n\tUNIQUE > ("desc")\n)\n\n' () > > The migration adds the event_type table. I've seen > this error occur > before when using SQLite due to SQLite's ALTER TABLE > statement not > allowing the rename of a column. In the > sqlalchemy-migrate migration, I > had a specialized SQLite migration upgrade [2] and > downgrade [3] > script, > but I'm not sure how I am supposed to handle this in > Alembic. Could > someone help me out? > > Thanks, > -jay > > [1] > > > https://review.openstack.org/#__/c/41316/16/ceilometer/__storage/sqlalchemy/ > > <https://review.openstack.org/#/c/41316/16/ceilometer/storage/sqlalchemy/> > alembic/versions/49036daaaafd___add_event_types.py > > [2] > > > https://review.openstack.org/#__/c/41316/14/ceilometer/__storage/sqlalchemy/ > > <https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/> > migrate_repo/versions/013___sqlite_upgrade.sql > > [3] > > > https://review.openstack.org/#__/c/41316/14/ceilometer/__storage/sqlalchemy/ > > <https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/> > migrate_repo/versions/013___sqlite_downgrade.sql > > > _________________________________________________ > OpenStack-dev mailing list > [email protected].__org > <mailto:[email protected]> > > http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev > > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > _________________________________________________ > OpenStack-dev mailing list > [email protected].__org > <mailto:[email protected]> > > http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev > > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > _________________________________________________ > OpenStack-dev mailing list > [email protected].__org > <mailto:[email protected]> > > http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev > > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > _________________________________________________ > OpenStack-dev mailing list > [email protected].__org > <mailto:[email protected]> > http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
