I think some things are broken in the oslo-incubator db migration code. Ironic moved to this when Juno opened and things seemed fine, until recently when Lucas tried to add a DB migration and noticed that it didn't run... So I looked into it a bit today. Below are my findings.
Firstly, I filed this bug and proposed a fix, because I think that tests that don't run any code should not report that they passed -- they should report that they were skipped. https://bugs.launchpad.net/oslo/+bug/1327397 "No notice given when db migrations are not run due to missing engine" Then, I edited the test_migrations.conf file appropriately for my local mysql service, ran the tests again, and verified that migration tests ran -- and they passed. Great! Now, a little background... Ironic's TestMigrations class inherits from oslo's BaseMigrationTestCase, then "opportunistically" checks each back-end, if it's available. This opportunistic checking was inherited from Nova so that tests could pass on developer workstations where not all backends are present (eg, I have mysql installed, but not postgres), and still transparently run on all backends in the gate. I couldn't find such opportunistic testing in the oslo db migration test code, unfortunately - but maybe it's well hidden. Anyhow. When I stopped the local mysql service (leaving the configuration unchanged), I expected the tests to be skipped, but instead I got two surprise failures: - test_mysql_opportunistically() failed because setUp() raises an exception before the test code could call calling _have_mysql() - test_mysql_connect_fail() actually failed! Again, because setUp() raises an exception before running the test itself Unfortunately, there's one more problem... when I run the tests in parallel, they fail randomly because sometimes two test threads run different migration tests, and the setUp() for one thread (remember, it calls _reset_databases) blows up the other test. Out of 10 runs, it failed three times, each with different errors: NoSuchTableError: `chassis` ERROR 1007 (HY000) at line 1: Can't create database 'test_migrations'; database exists ProgrammingError: (ProgrammingError) (1146, "Table 'test_migrations.alembic_version' doesn't exist") As far as I can tell, this is all coming from: https://github.com/openstack/oslo-incubator/blob/master/openstack/common/db/sqlalchemy/test_migrations.py#L86;L111 So, Ironic devs -- if you see a DB migration proposed, pay extra attention to it. We aren't running migration tests in our check or gate queues right now, and we shouldn't enable them until this fixed. Regards, Devananda
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
