On Wed, Mar 26, 2014 at 6:24 PM, Doug Hellmann <[email protected]>wrote:
> > > > On Wed, Mar 26, 2014 at 4:30 PM, James E. Blair <[email protected]>wrote: > >> Hi, >> >> I recently started looking into running the unit tests with MySQL. I >> pushed the start of a change here to do that: >> >> https://review.openstack.org/#/c/82872/ >> >> In particular, that change implements something we've wanted to do for a >> long time elsewhere in OpenStack -- create a dedicated MySQL database >> for each test so that tests can safely run in parallel. However, that >> has uncovered the following issues: >> >> * The tests do not use alembic, except for the migration test. This >> could lead to variations in what the unit tests are testing against >> as compared to production. >> >> * The model does not specify an engine for MySQL (this is a difference >> between the model and the alembic migrations). So when the unit tests >> are run on MySQL, they may end up on MyISAM (they do in the gate). >> >> * The two errors that show up in the test results for my patch relate >> to how timestamps are stored differently between MySQL and SQLite. >> >> * The errors that you can not see in the test report from Jenkins but >> would if you ran in an environment with InnoDB as the default engine >> for MySQL are several instances of foreign key constraint errors. >> >> I think it is desirable to run the unit tests with MySQL largely because >> I think we should be testing it in the way we intend to use it in >> production. Note especially the last point above -- our unit tests have >> significant problems with MySQL. That suggests one of two things to me: >> either the tests aren't testing real behavior, or our system is actually >> broken in production. Neither one seems like a good situation. >> >> My inclination would be to use MySQL for unit tests, use alembic to >> create the schema, and fix the errors currently uncovered by this. >> >> It's not going to be a trivial change, but I think it's worth doing. >> Before embarking on it, I'd like to see if we can get consensus. We can >> talk about it here or possibly at the meeting tomorrow. >> >> Ruslan has commented in that review with the following: >> >> > But, I think that we should consider work done in major OpenStack >> > projects. They don't use real MySQL to run unit-tests. Here is what >> > the do (or plan to do): >> > >> > * Run unit-tests on SQLite. DB schema is created from SQLA metadata >> > * Run DB migration tests on MySQL and Postgres >> > * Run tests to compare SQLA metadata and DB migrations to make sure >> > that they're in sync. Here is WIP patch >> > https://review.openstack.org/#/c/ >> > >> > I'm not opposed to this direction, I just want to make sure we're >> > aligned with the common practices in OpenStack >> >> (That last link doesn't seem to be correct.) >> >> That's a good point, but I personally find testing as close to >> production as possible desirable -- MySQL and SQLite are not compatible >> so we shouldn't pretend that they are. If we prove this is a viable >> strategy, then I think we should recommend OpenStack projects adopt it >> as well. >> > > Some of the folks from Mirantis who are working on oslo.db are working on > making it so we can use the models to create a database in our unit tests, > and then compare the resulting database with what the migrations do. That > *should* speed up tests, since we don't have to apply a set of migrations, > while still ensuring that we are testing using the same schema. It also has > the benefit of ensuring that our models represent what the migrations > create. I've copied Victor and Roman on this so they can add more detail. > And, that's the same work Ruslan was linking to based on his follow-up message. Doug > > Doug > > > >> >> -Jim >> >> _______________________________________________ >> OpenStack-Infra mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> > >
_______________________________________________ OpenStack-Infra mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
