This one was tricky. :-) This failure can be produced consistently by running the following two tests: neutron.tests.unit.brocade.test_brocade_plugin.TestBrocadePortsV2.test_delete_network_port_exists_owned_by_network neutron.tests.unit.db.test_migration.TestModelsMigrationsSyncMl2Psql.test_models_sync
This failure behavior started after change I6f67bb430c50ddacb2d53398de75fb5f494964a0 to use oslo.db for all of neutron connection handling instead of SQLAlchemy.[1] The failure message returned from the DB layer is very misleading. If you remove the catch that converts to that generic error about username/pass and DB being wrong/missing, you will get the real following error: OperationalError: (OperationalError) database "dzmhwmgrou" is being accessed by other users DETAIL: There are 1 other session(s) using the database. 'drop database dzmhwmgrou;' {} What is happening is that the oslo.db test case cleanup code is trying to destroy the db while a separate sqlalchemy engine (created from the alembic migration code) still has a connection to the db. The first test is required to help trigger the failure either because of imports or the run setting up database connections causing things to be cached in a module flag somewhere. I haven't looked into the exact source. Here is the diff to fix your patch to pass the same session into the alembic migration code that is setup and torn down by the test case. This should allow you to proceed forward with your work. ~/code/neutron$ git diff diff --git a/neutron/tests/unit/db/test_migration.py b/neutron/tests/unit/db/test_migration.py index 6db8ae0..c29ab67 100644 --- a/neutron/tests/unit/db/test_migration.py +++ b/neutron/tests/unit/db/test_migration.py @@ -136,9 +136,12 @@ class ModelsMigrationsSyncMixin(object): self.alembic_config.neutron_config = cfg.CONF def db_sync(self, engine): - cfg.CONF.set_override('connection', engine.url, group='database') - migration.do_alembic_command(self.alembic_config, 'upgrade', 'head') - cfg.CONF.clear_override('connection', group='database') + with mock.patch( + 'oslo.db.sqlalchemy.session.create_engine', + return_value=self.get_engine() + ): + migration.do_alembic_command(self.alembic_config, + 'upgrade', 'head') def get_engine(self): return self.engine 1. https://review.openstack.org/#/c/110016/ -- Kevin Benton On Fri, Sep 12, 2014 at 2:15 AM, Anna Kamyshnikova < akamyshnik...@mirantis.com> wrote: > This is implementing ModelsMigrationsSync test from oslo [1]. For running > it locally on Postgres you have to do the following things (it is mentioned > in comments to test): > > For the opportunistic testing you need to set up a db named > 'openstack_citest' with user 'openstack_citest' and password > 'openstack_citest' on localhost. > The test will then use that db and user/password combo to run the > tests. > > For PostgreSQL on Ubuntu this can be done with the following commands:: > > sudo -u postgres psql > postgres=# create user openstack_citest with createdb login > password > 'openstack_citest'; > postgres=# create database openstack_citest with owner > openstack_citest; > > For MySQL on Ubuntu this can be done with the following commands:: > > mysql -u root > >create database openstack_citest; > >grant all privilleges on openstack_citest.* to > openstack_citest@localhost identified by 'opensatck_citest'; > > As I said this error appeared only three weeks ago, although I'm working > on this test since 29 of April, it passed Jenkins in August without any > problems. Postgres is available there. > > [1] - > https://github.com/openstack/oslo.db/blob/master/oslo/db/sqlalchemy/test_migrations.py#L277 > > On Fri, Sep 12, 2014 at 12:28 PM, Kevin Benton <blak...@gmail.com> wrote: > >> Can you explain a bit about that test? I'm having trouble reproducing it. >> On the system (upstream Jenkins) that it's failing on, is postgres >> available with that database? >> >> On Thu, Sep 11, 2014 at 7:07 AM, Anna Kamyshnikova < >> akamyshnik...@mirantis.com> wrote: >> >>> Hello everyone! >>> >>> I'm working on implementing test in Neutron that checks that models are >>> synchronized with database state [1] [2]. This is very important change as >>> during Juno cycle big changes of database structure were done. >>> >>> I was working on it for rather long time but about three weeks ago >>> strange error appeared [3], using AssertionPool shows [4]. The problem is >>> that somehow there are more than one connection to database from each test. >>> I tried to use locks from lockutils, but it didn’t help. On db meeting we >>> decided to add TestCase just for one Ml2 plugin for starters, and then >>> continue working on this strange error, that is why there are two change >>> requests [1] and [2]. But I found out that somehow even one testcase fails >>> with the same error [5] from time to time. >>> >>> I’m asking for any suggestions that could be done in this case. It is >>> very important to get at least [1] merged in Juno. >>> >>> [1] - https://review.openstack.org/76520 >>> >>> [2] - https://review.openstack.org/120040 >>> >>> [3] - http://paste.openstack.org/show/110158/ >>> >>> [4] - http://paste.openstack.org/show/110159/ >>> >>> [5] - >>> http://logs.openstack.org/20/76520/68/check/gate-neutron-python27/63938f9/testr_results.html.gz >>> >>> Regards, >>> >>> Ann >>> >>> >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> >> -- >> Kevin Benton >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Kevin Benton
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev