Re: [openstack-dev] [nova] unit test migration failure specific to MySQL/MariaDB - 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey'

Matt Riedemann Wed, 07 Jan 2015 10:30:48 -0800


On 1/6/2015 5:40 PM, Mike Bayer wrote:

Hello -

Victor Sergeyev and I are both observing the following test failure which 
occurs with all the tests underneath 
nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.    This is against 
master with a brand new tox environment and everything at the default.

It does not seem to be occurring on gates that run these tests and 
interestingly the tests seem to complete very quickly (under seven seconds) on 
the gate as well; the failures here take between 50-100 seconds to occur, not 
fully deterministically, and only on the MySQL backend; the Postgresql and 
SQLite versions of these tests pass.  I’m running against MariaDB server 
10.0.14 with Python 2.7.8 on Fedora 21.

Below is the test just for test_walk_versions, but the warnings (not 
necessarily the failures themselves) here also occur for test_migration_267 as 
well as test_innodb_tables.

I’m still looking into what the cause of this is, I’d imagine it’s something 
related to newer MySQL versions or perhaps MariaDB vs. MySQL, I’m just putting 
it up here in case someone already knows what this is or has some clue to save 
me some time figuring it out.  I apologize if I’m just doing something dumb, 
I’ve only recently begun to run Nova’s test suite in full against all backends, 
so I haven’t yet put intelligent thought into this nor have I tried to yet look 
at the migration in question causing the problem.  Will do that next.


[mbayer@thinkpad nova]$ tox -e py27 -- 
nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions
py27 develop-inst-noop: /home/mbayer/dev/openstack/nova
py27 runtests: PYTHONHASHSEED='0'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | bash tools/pretty_tox.sh 
nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions
running testr
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./nova/tests} 
--list
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./nova/tests}  
--load-list /tmp/tmpw7zqhE

2015-01-06 18:28:12.913 32435 WARNING oslo.db.sqlalchemy.session 
[req-5cc6731f-00ef-43df-8aec-4914a44d12c5 ] MySQL SQL mode is '', consider 
enabling TRADITIONAL or STRICT_ALL_TABLES
{0} 
nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions 
[51.553131s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
     Traceback (most recent call last):
       File "nova/tests/unit/db/test_migrations.py", line 151, in 
test_walk_versions
         self.walk_versions(self.snake_walk, self.downgrade)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 193, in walk_versions
         self.migrate_up(version, with_data=True)
       File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
         super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 263, in migrate_up
         self.REPOSITORY, version)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 186, in upgrade
         return _migrate(url, repository, version, upgrade=True, err=err, 
**opts)
       File "<string>", line 2, in _migrate
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py",
 line 160, in with_engine
         return f(*a, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 366, in _migrate
         schema.runchange(ver, change, changeset.step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py",
 line 93, in runchange
         change.run(self.engine, step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py",
 line 148, in run
         script_func(engine)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 103, in upgrade
         process_null_records(meta, scan=False)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 89, in process_null_records
         table.columns.uuid.alter(nullable=False)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 534, in alter
         return alter_column(self, *p, **k)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 141, in alter_column
         engine._run_visitor(visitorcallable, delta)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1616, in _run_visitor
         conn._run_visitor(visitorcallable, element, **kwargs)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1245, in _run_visitor
         **kwargs).traverse_single(element)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 56, in traverse_single
         ret = super(AlterTableVisitor, self).traverse_single(elem)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py",
 line 120, in traverse_single
         return meth(obj, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py",
 line 47, in visit_column
         self.execute()
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 44, in execute
         return self.connection.execute(self.buffer.getvalue())
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 721, in execute
         return self._execute_text(object, multiparams, params)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 870, in _execute_text
         statement, parameters
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 958, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1155, in _handle_dbapi_exception
         util.raise_from_cause(newraise, exc_info)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
 line 199, in raise_from_cause
         reraise(type(exception), exception, tb=exc_tb)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 951, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py",
 line 436, in do_execute
         cursor.execute(statement, parameters)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py",
 line 205, in execute
         self.errorhandler(self, exc, value)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py",
 line 36, in defaulterrorhandler
         raise errorclass, errorvalue
     OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used 
in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 
'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid 
VARCHAR(36) NOT NULL' ()
     Traceback (most recent call last):
     _StringException: Empty attachments:
       stdout

     stderr: {{{
     
/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436:
 Warning: Duplicate index 
'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on 
the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be 
disallowed in a future release.
       cursor.execute(statement, parameters)
     }}}

     Traceback (most recent call last):
       File "nova/tests/unit/db/test_migrations.py", line 151, in 
test_walk_versions
         self.walk_versions(self.snake_walk, self.downgrade)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 193, in walk_versions
         self.migrate_up(version, with_data=True)
       File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
         super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 263, in migrate_up
         self.REPOSITORY, version)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 186, in upgrade
         return _migrate(url, repository, version, upgrade=True, err=err, 
**opts)
       File "<string>", line 2, in _migrate
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py",
 line 160, in with_engine
         return f(*a, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 366, in _migrate
         schema.runchange(ver, change, changeset.step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py",
 line 93, in runchange
         change.run(self.engine, step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py",
 line 148, in run
         script_func(engine)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 103, in upgrade
         process_null_records(meta, scan=False)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 89, in process_null_records
         table.columns.uuid.alter(nullable=False)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 534, in alter
         return alter_column(self, *p, **k)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 141, in alter_column
         engine._run_visitor(visitorcallable, delta)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1616, in _run_visitor
         conn._run_visitor(visitorcallable, element, **kwargs)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1245, in _run_visitor
         **kwargs).traverse_single(element)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 56, in traverse_single
         ret = super(AlterTableVisitor, self).traverse_single(elem)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py",
 line 120, in traverse_single
         return meth(obj, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py",
 line 47, in visit_column
         self.execute()
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 44, in execute
         return self.connection.execute(self.buffer.getvalue())
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 721, in execute
         return self._execute_text(object, multiparams, params)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 870, in _execute_text
         statement, parameters
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 958, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1155, in _handle_dbapi_exception
         util.raise_from_cause(newraise, exc_info)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
 line 199, in raise_from_cause
         reraise(type(exception), exception, tb=exc_tb)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 951, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py",
 line 436, in do_execute
         cursor.execute(statement, parameters)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py",
 line 205, in execute
         self.errorhandler(self, exc, value)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py",
 line 36, in defaulterrorhandler
         raise errorclass, errorvalue
     OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used 
in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 
'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid 
VARCHAR(36) NOT NULL' ()

     Traceback (most recent call last):
     _StringException: Empty attachments:
       stdout

     stderr: {{{
     
/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436:
 Warning: Duplicate index 
'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on 
the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be 
disallowed in a future release.
       cursor.execute(statement, parameters)
     }}}

     Traceback (most recent call last):
       File "nova/tests/unit/db/test_migrations.py", line 151, in 
test_walk_versions
         self.walk_versions(self.snake_walk, self.downgrade)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 193, in walk_versions
         self.migrate_up(version, with_data=True)
       File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
         super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 263, in migrate_up
         self.REPOSITORY, version)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 186, in upgrade
         return _migrate(url, repository, version, upgrade=True, err=err, 
**opts)
       File "<string>", line 2, in _migrate
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py",
 line 160, in with_engine
         return f(*a, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 366, in _migrate
         schema.runchange(ver, change, changeset.step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py",
 line 93, in runchange
         change.run(self.engine, step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py",
 line 148, in run
         script_func(engine)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 103, in upgrade
         process_null_records(meta, scan=False)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 89, in process_null_records
         table.columns.uuid.alter(nullable=False)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 534, in alter
         return alter_column(self, *p, **k)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 141, in alter_column
         engine._run_visitor(visitorcallable, delta)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1616, in _run_visitor
         conn._run_visitor(visitorcallable, element, **kwargs)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1245, in _run_visitor
         **kwargs).traverse_single(element)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 56, in traverse_single
         ret = super(AlterTableVisitor, self).traverse_single(elem)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py",
 line 120, in traverse_single
         return meth(obj, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py",
 line 47, in visit_column
         self.execute()
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 44, in execute
         return self.connection.execute(self.buffer.getvalue())
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 721, in execute
         return self._execute_text(object, multiparams, params)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 870, in _execute_text
         statement, parameters
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 958, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1155, in _handle_dbapi_exception
         util.raise_from_cause(newraise, exc_info)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
 line 199, in raise_from_cause
         reraise(type(exception), exception, tb=exc_tb)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 951, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py",
 line 436, in do_execute
         cursor.execute(statement, parameters)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py",
 line 205, in execute
         self.errorhandler(self, exc, value)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py",
 line 36, in defaulterrorhandler
         raise errorclass, errorvalue
     OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used 
in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 
'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid 
VARCHAR(36) NOT NULL' ()



Captured stderr:
~~~~~~~~~~~~~~~~
     
/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436:
 Warning: Duplicate index 
'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on 
the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be 
disallowed in a future release.
       cursor.execute(statement, parameters)

Slowest Tests
Test id                                                                        
Runtime (s)
-----------------------------------------------------------------------------  
-----------
nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions  
51.553

==============================
Failed 1 tests - output below:
==============================

nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions
-----------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
     Traceback (most recent call last):
       File "nova/tests/unit/db/test_migrations.py", line 151, in 
test_walk_versions
         self.walk_versions(self.snake_walk, self.downgrade)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 193, in walk_versions
         self.migrate_up(version, with_data=True)
       File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
         super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 263, in migrate_up
         self.REPOSITORY, version)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 186, in upgrade
         return _migrate(url, repository, version, upgrade=True, err=err, 
**opts)
       File "<string>", line 2, in _migrate
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py",
 line 160, in with_engine
         return f(*a, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 366, in _migrate
         schema.runchange(ver, change, changeset.step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py",
 line 93, in runchange
         change.run(self.engine, step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py",
 line 148, in run
         script_func(engine)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 103, in upgrade
         process_null_records(meta, scan=False)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 89, in process_null_records
         table.columns.uuid.alter(nullable=False)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 534, in alter
         return alter_column(self, *p, **k)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 141, in alter_column
         engine._run_visitor(visitorcallable, delta)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1616, in _run_visitor
         conn._run_visitor(visitorcallable, element, **kwargs)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1245, in _run_visitor
         **kwargs).traverse_single(element)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 56, in traverse_single
         ret = super(AlterTableVisitor, self).traverse_single(elem)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py",
 line 120, in traverse_single
         return meth(obj, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py",
 line 47, in visit_column
         self.execute()
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 44, in execute
         return self.connection.execute(self.buffer.getvalue())
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 721, in execute
         return self._execute_text(object, multiparams, params)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 870, in _execute_text
         statement, parameters
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 958, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1155, in _handle_dbapi_exception
         util.raise_from_cause(newraise, exc_info)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
 line 199, in raise_from_cause
         reraise(type(exception), exception, tb=exc_tb)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 951, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py",
 line 436, in do_execute
         cursor.execute(statement, parameters)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py",
 line 205, in execute
         self.errorhandler(self, exc, value)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py",
 line 36, in defaulterrorhandler
         raise errorclass, errorvalue
     OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used 
in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 
'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid 
VARCHAR(36) NOT NULL' ()
     Traceback (most recent call last):
     _StringException: Empty attachments:
       stdout

     stderr: {{{
     
/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436:
 Warning: Duplicate index 
'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on 
the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be 
disallowed in a future release.
       cursor.execute(statement, parameters)
     }}}

     Traceback (most recent call last):
       File "nova/tests/unit/db/test_migrations.py", line 151, in 
test_walk_versions
         self.walk_versions(self.snake_walk, self.downgrade)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 193, in walk_versions
         self.migrate_up(version, with_data=True)
       File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
         super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 263, in migrate_up
         self.REPOSITORY, version)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 186, in upgrade
         return _migrate(url, repository, version, upgrade=True, err=err, 
**opts)
       File "<string>", line 2, in _migrate
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py",
 line 160, in with_engine
         return f(*a, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 366, in _migrate
         schema.runchange(ver, change, changeset.step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py",
 line 93, in runchange
         change.run(self.engine, step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py",
 line 148, in run
         script_func(engine)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 103, in upgrade
         process_null_records(meta, scan=False)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 89, in process_null_records
         table.columns.uuid.alter(nullable=False)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 534, in alter
         return alter_column(self, *p, **k)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 141, in alter_column
         engine._run_visitor(visitorcallable, delta)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1616, in _run_visitor
         conn._run_visitor(visitorcallable, element, **kwargs)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1245, in _run_visitor
         **kwargs).traverse_single(element)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 56, in traverse_single
         ret = super(AlterTableVisitor, self).traverse_single(elem)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py",
 line 120, in traverse_single
         return meth(obj, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py",
 line 47, in visit_column
         self.execute()
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 44, in execute
         return self.connection.execute(self.buffer.getvalue())
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 721, in execute
         return self._execute_text(object, multiparams, params)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 870, in _execute_text
         statement, parameters
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 958, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1155, in _handle_dbapi_exception
         util.raise_from_cause(newraise, exc_info)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
 line 199, in raise_from_cause
         reraise(type(exception), exception, tb=exc_tb)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 951, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py",
 line 436, in do_execute
         cursor.execute(statement, parameters)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py",
 line 205, in execute
         self.errorhandler(self, exc, value)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py",
 line 36, in defaulterrorhandler
         raise errorclass, errorvalue
     OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used 
in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 
'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid 
VARCHAR(36) NOT NULL' ()

     Traceback (most recent call last):
     _StringException: Empty attachments:
       stdout

     stderr: {{{
     
/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436:
 Warning: Duplicate index 
'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on 
the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be 
disallowed in a future release.
       cursor.execute(statement, parameters)
     }}}

     Traceback (most recent call last):
       File "nova/tests/unit/db/test_migrations.py", line 151, in 
test_walk_versions
         self.walk_versions(self.snake_walk, self.downgrade)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 193, in walk_versions
         self.migrate_up(version, with_data=True)
       File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
         super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py",
 line 263, in migrate_up
         self.REPOSITORY, version)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 186, in upgrade
         return _migrate(url, repository, version, upgrade=True, err=err, 
**opts)
       File "<string>", line 2, in _migrate
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py",
 line 160, in with_engine
         return f(*a, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py",
 line 366, in _migrate
         schema.runchange(ver, change, changeset.step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py",
 line 93, in runchange
         change.run(self.engine, step)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py",
 line 148, in run
         script_func(engine)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 103, in upgrade
         process_null_records(meta, scan=False)
       File 
"/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py",
 line 89, in process_null_records
         table.columns.uuid.alter(nullable=False)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 534, in alter
         return alter_column(self, *p, **k)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py",
 line 141, in alter_column
         engine._run_visitor(visitorcallable, delta)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1616, in _run_visitor
         conn._run_visitor(visitorcallable, element, **kwargs)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1245, in _run_visitor
         **kwargs).traverse_single(element)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 56, in traverse_single
         ret = super(AlterTableVisitor, self).traverse_single(elem)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py",
 line 120, in traverse_single
         return meth(obj, **kw)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py",
 line 47, in visit_column
         self.execute()
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py",
 line 44, in execute
         return self.connection.execute(self.buffer.getvalue())
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 721, in execute
         return self._execute_text(object, multiparams, params)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 870, in _execute_text
         statement, parameters
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 958, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 1155, in _handle_dbapi_exception
         util.raise_from_cause(newraise, exc_info)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
 line 199, in raise_from_cause
         reraise(type(exception), exception, tb=exc_tb)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
 line 951, in _execute_context
         context)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py",
 line 436, in do_execute
         cursor.execute(statement, parameters)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py",
 line 205, in execute
         self.errorhandler(self, exc, value)
       File 
"/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py",
 line 36, in defaulterrorhandler
         raise errorclass, errorvalue
     OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used 
in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 
'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid 
VARCHAR(36) NOT NULL' ()



Captured stderr:
~~~~~~~~~~~~~~~~
     
/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436:
 Warning: Duplicate index 
'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on 
the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be 
disallowed in a future release.
       cursor.execute(statement, parameters)



======
Totals
======
Run: 1 in 51.553131 sec.
  - Passed: 0
  - Skipped: 0
  - Failed: 1

==============
Worker Balance
==============
  - Worker 0 (1 tests) => 0:00:51.553131s
ERROR: InvocationError: '/usr/bin/bash tools/pretty_tox.sh 
nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions'
____________________________________________________________________________________
 summary 
____________________________________________________________________________________
ERROR:   py27: commands failed
[mbayer@thinkpad nova]$




_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

We're running the migrations with Tempest in an internal CI with MariaDBand RHEL 7 and haven't had migration issues like this.

The deprecation warning is odd, unless that's coming from the 216migration where the duplicate index is defined, but it's removed in the249 migration so that shouldn't be a problem.

As for the 267 migration, it's failing to set the instances.uuid columnto nullable=False. Before it does this, it scans the tables looking fortables with an instance_uuid column that has a foreign key back to theinstances table and where the instance_uuid value is NULL, it will failit if finds one of those (on purpose, you have to manually purge thoserows).

Once it's checked that there aren't any null instance_uuid entries withfkeys back to instances, it alters the instances.uuid column to benon-nullable so the UniqueConstraint on instances.uuid can be created.

There is a known issue with a unique index being created oninstances.uuid in the 216 migration and the 267 migration adds theunique constraint, so there is some duplication there, but I don't thinkthat's what's causing your failure.


--

Thanks,

Matt Riedemann


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] unit test migration failure specific to MySQL/MariaDB - 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey'

Reply via email to