** Also affects: charm-nova-compute
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1824437
Title:
Issue with upgrade of nova-compute from Pike to Queens (RemoteError:
Remote error: IncompatibleObjectVersion Version 1.4 of MigrationList
is not supported)
Status in OpenStack nova-compute charm:
New
Status in OpenStack Compute (nova):
New
Bug description:
This issue is hapenning uring openstack upgrade on xenial. The procedure use
was just changing the config "openstack-origin", and not using
action-managed-upgrade, there were no
instances spawned at the time of upgrade.
Juju verion: 2.3.7-xenial-amd64
When I upgrade from Pike to Queens the nova-compute charm gets blocked
with error:
nova-compute/0* blocked idle 6 10.10.245.9
Services not running that should be: nova-compute
When I saw this I decided to create a fresh Pike installation (instead
of upgrading from Ocata). And then tried to upgrade it to queens. I
git the same error. I do see many postings on internet where people
had same issue but none of those posts were answered. I may need you
help. I tried to post this on our launchpad channel, but I do not see
any place to file a bug or post a question (probably will need your
help with that too).
After upgrade, I see below error and stack trace
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service
[req-e37138fc-10f2-484d-9355-41d559d36e75 - - - - -] Error starting thread.:
RemoteError: Remote error: IncompatibleObjectVersion Version 1.4 of
MigrationList is not supported
[u'Traceback (most recent call last):\n', u' File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in
_process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213,
in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u'
File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line
183, in _do_dispatch\n result = func(ctxt, **new_args)\n', u' File
"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 131, in
object_class_action_versions\n objname, object_versions[objname])\n', u'
File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line
387, in obj_class_from_name\n supported=latest_ver)\n',
u'IncompatibleObjectVersion: Version 1.4 of MigrationList is not supported\n'].
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service Traceback (most
recent call last):
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 729, in
run_service
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service service.start()
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 161, in start
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service
self.manager.init_host()
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1161, in
init_host
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service
evacuated_instances = self._destroy_evacuated_instances(context)
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 658, in
_destroy_evacuated_instances
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service filters)
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 177, in
wrapper
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service args, kwargs)
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 240, in
object_class_action_versions
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service args=args,
kwargs=kwargs)
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 174, in
call
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service retry=self.retry)
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 131, in
_send
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service timeout=timeout,
retry=retry)
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line
559, in send
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service retry=retry)
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service File
"/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line
550, in _send
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service raise result
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service RemoteError: Remote
error: IncompatibleObjectVersion Version 1.4 of MigrationList is not supported
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service [u'Traceback (most
recent call last):\n', u' File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in
_process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213,
in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u'
File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line
183, in _do_dispatch\n result = func(ctxt, **new_args)\n', u' File
"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 131, in
object_class_action_versions\n objname, object_versions[objname])\n', u'
File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line
387, in obj_class_from_name\n supported=latest_ver)\n',
u'IncompatibleObjectVersion: Version 1.4 of MigrationList is not supported\n'].
2019-04-10 13:13:42.729 7883 ERROR oslo_service.service
Here is the DB before and after upgrade
Pre upgrade (at Pike)
nova_api DB
mysql> select * from migrate_version;
+---------------+---------------------------------------------------------------------------------+---------+
| repository_id | repository_path
| version |
+---------------+---------------------------------------------------------------------------------+---------+
| nova_api |
/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api_migrations/migrate_repo
| 45 |
+---------------+---------------------------------------------------------------------------------+---------+
1 row in set (0.00 sec)
nova DB
mysql> select * from migrate_version;
+---------------+------------------------------------------------------------------+---------+
| repository_id | repository_path
| version |
+---------------+------------------------------------------------------------------+---------+
| nova |
/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migrate_repo | 362 |
+---------------+------------------------------------------------------------------+---------+
1 row in set (0.00 sec)
After upgrade
nova_api DB
mysql> select * from migrate_version;
+---------------+---------------------------------------------------------------------------------+---------+
| repository_id | repository_path
| version |
+---------------+---------------------------------------------------------------------------------+---------+
| nova_api |
/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api_migrations/migrate_repo
| 52 |
+---------------+---------------------------------------------------------------------------------+---------+
1 row in set (0.00 sec)
nova DB
mysql> select * from migrate_version;
+---------------+------------------------------------------------------------------+---------+
| repository_id | repository_path
| version |
+---------------+------------------------------------------------------------------+---------+
| nova |
/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migrate_repo | 378 |
+---------------+------------------------------------------------------------------+---------+
1 row in set (0.00 sec)
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-nova-compute/+bug/1824437/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
