Public bug reported: Seen here:
http://logs.openstack.org/48/461448/1/check/gate-tempest-dsvm-multinode- live-migration-ubuntu- xenial/a3f4426/logs/screen-n-cpu.txt.gz?level=TRACE#_May_01_22_15_43_039605 May 01 22:15:43.039605 ubuntu-xenial-2-node-osic-cloud1-s3500-8668392 nova-compute[22173]: ERROR oslo_messaging.rpc.server [req-233741a8-bb50-4bf6-a9ec-6c67e6360d39 tempest-LiveBlockMigrationTestJSON-978755190 tempest-LiveBlockMigrationTestJSON-978755190] Exception during message handling ERROR oslo_messaging.rpc.server Traceback (most recent call last): ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 157, in _process_incoming ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/exception_wrapper.py", line 77, in wrapped ERROR oslo_messaging.rpc.server function_name, call_dict, binary) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/exception_wrapper.py", line 68, in wrapped ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/utils.py", line 660, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 215, in decorated_function ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 203, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 5178, in check_can_live_migrate_destination ERROR oslo_messaging.rpc.server disk_over_commit) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 5194, in _do_check_can_live_migrate_destination ERROR oslo_messaging.rpc.server dest_check_data) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/rpcapi.py", line 478, in check_can_live_migrate_source ERROR oslo_messaging.rpc.server dest_check_data=dest_check_data) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call ERROR oslo_messaging.rpc.server retry=self.retry) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 98, in _send ERROR oslo_messaging.rpc.server timeout=timeout, retry=retry) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 505, in send ERROR oslo_messaging.rpc.server retry=retry) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 496, in _send ERROR oslo_messaging.rpc.server raise result ERROR oslo_messaging.rpc.server InvalidSharedStorage_Remote: ubuntu-xenial-2-node-osic-cloud1-s3500-8668392-565309 is not on shared storage: Live migration can not be used without shared storage except a booted from volume VM which does not have a local disk. Looks like this just started today (5/1): http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22is%20not%20on%20shared%20storage%3A%20Live%20migration%20can%20not%20be%20used%20without%20shared%20storage%20except%20a%20booted%20from%20volume%20VM%20which%20does%20not%20have%20a%20local%20disk%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d ** Affects: nova Importance: Undecided Status: Confirmed ** Tags: live-migration -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1687511 Title: live migration job blowing up with "InvalidSharedStorage: ubuntu- xenial-2-node-osic-cloud1-s3500-8668392-565309 is not on shared storage: Live migration can not be used without shared storage except a booted from volume VM which does not have a local disk." Status in OpenStack Compute (nova): Confirmed Bug description: Seen here: http://logs.openstack.org/48/461448/1/check/gate-tempest-dsvm- multinode-live-migration-ubuntu- xenial/a3f4426/logs/screen-n-cpu.txt.gz?level=TRACE#_May_01_22_15_43_039605 May 01 22:15:43.039605 ubuntu-xenial-2-node-osic-cloud1-s3500-8668392 nova-compute[22173]: ERROR oslo_messaging.rpc.server [req-233741a8-bb50-4bf6-a9ec-6c67e6360d39 tempest-LiveBlockMigrationTestJSON-978755190 tempest-LiveBlockMigrationTestJSON-978755190] Exception during message handling ERROR oslo_messaging.rpc.server Traceback (most recent call last): ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 157, in _process_incoming ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/exception_wrapper.py", line 77, in wrapped ERROR oslo_messaging.rpc.server function_name, call_dict, binary) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/exception_wrapper.py", line 68, in wrapped ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/utils.py", line 660, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 215, in decorated_function ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 203, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 5178, in check_can_live_migrate_destination ERROR oslo_messaging.rpc.server disk_over_commit) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/manager.py", line 5194, in _do_check_can_live_migrate_destination ERROR oslo_messaging.rpc.server dest_check_data) ERROR oslo_messaging.rpc.server File "/opt/stack/new/nova/nova/compute/rpcapi.py", line 478, in check_can_live_migrate_source ERROR oslo_messaging.rpc.server dest_check_data=dest_check_data) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call ERROR oslo_messaging.rpc.server retry=self.retry) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 98, in _send ERROR oslo_messaging.rpc.server timeout=timeout, retry=retry) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 505, in send ERROR oslo_messaging.rpc.server retry=retry) ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 496, in _send ERROR oslo_messaging.rpc.server raise result ERROR oslo_messaging.rpc.server InvalidSharedStorage_Remote: ubuntu-xenial-2-node-osic-cloud1-s3500-8668392-565309 is not on shared storage: Live migration can not be used without shared storage except a booted from volume VM which does not have a local disk. Looks like this just started today (5/1): http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22is%20not%20on%20shared%20storage%3A%20Live%20migration%20can%20not%20be%20used%20without%20shared%20storage%20except%20a%20booted%20from%20volume%20VM%20which%20does%20not%20have%20a%20local%20disk%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1687511/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
