** Changed in: nova
Status: Fix Committed => Fix Released
** Changed in: nova
Milestone: None => liberty-3
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1435633
Title:
live migration fails at check_can_live_migrate_destination, but the
status of this instance is still "migrating".
Status in OpenStack Compute (nova):
Fix Released
Bug description:
I have two compute node, host lxlconductor1 and host lxlcompute1.
A migration failed at function of "_call_livem_checks_on_host" because
check_can_live_migrate_destination rpc call's MessagingTimeout Exception, but
the status of this instance is still migrating.
2015-05-23 09:11:53.456 26651 ERROR nova.conductor.manager
[req-b836290d-93d7-4bee-b974-433d067fc287 fdfd8f6a96ed4a1a9d60dbb4be1e0cf7
7f6c0898a35d4de9b53642ae984d3cf3] Migration of instance
8f50c28d-0caf-43df-b154-97f56cea7d2d to host None unexpectedly failed.
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager Traceback (most
recent call last):
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 768, in
_live_migrate
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager
block_migration, disk_over_commit)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/nova/conductor/tasks/live_migrate.py", line
201, in execute
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager return
task.execute()
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/nova/conductor/tasks/live_migrate.py", line
62, in execute
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager
self.destination = self._find_destination()
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/nova/conductor/tasks/live_migrate.py", line
173, in _find_destination
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager
self._call_livem_checks_on_host(host)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/nova/conductor/tasks/live_migrate.py", line
144, in _call_livem_checks_on_host
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager destination,
self.block_migration, self.disk_over_commit)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/nova/compute/rpcapi.py", line 360, in
check_can_live_migrate_destination
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager
disk_over_commit=disk_over_commit)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/client.py", line 150, in
call
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager
wait_for_reply=True, timeout=timeout)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/oslo/messaging/transport.py", line 90, in
_send
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager
timeout=timeout)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line
412, in send
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager return
self._send(target, ctxt, message, wait_for_reply, timeout)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line
403, in _send
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager result =
self._waiter.wait(msg_id, timeout)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line
267, in wait
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager reply, ending
= self._poll_connection(msg_id, timeout)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager File
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line
217, in _poll_connection
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager % msg_id)
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager MessagingTimeout:
Timed out waiting for a reply to message ID ff504056d4cd462a9fca96d3cfa8183a
2015-05-23 09:11:53.456 26651 TRACE nova.conductor.manager
2015-05-23 09:11:53.483 26651 ERROR oslo.messaging.rpc.dispatcher [-]
Exception during message handling: Migration error: Timed out waiting for a
reply to message ID ff504056d4cd462a9fca96d3cfa8183a
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher Traceback
(most recent call last):
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 133,
in _dispatch_and_reply
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher
incoming.message))
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 176,
in _dispatch
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher return
self._do_dispatch(endpoint, method, ctxt, args)
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 122,
in _do_dispatch
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher result
= getattr(endpoint, method)(ctxt, **new_args)
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in
inner
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher return
func(*args, **kwargs)
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 685, in
migrate_server
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher
block_migration, disk_over_commit)
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 796, in
_live_migrate
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher raise
exception.MigrationError(reason=ex)
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher
MigrationError: Migration error: Timed out waiting for a reply to message ID
ff504056d4cd462a9fca96d3cfa8183a
2015-05-23 09:11:53.483 26651 TRACE oslo.messaging.rpc.dispatcher
2015-05-23 09:11:53.484 26651 ERROR oslo.messaging._drivers.common [-]
Returning exception Migration error: Timed out waiting for a reply to message
ID ff504056d4cd462a9fca96d3cfa8183a to caller
2015-05-23 09:11:53.484 26651 ERROR oslo.messaging._drivers.common [-]
['Traceback (most recent call last):\n', ' File
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 133,
in _dispatch_and_reply\n incoming.message))\n', ' File
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 176,
in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', '
File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line
122, in _do_dispatch\n result = getattr(endpoint, method)(ctxt,
**new_args)\n', ' File
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in
inner\n return func(*args, **kwargs)\n', ' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 685, in
migrate_server\n block_migration, disk_over_commit)\n', ' File
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 796, in
_live_migrate\n raise exception.MigrationError(reason=ex)\n
', 'MigrationError: Migration error: Timed out waiting for a reply to message
ID ff504056d4cd462a9fca96d3cfa8183a\n']
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1435633/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
