Public bug reported: Environment: distribution: $ cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=14.04 DISTRIB_CODENAME=trusty DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
$ sudo virsh version
Compiled against library: libvirt 1.2.2
Using library: libvirt 1.2.2
Using API: QEMU 1.2.2
Running hypervisor: QEMU 2.0.0
$ git log --oneline
806113e Merge "Changed filter_by() to filter() during filtering instances in db
API"
...
There are two hosts in my environment.
Host A is controller with compute node. Host B is only as compute
node.
Produce:
1. I upgrade the nova code in Host A. and restart n-sch, n-cond,
n-cpu.
And keep the n-cpu old version on Host B.
2. do live migration
$ nova live-migration tt
it report error.
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-SRV-ATTR:host | shaohe1
|
| OS-EXT-SRV-ATTR:hostname | tt
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | shaohe1
|
| OS-EXT-SRV-ATTR:instance_name | instance-00000002
|
| OS-EXT-SRV-ATTR:kernel_id | 89e91cc1-a40c-4c9f-bcfa-37b0e94d5f57
|
| OS-EXT-SRV-ATTR:launch_index | 0
|
| OS-EXT-SRV-ATTR:ramdisk_id | d4c09101-c0fa-40de-9c88-28b9428d03fb
|
| OS-EXT-SRV-ATTR:reservation_id | r-p0884maw
|
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda
|
| OS-EXT-SRV-ATTR:user_data | -
|
| OS-EXT-STS:power_state | 1
|
| OS-EXT-STS:task_state | migrating
|
| OS-EXT-STS:vm_state | error
|
| OS-SRV-USG:launched_at | 2015-12-18T07:41:00.000000
|
| OS-SRV-USG:terminated_at | -
|
| accessIPv4 |
|
| accessIPv6 |
|
| config_drive | True
|
|
| created | 2015-12-18T07:40:44Z
|
| fault | {"message": "Remote error:
UnsupportedVersion Endpoint does not support RPC version 4.3. Attempted method:
select_destinations
|
| | [u'Traceback (most recent call last):\
|
| | ', u' File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\",
line 142, in _d", "code": 500, "details": " File
\"/opt/stack/nova/nova/conductor/manager.py\", line 295, in _live_migrate |
| | task.execute()
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/base.py\", line 27, in wrap
|
| | self.rollback()
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py\", line 204, in
__exit__
|
| | six.reraise(self.type_,
self.value, self.tb)
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/base.py\", line 24, in wrap
|
| | return original(self)
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/base.py\", line 42, in execute
|
| | return self._execute()
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/live_migrate.py\", line 58, in _execute
|
| | self.destination =
self._find_destination()
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/live_migrate.py\", line 181, in
_find_destination
|
| | spec_obj)[0]['host']
|
| | File
\"/opt/stack/nova/nova/scheduler/utils.py\", line 358, in wrapped
|
| | return func(*args, **kwargs)
|
| | File
\"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 51, in
select_destinations
|
| | return
self.queryclient.select_destinations(context, spec_obj)
|
| | File
\"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 37, in __run_method
|
| | return getattr(self.instance,
__name)(*args, **kwargs)
|
| | File
\"/opt/stack/nova/nova/scheduler/client/query.py\", line 32, in
select_destinations
|
| | return
self.scheduler_rpcapi.select_destinations(context, spec_obj)
|
| | File
\"/opt/stack/nova/nova/scheduler/rpcapi.py\", line 121, in select_destinations
|
| | return cctxt.call(ctxt,
'select_destinations', **msg_args)
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py\", line
158, in call
|
| | retry=self.retry)
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py\", line
90, in _send
|
| | timeout=timeout, retry=retry)
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\",
line 464, in send
|
| | retry=retry)
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\",
line 455, in _send
|
| | raise result
|
| | ", "created": "2016-01-22T03:41:21Z"}
|
| flavor | m1.tiny (1)
|
| hostId |
94170a2af829a4a136f9f9c9d3c3f8f10df72bc1238a5eeef5b4ffa5
|
| id | 058fc419-a8a8-4e08-b62c-a9841ef9cd3f
|
| image | cirros-0.3.4-x86_64-uec
(3476ee05-b80e-4a80-aac4-78350ce132a2)
|
| key_name | -
|
| metadata | {}
|
| name | tt
|
| os-extended-volumes:volumes_attached | []
|
| private network | fdf1:3ce8:94a:0:f816:3eff:fee1:2a95,
10.0.0.4
|
| security_groups | default
|
| status | ERROR
|
| tenant_id | f5a8829cc14c4825a2728b273aa91aa1
|
| updated | 2016-01-22T03:41:21Z
|
| user_id | b5450f0c30154d2bb1506968a05c6f80
|
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
3. when I do live migration again.
$ nova live-migration tt
ERROR (Conflict): Cannot 'os-migrateLive' instance
058fc419-a8a8-4e08-b62c-a9841ef9cd3f while it is in vm_state error (HTTP 409)
(Request-ID: req-2712a326-c3ab-4a6d-9afb-5001b3822047)
4. and stop the server.
$ nova stop tt
Cannot 'stop' instance 058fc419-a8a8-4e08-b62c-a9841ef9cd3f while it is in
task_state migrating (HTTP 409) (Request-ID:
req-580ae847-1a5d-44c8-a285-7cc31dc9e0e2)
ERROR (CommandError): Unable to stop the specified server(s).
** Affects: nova
Importance: Undecided
Assignee: lvmxh (shaohef)
Status: New
** Changed in: nova
Assignee: (unassigned) => lvmxh (shaohef)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1536916
Title:
nova's task_state still is migrating after live migration failed
Status in OpenStack Compute (nova):
New
Bug description:
Environment:
distribution:
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
$ sudo virsh version
Compiled against library: libvirt 1.2.2
Using library: libvirt 1.2.2
Using API: QEMU 1.2.2
Running hypervisor: QEMU 2.0.0
$ git log --oneline
806113e Merge "Changed filter_by() to filter() during filtering instances in
db API"
...
There are two hosts in my environment.
Host A is controller with compute node. Host B is only as compute
node.
Produce:
1. I upgrade the nova code in Host A. and restart n-sch, n-cond,
n-cpu.
And keep the n-cpu old version on Host B.
2. do live migration
$ nova live-migration tt
it report error.
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-SRV-ATTR:host | shaohe1
|
| OS-EXT-SRV-ATTR:hostname | tt
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | shaohe1
|
| OS-EXT-SRV-ATTR:instance_name | instance-00000002
|
| OS-EXT-SRV-ATTR:kernel_id | 89e91cc1-a40c-4c9f-bcfa-37b0e94d5f57
|
| OS-EXT-SRV-ATTR:launch_index | 0
|
| OS-EXT-SRV-ATTR:ramdisk_id | d4c09101-c0fa-40de-9c88-28b9428d03fb
|
| OS-EXT-SRV-ATTR:reservation_id | r-p0884maw
|
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda
|
| OS-EXT-SRV-ATTR:user_data | -
|
| OS-EXT-STS:power_state | 1
|
| OS-EXT-STS:task_state | migrating
|
| OS-EXT-STS:vm_state | error
|
| OS-SRV-USG:launched_at | 2015-12-18T07:41:00.000000
|
| OS-SRV-USG:terminated_at | -
|
| accessIPv4 |
|
| accessIPv6 |
|
| config_drive | True
|
|
| created | 2015-12-18T07:40:44Z
|
| fault | {"message": "Remote error:
UnsupportedVersion Endpoint does not support RPC version 4.3. Attempted method:
select_destinations
|
| | [u'Traceback (most recent call
last):\
|
| | ', u' File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\",
line 142, in _d", "code": 500, "details": " File
\"/opt/stack/nova/nova/conductor/manager.py\", line 295, in _live_migrate |
| | task.execute()
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/base.py\", line 27, in wrap
|
| | self.rollback()
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py\", line 204, in
__exit__
|
| | six.reraise(self.type_,
self.value, self.tb)
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/base.py\", line 24, in wrap
|
| | return original(self)
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/base.py\", line 42, in execute
|
| | return self._execute()
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/live_migrate.py\", line 58, in _execute
|
| | self.destination =
self._find_destination()
|
| | File
\"/opt/stack/nova/nova/conductor/tasks/live_migrate.py\", line 181, in
_find_destination
|
| | spec_obj)[0]['host']
|
| | File
\"/opt/stack/nova/nova/scheduler/utils.py\", line 358, in wrapped
|
| | return func(*args, **kwargs)
|
| | File
\"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 51, in
select_destinations
|
| | return
self.queryclient.select_destinations(context, spec_obj)
|
| | File
\"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 37, in __run_method
|
| | return getattr(self.instance,
__name)(*args, **kwargs)
|
| | File
\"/opt/stack/nova/nova/scheduler/client/query.py\", line 32, in
select_destinations
|
| | return
self.scheduler_rpcapi.select_destinations(context, spec_obj)
|
| | File
\"/opt/stack/nova/nova/scheduler/rpcapi.py\", line 121, in select_destinations
|
| | return cctxt.call(ctxt,
'select_destinations', **msg_args)
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py\", line
158, in call
|
| | retry=self.retry)
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py\", line
90, in _send
|
| | timeout=timeout, retry=retry)
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\",
line 464, in send
|
| | retry=retry)
|
| | File
\"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\",
line 455, in _send
|
| | raise result
|
| | ", "created":
"2016-01-22T03:41:21Z"}
|
| flavor | m1.tiny (1)
|
| hostId |
94170a2af829a4a136f9f9c9d3c3f8f10df72bc1238a5eeef5b4ffa5
|
| id | 058fc419-a8a8-4e08-b62c-a9841ef9cd3f
|
| image | cirros-0.3.4-x86_64-uec
(3476ee05-b80e-4a80-aac4-78350ce132a2)
|
| key_name | -
|
| metadata | {}
|
| name | tt
|
| os-extended-volumes:volumes_attached | []
|
| private network | fdf1:3ce8:94a:0:f816:3eff:fee1:2a95,
10.0.0.4
|
| security_groups | default
|
| status | ERROR
|
| tenant_id | f5a8829cc14c4825a2728b273aa91aa1
|
| updated | 2016-01-22T03:41:21Z
|
| user_id | b5450f0c30154d2bb1506968a05c6f80
|
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
3. when I do live migration again.
$ nova live-migration tt
ERROR (Conflict): Cannot 'os-migrateLive' instance
058fc419-a8a8-4e08-b62c-a9841ef9cd3f while it is in vm_state error (HTTP 409)
(Request-ID: req-2712a326-c3ab-4a6d-9afb-5001b3822047)
4. and stop the server.
$ nova stop tt
Cannot 'stop' instance 058fc419-a8a8-4e08-b62c-a9841ef9cd3f while it is in
task_state migrating (HTTP 409) (Request-ID:
req-580ae847-1a5d-44c8-a285-7cc31dc9e0e2)
ERROR (CommandError): Unable to stop the specified server(s).
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1536916/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
