** Changed in: nova
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1425352
Title:
A volume remains attached and cannot be detached after attaching it
fails
Status in OpenStack Compute (nova):
Fix Released
Bug description:
A volume has been attached in nova after attaching it fails.
But its status is 'available' in cinder.
And then, the volume cannot be detached
because an error(Invalid volume: Volume must be attached in order to detach)
occurs.
The cause of the failure of attaching the volume is a rpc call timeout
between nova-api and nova-compute
in 'reserve_block_device_name' call.
(a 'reserve_block_device_name' rpc call is in '_attach_volume' method in
nova/compute/api.py)
A rpc call timeout occurs because of waiting to get the semaphore in
nova-compute.
It takes long time when many requests for same VM instance comes.
The cause of the failure of detaching the volume is that the record
remains in a 'block_device_mapping' table.
stack@devstack-juno:/opt/devstack$ nova list
+--------------------------------------+---------+--------+------------+-------------+-------------------+
| ID | Name | Status | Task State |
Power State | Networks |
+--------------------------------------+---------+--------+------------+-------------+-------------------+
| 16eb66e3-3cda-44c5-9134-cb759ffb211b | server1 | ACTIVE | - |
Running | public=10.0.2.195 |
+--------------------------------------+---------+--------+------------+-------------+-------------------+
stack@devstack-juno:/opt/devstack$ cinder list
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume
Type | Bootable | Attached to |
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| 474964ed-e334-44c0-a904-4612c52e6618 | available | TEST | 1 |
lvmdriver-1 | false | |
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
stack@devstack-juno:/opt/devstack$ nova volume-attach server1
474964ed-e334-44c0-a904-4612c52e6618
ERROR (ClientException): The server has either erred or is incapable of
performing the requested operation. (HTTP 500) (Request-ID:
req-c371e287-5e28-410f-812c-43c193bdcffc)
* RPC time out occurred between nova-api and nova-compute in
'reserve_block_device_name' call.
stack@devstack-juno:/opt/devstack$ nova show server1
+--------------------------------------+----------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-SRV-ATTR:host | devstack-juno
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | devstack-juno
|
| OS-EXT-SRV-ATTR:instance_name | instance-00000001
|
| OS-EXT-STS:power_state | 1
|
| OS-EXT-STS:task_state | -
|
| OS-EXT-STS:vm_state | active
|
| OS-SRV-USG:launched_at | 2015-02-24T07:54:06.000000
|
| OS-SRV-USG:terminated_at | -
|
| accessIPv4 |
|
| accessIPv6 |
|
| config_drive |
|
| created | 2015-02-24T07:53:54Z
|
| flavor | m1.tiny (1)
|
| hostId |
358f8c0b777e6a5218b4e297676604bbc730afe5bcc2760e64a32408 |
| id | 16eb66e3-3cda-44c5-9134-cb759ffb211b
|
| image | cirros-0.3.2-x86_64-uec
(7f38517f-adb5-4fe6-86be-f60d9065f00d) |
| key_name | -
|
| metadata | {}
|
| name | server1
|
| os-extended-volumes:volumes_attached | [{"id":
"474964ed-e334-44c0-a904-4612c52e6618"}] |
| progress | 0
|
| public network | 10.0.2.195
|
| security_groups | default
|
| status | ACTIVE
|
| tenant_id | 47b5fac1e13c4e97b36cd76bcb4ca661
|
| updated | 2015-02-24T07:54:06Z
|
| user_id | 94cf43ae770c4a32bf8ce817cd2608cb
|
+--------------------------------------+----------------------------------------------------------------+
stack@devstack-juno:/opt/devstack$ cinder list
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume
Type | Bootable | Attached to |
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
| 474964ed-e334-44c0-a904-4612c52e6618 | available | TEST | 1 |
lvmdriver-1 | false | |
+--------------------------------------+-----------+------+------+-------------+----------+-------------+
stack@devstack-juno:/opt/devstack$ nova volume-detach server1
474964ed-e334-44c0-a904-4612c52e6618
ERROR (BadRequest): Invalid volume: Volume must be attached in order to
detach. (HTTP 400) (Request-ID: req-cdc4b447-60b8-424c-b151-73d2f2874068)
stack@devstack-juno:/opt/devstack$ nova show server1
+--------------------------------------+----------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-SRV-ATTR:host | devstack-juno
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | devstack-juno
|
| OS-EXT-SRV-ATTR:instance_name | instance-00000001
|
| OS-EXT-STS:power_state | 1
|
| OS-EXT-STS:task_state | -
|
| OS-EXT-STS:vm_state | active
|
| OS-SRV-USG:launched_at | 2015-02-24T07:54:06.000000
|
| OS-SRV-USG:terminated_at | -
|
| accessIPv4 |
|
| accessIPv6 |
|
| config_drive |
|
| created | 2015-02-24T07:53:54Z
|
| flavor | m1.tiny (1)
|
| hostId |
358f8c0b777e6a5218b4e297676604bbc730afe5bcc2760e64a32408 |
| id | 16eb66e3-3cda-44c5-9134-cb759ffb211b
|
| image | cirros-0.3.2-x86_64-uec
(7f38517f-adb5-4fe6-86be-f60d9065f00d) |
| key_name | -
|
| metadata | {}
|
| name | server1
|
| os-extended-volumes:volumes_attached | [{"id":
"474964ed-e334-44c0-a904-4612c52e6618"}] |
| progress | 0
|
| public network | 10.0.2.195
|
| security_groups | default
|
| status | ACTIVE
|
| tenant_id | 47b5fac1e13c4e97b36cd76bcb4ca661
|
| updated | 2015-02-24T07:54:06Z
|
| user_id | 94cf43ae770c4a32bf8ce817cd2608cb
|
+--------------------------------------+----------------------------------------------------------------+
[Environment]
OS: Ubuntu 14.04 LTS
nova: stable/juno(2014.2.1)
cinder: stable/juno(2014.2.1)
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1425352/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
