Should nova do some tidying up if it gets a permission denied?
** Also affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1413610
Title:
Nova volume-update leaves volumes stuck in attaching/detaching
Status in Cinder:
New
Status in OpenStack Compute (Nova):
New
Bug description:
There is a problem with the nova command 'volume-update' that leaves
cinder volumes in the states 'attaching' and 'deleting'.
If the nova command 'volume-update' is used by a non admin user the
command fails and the volumes referenced in the command are left in
the states 'attaching' and 'deleting'.
For example, if a non admin user runs the command
$ nova volume-update d39dc7f2-929d-49bb-b22f-56adb3f378c7
f0c3ea8f-c00f-4db8-aa20-e2dc6a1ddc9b 59b0cf66-67c8-4041-a505-78000b9c71f6
Will result in the two volumes stuck like this:
$ cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
| ID | Status | Display Name | Size |
Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
| 59b0cf66-67c8-4041-a505-78000b9c71f6 | attaching | vol2 | 1 |
None | false | |
| f0c3ea8f-c00f-4db8-aa20-e2dc6a1ddc9b | detaching | vol1 | 1 |
None | false | d39dc7f2-929d-49bb-b22f-56adb3f378c7 |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
And the following in the cinder-api log:
2015-01-21 11:00:03.969 13588 DEBUG keystonemiddleware.auth_token [-]
Received request from user: user_id None, project_id None, roles None service:
user_id None, project_id None, roles None __call__
/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/keystonemiddleware/auth_token.py:746
2015-01-21 11:00:03.970 13588 DEBUG routes.middleware
[req-d08bc456-14e9-4f97-9f4e-a9f87e7d6add a00b77059c8c4bbbb00a4ff9a8496b2d
d40e3207e34a4b558bf2d58bd3fe268a - - -] Matched POST
/d40e3207e34a4b558bf2d58bd3fe268a/volumes/f0c3ea8f-c00f-4db8-aa20-e2dc6a1ddc9b/action
__call__
/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/routes/middleware.py:100
2015-01-21 11:00:03.971 13588 DEBUG routes.middleware
[req-d08bc456-14e9-4f97-9f4e-a9f87e7d6add a00b77059c8c4bbbb00a4ff9a8496b2d
d40e3207e34a4b558bf2d58bd3fe268a - - -] Route path:
'/{project_id}/volumes/:(id)/action', defaults: {'action': u'action',
'controller': <cinder.api.openstack.wsgi.Resource object at 0x7febe7a89850>}
__call__
/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/routes/middleware.py:102
2015-01-21 11:00:03.971 13588 DEBUG routes.middleware
[req-d08bc456-14e9-4f97-9f4e-a9f87e7d6add a00b77059c8c4bbbb00a4ff9a8496b2d
d40e3207e34a4b558bf2d58bd3fe268a - - -] Match dict: {'action': u'action',
'controller': <cinder.api.openstack.wsgi.Resource object at 0x7febe7a89850>,
'project_id': u'd40e3207e34a4b558bf2d58bd3fe268a', 'id':
u'f0c3ea8f-c00f-4db8-aa20-e2dc6a1ddc9b'} __call__
/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/routes/middleware.py:103
2015-01-21 11:00:03.972 13588 INFO cinder.api.openstack.wsgi
[req-d08bc456-14e9-4f97-9f4e-a9f87e7d6add a00b77059c8c4bbbb00a4ff9a8496b2d
d40e3207e34a4b558bf2d58bd3fe268a - - -] POST
http://192.0.2.24:8776/v1/d40e3207e34a4b558bf2d58bd3fe268a/volumes/f0c3ea8f-c00f-4db8-aa20-e2dc6a1ddc9b/action
2015-01-21 11:00:03.972 13588 DEBUG cinder.api.openstack.wsgi
[req-d08bc456-14e9-4f97-9f4e-a9f87e7d6add a00b77059c8c4bbbb00a4ff9a8496b2d
d40e3207e34a4b558bf2d58bd3fe268a - - -] Action body:
{"os-migrate_volume_completion": {"new_volume":
"59b0cf66-67c8-4041-a505-78000b9c71f6", "error": false}} get_method
/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:1010
2015-01-21 11:00:03.973 13588 INFO cinder.api.openstack.wsgi
[req-d08bc456-14e9-4f97-9f4e-a9f87e7d6add a00b77059c8c4bbbb00a4ff9a8496b2d
d40e3207e34a4b558bf2d58bd3fe268a - - -]
http://192.0.2.24:8776/v1/d40e3207e34a4b558bf2d58bd3fe268a/volumes/f0c3ea8f-c00f-4db8-aa20-e2dc6a1ddc9b/action
returned with HTTP 403
2015-01-21 11:00:03.975 13588 INFO eventlet.wsgi.server
[req-d08bc456-14e9-4f97-9f4e-a9f87e7d6add a00b77059c8c4bbbb00a4ff9a8496b2d
d40e3207e34a4b558bf2d58bd3fe268a - - -] 127.0.0.1 - - [21/Jan/2015 11:00:03]
"POST
/v1/d40e3207e34a4b558bf2d58bd3fe268a/volumes/f0c3ea8f-c00f-4db8-aa20-e2dc6a1ddc9b/action
HTTP/1.1" 403 429 0.123613
The problem is that the nova policy.json file allows a non admin user to run
the command 'volume-update', but the cinder policy.json file requires the admin
role to run the action os-migrate_volume_completion, which is called by nova as
part of the 'update-volume' process.
The operation will complete successfully if it is performed by an
admin user, or if it is run by a non admin user after the cinder
policy.json file is updated.
The simplest fix would be to remove the "rule:admin_api" requirement from the
"os-migrate_volume_completion" action in cinder policy.json
To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1413610/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
