Reviewed: https://review.openstack.org/632904
Committed:
https://git.openstack.org/cgit/openstack/nova/commit/?id=6489f2d2b44827d133dad9a3bb52436ee304a934
Submitter: Zuul
Branch: master
commit 6489f2d2b44827d133dad9a3bb52436ee304a934
Author: melanie witt <melwi...@gmail.com>
Date: Fri Jan 18 16:30:40 2019 +0000
Raise 403 instead of 500 error from attach volume API
Currently, the libvirt driver has a limit on the maximum number of
disk devices allowed to attach to a single instance of 26. If a user
attempts to attach a volume which would make the total number of
attached disk devices > 26 for the instance, the user receives a
500 error from the API.
This adds a new exception type TooManyDiskDevices and raises it for the
"No free disk devices names" condition, instead of InternalError, and
handles it in the attach volume API. We raise TooManyDiskDevices
directly from the libvirt driver because InternalError is ambiguous and
can be raised for different error reasons within the same method call.
Closes-Bug: #1770527
Change-Id: I1b08ed6826d7eb41ecdfc7102e5e8fcf3d1eb2e1
** Changed in: nova
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1770527
Title:
add volume fails over 26vols and returns 500 API error with libvirt
driver
Status in OpenStack Compute (nova):
Fix Released
Bug description:
Description
===========
openstack server add volume fails over 26vols
Steps to reproduce
==================
* I did execute this openstack command.
# create a instance with single volume named sles15rc
# openstack server add volume sles15rc vol2
# openstack server add volume sles15rc vol3
:
# openstack server add volume sles15rc vol26
# openstack server add volume sles15rc vol27
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/
and attach the Nova API log if possible.
<class 'NovaException_Remote'> (HTTP 500) (Request-ID:
req-d95fea94-31fe-4063-9262-a84088cbaf29)
#
Expected result
===============
This command will success.
# openstack server add volume sles15rc vol27
and instance will had '/dev/vdaa' device volumes.
Actual result
=============
# openstack server add volume sles15rc vol26
instance will get named /dev/vdz volume.
Next,
# openstack server add volume sles15rc vol27
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/
and attach the Nova API log if possible.
<class 'NovaException_Remote'> (HTTP 500) (Request-ID:
req-d95fea94-31fe-4063-9262-a84088cbaf29)
I did kvm based command, Its OK.
#virsh attach-device instance-0000001e ~/vol27.xml
#virsh attach-device instance-0000001e ~/vol28.xml
and instance will had a volumes, vdaa,vdab
It's nova limitation problem.
I made a concept blueprint.
https://blueprints.launchpad.net/nova/+spec/nova-improvement-of-maximum-attach-volumes-more-than-26-vols
Environment
===========
SOC7(Suse Openstack Cloud version 7 openstack-newton)
Logs & Configs
==============
Nova API returned this result.
NovaException_Remote: No free disk device names for prefix 'vd'
2018-05-11 08:51:11.602 3667 INFO nova.osapi_compute.wsgi.server
[req-6cf5fdbf-9681-445b-8117-a61acd4057f4 2e2bd0e5b23c4665a5a78d710e2bbeb5
b516d75b77bf4dc2b70
5d680487f6f19 - default default] 10.19.3.70 "POST
/v2.1/b516d75b77bf4dc2b705d680487f6f19/servers/6a324dc8-d2ea-4eba-9870-e2ada1cb2bf4/os-volume_attachments
HTT
P/1.1" status: 200 len: 522 time: 0.2834640
2018-05-11 08:51:11.716 3668 INFO nova.osapi_compute.wsgi.server [-]
127.0.0.1 "GET / HTTP/1.1" status: 200 len: 491 time: 0.0007210
2018-05-11 08:51:13.880 3668 INFO nova.api.openstack.wsgi
[req-c7fc0e7a-13ec-48e7-947c-cd22232aa31f 2e2bd0e5b23c4665a5a78d710e2bbeb5
b516d75b77bf4dc2b705d68048
7f6f19 - default default] HTTP exception thrown: Instance sles15rc could not
be found.
2018-05-11 08:51:13.881 3668 INFO nova.osapi_compute.wsgi.server
[req-c7fc0e7a-13ec-48e7-947c-cd22232aa31f 2e2bd0e5b23c4665a5a78d710e2bbeb5
b516d75b77bf4dc2b70
5d680487f6f19 - default default] 10.19.3.70 "GET
/v2.1/b516d75b77bf4dc2b705d680487f6f19/servers/sles15rc HTTP/1.1" status: 404
len: 432 time: 0.0449822
2018-05-11 08:51:13.887 3668 INFO nova.api.openstack.wsgi
[req-7cbf4817-8810-4c30-8e09-8b106f66dee4 2e2bd0e5b23c4665a5a78d710e2bbeb5
b516d75b77bf4dc2b705d68048
7f6f19 - default default] HTTP exception thrown: Instance sles15rc could not
be found.
2018-05-11 08:51:13.889 3668 INFO nova.osapi_compute.wsgi.server
[req-7cbf4817-8810-4c30-8e09-8b106f66dee4 2e2bd0e5b23c4665a5a78d710e2bbeb5
b516d75b77bf4dc2b70
5d680487f6f19 - default default] 10.19.3.70 "GET
/v2.1/b516d75b77bf4dc2b705d680487f6f19/servers/sles15rc HTTP/1.1" status: 404
len: 432 time: 0.0053861
2018-05-11 08:51:13.929 3668 INFO nova.osapi_compute.wsgi.server
[req-40067327-b2a6-4194-acc6-17696377d123 2e2bd0e5b23c4665a5a78d710e2bbeb5
b516d75b77bf4dc2b70
5d680487f6f19 - default default] 10.19.3.70 "GET
/v2.1/b516d75b77bf4dc2b705d680487f6f19/servers?name=sles15rc HTTP/1.1" status:
200 len: 687 time: 0.0372620
2018-05-11 08:51:14.096 3668 INFO nova.osapi_compute.wsgi.server
[req-4a17ff34-f457-4e4b-8975-ad9fafa2e635 2e2bd0e5b23c4665a5a78d710e2bbeb5
b516d75b77bf4dc2b70
5d680487f6f19 - default default] 10.19.3.70 "GET
/v2.1/b516d75b77bf4dc2b705d680487f6f19/servers/6a324dc8-d2ea-4eba-9870-e2ada1cb2bf4
HTTP/1.1" status: 200 len:
3110 time: 0.1642580
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
[req-d95fea94-31fe-4063-9262-a84088cbaf29 2e2bd0e5b23c4665a5a78d710e2bbeb5
b516d75b77bf4dc2b70
5d680487f6f19 - default default] Unexpected exception in API method
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions Traceback
(most recent call last):
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 338,
in wrapp
ed
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
f(*args, **kwargs)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in
wrapper
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
func(*args, **kwargs)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/api/openstack/compute/volumes.py", line
325, in
create
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
volume_id, device)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/api.py", line 166, in inner
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
function(self, context, instance, *args, **kwargs)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/api.py", line 147, in inner
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
f(self, context, instance, *args, **kw)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3518, in
attach_volume
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
disk_bus, device_type)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3455, in
_attach_volume
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
device_type=device_type)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3437, in
_create_volume_bd
m
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
device_type=device_type)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/rpcapi.py", line 926, in
reserve_block_d
evice_name
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
cctxt.call(ctxt, 'reserve_block_device_name', **kw)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in
call
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
retry=self.retry)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in
_send
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
timeout=timeout, retry=retry)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
584, in send
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
retry=retry)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
575, in _send
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions raise
result
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
NovaException_Remote: No free disk device names for prefix 'vd'
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions Traceback
(most recent call last):
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in
_process_incoming
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions res =
self.dispatcher.dispatch(message)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150,
in dispatch
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
self._do_dispatch(endpoint, method, ctxt, args)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121,
in _do_dispatch
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions result =
func(ctxt, **new_args)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 75, in
wrapped
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
function_name, call_dict, binary)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
self.force_reraise()
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
six.reraise(self.type_, self.value, self.tb)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 66, in
wrapped
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
f(self, context, *args, **kw)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 188, in
decorated_function
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
LOG.warning(msg, e, instance=instance)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
self.force_reraise()
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
six.reraise(self.type_, self.value, self.tb)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 157, in
decorated_function
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
function(self, context, *args, **kwargs)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 216, in
decorated_function
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
kwargs['instance'], e, sys.exc_info())
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
self.force_reraise()
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
six.reraise(self.type_, self.value, self.tb)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 204, in
decorated_function
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
function(self, context, *args, **kwargs)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4683, in
reserve_block_device_name
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
do_reserve()
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in
inner
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions return
f(*args, **kwargs)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4677, in
do_reserve
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
instance, bdms, new_bdm)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1496, in
_get_device_name_for_instance
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
instance, bdms, block_device_obj)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7864, in
get_device_name_for_instance
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
block_device_obj, mapping=instance_info['mapping'])
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/blockinfo.py", line 394, in
get_info_from_bdm
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
device_name = find_disk_dev_for_disk_bus(padded_mapping, bdm_bus)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/blockinfo.py", line 197, in
find_disk_dev_for_disk_bus
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
dev_prefix)
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
NovaException: No free disk device names for prefix 'vd'
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.611 3668 ERROR nova.api.openstack.extensions
2018-05-11 08:51:14.613 3668 INFO nova.api.openstack.wsgi
[req-d95fea94-31fe-4063-9262-a84088cbaf29 2e2bd0e5b23c4665a5a78d710e2bbeb5
b516d75b77bf4dc2b705d680487f6f19 - default default] HTTP exception thrown:
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and
attach the Nova API log if possible.
<class 'NovaException_Remote'>
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1770527/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
