Public bug reported: When detaching multiple volumes from multiple instances the stack trace below was encountered. The db shows a device name present for all volumes attempted to be detached (also below).
A similar stack trace was found at: http://pastebin.com/sDR2AVGH 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp **args) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 93, in wrapped 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp payload) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 76, in wrapped 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp return f(self, context, *args, **kw) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 247, in decorated_function 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp pass 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 233, in decorated_function 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 275, in decorated_function 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp e, sys.exc_info()) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 262, in decorated_function 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3784, in detach_volume 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp self._detach_volume(context, instance, bdm) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3722, in _detach_volume 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp mp = bdm['device_name'] 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp TypeError: 'NoneType' object is unsubscriptable [root@ci-5254003B9180 nova]# nova list +--------------------------------------+----------------------------------------------+--------+------------+-------------+----------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+----------------------------------------------+--------+------------+-------------+----------------------+ | 1d80f083-d64b-492e-90fe-f92da31efb07 | 3nsg-T5-313eff9a-7958-4a28-9188-7c9ac6ab29f8 | ACTIVE | None | Running | public=16.125.108.48 | | 89a8c95c-9831-4645-acaf-3879b4ac9046 | 3nsg-T5-4b06bfe3-2507-4a5c-8d47-9d66af1b1383 | ACTIVE | None | Running | public=16.125.108.47 | | 5c6fdac4-01e9-4e1c-9d15-dde9bc4f1b77 | 3nsg-T5-893f6fd3-93a7-4abb-91cf-5fee1474746d | ACTIVE | None | Running | public=16.125.108.51 | +--------------------------------------+----------------------------------------------+--------+------------+-------------+----------------------+ [root@ci-5254003B9180 nova]# cinder list +--------------------------------------+-----------+---------------------------------------------+------+-------------+----------+-------------+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+---------------------------------------------+------+-------------+----------+-------------+ | 031e915c-adf7-47b0-be84-61c41759b747 | available | vg6-T5-27e51943-4c38-4036-a874-bfb583d4adae | 5 | jimvt1 | false | | | 08fa1624-a0b1-45c4-84cd-6cb7d2eddbd6 | available | vg5-T5-93e3a2c6-6e47-4ac8-b429-0209b5a73c7d | 5 | jimvt1 | false | | | 23ce9948-5885-40b3-90f4-5f690411274b | available | vg4-T5-aeb8b220-12d2-4fc2-8b17-448477ebfbd3 | 5 | jimvt1 | false | | | 3aac388e-5722-4a65-a70d-aaeab26abaf3 | available | vg2-T5-3d0a33e8-374e-4bc9-9b5c-3c1354b58769 | 5 | jimvt1 | false | | | 488a7101-694e-485a-ab9b-a2fa31ca033c | available | vg5-T5-2a2b8141-8593-4639-bbab-7648fbc04281 | 5 | jimvt1 | false | | | 5d4d2ce4-d7ba-4eff-af55-3e1d08c49e33 | available | vg5-T5-349c93bb-629e-4e8d-afa4-f7425a5bb477 | 5 | jimvt1 | false | | | 7088730f-bdf6-46c7-84d7-822aaad17d18 | available | vg3-T5-08356a97-d648-44d7-a8c6-78a9c77ecd01 | 5 | jimvt1 | false | | | 71828b9b-c966-4be2-8f54-0371a4dd3f02 | available | vg2-T5-f9d35eea-a36c-46c6-98db-4717494cf7c9 | 5 | jimvt1 | false | | | 7a97cd36-1908-4a90-b991-dab391a97b95 | available | vg6-T5-cf116e7a-af7a-4714-bb5b-2b209eba2c23 | 5 | jimvt1 | false | | | 7da825a7-f6db-4010-bdd6-353ee885092d | available | vg3-T5-4bf9ba8c-8668-4341-b259-504f1d8ad23c | 5 | jimvt1 | false | | | b3dcf5c2-c3be-4e19-b4ae-4d216c99debb | available | vg1-T5-7ee09b70-eba2-4392-b299-c0f9f1ffefbb | 5 | jimvt1 | false | | | b99d6e41-74d9-45cf-9b3c-c0471c9eda5b | available | vg4-T5-d39fd38f-8087-4f33-8f7a-fbd4c54b2191 | 5 | jimvt1 | false | | | d0f1ff77-1e78-426f-9689-2a59e144bf8c | available | vg6-T5-3f49e820-4745-4253-9f69-3d995dae08b4 | 5 | jimvt1 | false | | | f503598c-8d9e-49be-aeb3-b172940c7344 | available | vg2-T5-030206e0-a91d-463b-8a87-459aafe0d63d | 5 | jimvt1 | false | | +--------------------------------------+-----------+---------------------------------------------+------+-------------+----------+-------------+ nova=> select device_name, volume_id from block_device_mapping where volume_id='031e915c-adf7-47b0-be84-61c41759b747' or volume_id='08fa1624-a0b1-45c4-84cd-6cb7d2eddbd6' or volume_id='23ce9948-5885-40b3-90f4-5f690411274b' or volume_id='3aac388e-5722-4a65-a70d-aaeab26abaf3' or volume_id='488a7101-694e-485a-ab9b-a2fa31ca033c' or volume_id='5d4d2ce4-d7ba-4eff-af55-3e1d08c49e33' or volume_id='7088730f-bdf6-46c7-84d7-822aaad17d18' or volume_id='71828b9b-c966-4be2-8f54-0371a4dd3f02' or volume_id='7a97cd36-1908-4a90-b991-dab391a97b95' or volume_id='7da825a7-f6db-4010-bdd6-353ee885092d' or volume_id='b3dcf5c2-c3be-4e19-b4ae-4d216c99debb' or volume_id='b99d6e41-74d9-45cf-9b3c-c0471c9eda5b' or volume_id='d0f1ff77-1e78-426f-9689-2a59e144bf8c' or volume_id='f503598c-8d9e-49be-aeb3-b172940c7344'; device_name | volume_id -------------+-------------------------------------- /dev/vdf | 5d4d2ce4-d7ba-4eff-af55-3e1d08c49e33 /dev/vdd | 7da825a7-f6db-4010-bdd6-353ee885092d /dev/vde | b99d6e41-74d9-45cf-9b3c-c0471c9eda5b /dev/vdf | 488a7101-694e-485a-ab9b-a2fa31ca033c /dev/vdg | 7a97cd36-1908-4a90-b991-dab391a97b95 /dev/vdc | 3aac388e-5722-4a65-a70d-aaeab26abaf3 /dev/vde | 23ce9948-5885-40b3-90f4-5f690411274b /dev/vdg | 031e915c-adf7-47b0-be84-61c41759b747 /dev/vdf | 08fa1624-a0b1-45c4-84cd-6cb7d2eddbd6 /dev/vdb | b3dcf5c2-c3be-4e19-b4ae-4d216c99debb /dev/vdc | f503598c-8d9e-49be-aeb3-b172940c7344 /dev/vdc | 71828b9b-c966-4be2-8f54-0371a4dd3f02 /dev/vdg | d0f1ff77-1e78-426f-9689-2a59e144bf8c /dev/vdd | 7088730f-bdf6-46c7-84d7-822aaad17d18 (14 rows) ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1290566 Title: nova detach results in TypeError: 'NoneType' object is unsubscriptable Status in OpenStack Compute (Nova): New Bug description: When detaching multiple volumes from multiple instances the stack trace below was encountered. The db shows a device name present for all volumes attempted to be detached (also below). A similar stack trace was found at: http://pastebin.com/sDR2AVGH 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp **args) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 93, in wrapped 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp payload) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 76, in wrapped 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp return f(self, context, *args, **kw) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 247, in decorated_function 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp pass 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 233, in decorated_function 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 275, in decorated_function 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp e, sys.exc_info()) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 262, in decorated_function 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3784, in detach_volume 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp self._detach_volume(context, instance, bdm) 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3722, in _detach_volume 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp mp = bdm['device_name'] 2014-03-10 07:59:38.220 22304 TRACE nova.openstack.common.rpc.amqp TypeError: 'NoneType' object is unsubscriptable [root@ci-5254003B9180 nova]# nova list +--------------------------------------+----------------------------------------------+--------+------------+-------------+----------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+----------------------------------------------+--------+------------+-------------+----------------------+ | 1d80f083-d64b-492e-90fe-f92da31efb07 | 3nsg-T5-313eff9a-7958-4a28-9188-7c9ac6ab29f8 | ACTIVE | None | Running | public=16.125.108.48 | | 89a8c95c-9831-4645-acaf-3879b4ac9046 | 3nsg-T5-4b06bfe3-2507-4a5c-8d47-9d66af1b1383 | ACTIVE | None | Running | public=16.125.108.47 | | 5c6fdac4-01e9-4e1c-9d15-dde9bc4f1b77 | 3nsg-T5-893f6fd3-93a7-4abb-91cf-5fee1474746d | ACTIVE | None | Running | public=16.125.108.51 | +--------------------------------------+----------------------------------------------+--------+------------+-------------+----------------------+ [root@ci-5254003B9180 nova]# cinder list +--------------------------------------+-----------+---------------------------------------------+------+-------------+----------+-------------+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+---------------------------------------------+------+-------------+----------+-------------+ | 031e915c-adf7-47b0-be84-61c41759b747 | available | vg6-T5-27e51943-4c38-4036-a874-bfb583d4adae | 5 | jimvt1 | false | | | 08fa1624-a0b1-45c4-84cd-6cb7d2eddbd6 | available | vg5-T5-93e3a2c6-6e47-4ac8-b429-0209b5a73c7d | 5 | jimvt1 | false | | | 23ce9948-5885-40b3-90f4-5f690411274b | available | vg4-T5-aeb8b220-12d2-4fc2-8b17-448477ebfbd3 | 5 | jimvt1 | false | | | 3aac388e-5722-4a65-a70d-aaeab26abaf3 | available | vg2-T5-3d0a33e8-374e-4bc9-9b5c-3c1354b58769 | 5 | jimvt1 | false | | | 488a7101-694e-485a-ab9b-a2fa31ca033c | available | vg5-T5-2a2b8141-8593-4639-bbab-7648fbc04281 | 5 | jimvt1 | false | | | 5d4d2ce4-d7ba-4eff-af55-3e1d08c49e33 | available | vg5-T5-349c93bb-629e-4e8d-afa4-f7425a5bb477 | 5 | jimvt1 | false | | | 7088730f-bdf6-46c7-84d7-822aaad17d18 | available | vg3-T5-08356a97-d648-44d7-a8c6-78a9c77ecd01 | 5 | jimvt1 | false | | | 71828b9b-c966-4be2-8f54-0371a4dd3f02 | available | vg2-T5-f9d35eea-a36c-46c6-98db-4717494cf7c9 | 5 | jimvt1 | false | | | 7a97cd36-1908-4a90-b991-dab391a97b95 | available | vg6-T5-cf116e7a-af7a-4714-bb5b-2b209eba2c23 | 5 | jimvt1 | false | | | 7da825a7-f6db-4010-bdd6-353ee885092d | available | vg3-T5-4bf9ba8c-8668-4341-b259-504f1d8ad23c | 5 | jimvt1 | false | | | b3dcf5c2-c3be-4e19-b4ae-4d216c99debb | available | vg1-T5-7ee09b70-eba2-4392-b299-c0f9f1ffefbb | 5 | jimvt1 | false | | | b99d6e41-74d9-45cf-9b3c-c0471c9eda5b | available | vg4-T5-d39fd38f-8087-4f33-8f7a-fbd4c54b2191 | 5 | jimvt1 | false | | | d0f1ff77-1e78-426f-9689-2a59e144bf8c | available | vg6-T5-3f49e820-4745-4253-9f69-3d995dae08b4 | 5 | jimvt1 | false | | | f503598c-8d9e-49be-aeb3-b172940c7344 | available | vg2-T5-030206e0-a91d-463b-8a87-459aafe0d63d | 5 | jimvt1 | false | | +--------------------------------------+-----------+---------------------------------------------+------+-------------+----------+-------------+ nova=> select device_name, volume_id from block_device_mapping where volume_id='031e915c-adf7-47b0-be84-61c41759b747' or volume_id='08fa1624-a0b1-45c4-84cd-6cb7d2eddbd6' or volume_id='23ce9948-5885-40b3-90f4-5f690411274b' or volume_id='3aac388e-5722-4a65-a70d-aaeab26abaf3' or volume_id='488a7101-694e-485a-ab9b-a2fa31ca033c' or volume_id='5d4d2ce4-d7ba-4eff-af55-3e1d08c49e33' or volume_id='7088730f-bdf6-46c7-84d7-822aaad17d18' or volume_id='71828b9b-c966-4be2-8f54-0371a4dd3f02' or volume_id='7a97cd36-1908-4a90-b991-dab391a97b95' or volume_id='7da825a7-f6db-4010-bdd6-353ee885092d' or volume_id='b3dcf5c2-c3be-4e19-b4ae-4d216c99debb' or volume_id='b99d6e41-74d9-45cf-9b3c-c0471c9eda5b' or volume_id='d0f1ff77-1e78-426f-9689-2a59e144bf8c' or volume_id='f503598c-8d9e-49be-aeb3-b172940c7344'; device_name | volume_id -------------+-------------------------------------- /dev/vdf | 5d4d2ce4-d7ba-4eff-af55-3e1d08c49e33 /dev/vdd | 7da825a7-f6db-4010-bdd6-353ee885092d /dev/vde | b99d6e41-74d9-45cf-9b3c-c0471c9eda5b /dev/vdf | 488a7101-694e-485a-ab9b-a2fa31ca033c /dev/vdg | 7a97cd36-1908-4a90-b991-dab391a97b95 /dev/vdc | 3aac388e-5722-4a65-a70d-aaeab26abaf3 /dev/vde | 23ce9948-5885-40b3-90f4-5f690411274b /dev/vdg | 031e915c-adf7-47b0-be84-61c41759b747 /dev/vdf | 08fa1624-a0b1-45c4-84cd-6cb7d2eddbd6 /dev/vdb | b3dcf5c2-c3be-4e19-b4ae-4d216c99debb /dev/vdc | f503598c-8d9e-49be-aeb3-b172940c7344 /dev/vdc | 71828b9b-c966-4be2-8f54-0371a4dd3f02 /dev/vdg | d0f1ff77-1e78-426f-9689-2a59e144bf8c /dev/vdd | 7088730f-bdf6-46c7-84d7-822aaad17d18 (14 rows) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1290566/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
