Public bug reported:
We are seeing nova REST calls to delete an attached interface fail to work:
DELETE http://127.0.0.1:8774/v3/servers/UUID/os-attach-interfaces/UUID
DELETE http://127.0.0.1:8774/v2/UUID/servers/UUID/os-interface/UUID
In nova-cpu we see
DEBUG neutronclient.client [-] RESP:{'status': '200', 'content-length': '13',
'content-location':
'http://127.0.0.1:9696/v2.0/ports.json?network_id=0c95c39b-611f-4880-9e56-cae0eb948225&device_owner=network%3Adhcp',
'date': 'Wed, 26 Mar 2014 00:44:04 GMT', 'content-type': 'application/json;
charset=UTF-8', 'x-openstack-request-id':
'req-59333b8f-072d-4bce-b5cc-03d9af48dcc8'} {"ports": []}
/python-neutronclient/neutronclient/common/utils.py:179
nova.network.api [req-61feb3de-431b-4dcc-81c7-8a8e5b6b03ed
AttachInterfacesV3Test-673399961 AttachInterfacesV3Test-295185211] Updating
cache with info: [VIF({'ovs_interfaceid':
u'530319a7-b4d7-4289-80f8-2a1ec39c642a', 'network': Network({'bridge':
'br-int', 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'version': 4,
'type': 'fixed', 'floating_ips': [], 'address': u'10.100.0.4'})], 'version': 4,
'meta': {}, 'dns': [], 'routes': [], 'cidr': u'10.100.0.0/28', 'gateway':
IP({'meta': {}, 'version': 4, 'type': 'gateway', 'address': u'10.100.0.1'})})],
'meta': {'injected': False, 'tenant_id': u'e9a96753b0004178bf8dd2007b01d55f'},
'id': u'0c95c39b-611f-4880-9e56-cae0eb948225', 'label':
u'AttachInterfacesV3Test-1465112406-network'}), 'devname': u'tap530319a7-b4',
'qbh_params': None, 'meta': {}, 'address': u'fa:16:3e:e8:72:f4', 'active':
True, 'type': u'ovs', 'id': u'530319a7-b4d7-4289-80f8-2a1ec39c642a',
'qbg_params': None}), VIF({'ovs_interfaceid':
u'8077e5f2-e683-42f4-b9df-9c049be25cad'
, 'network': Network({'bridge': 'br-int', 'subnets': [Subnet({'ips':
[FixedIP({'meta': {}, 'version': 4, 'type': 'fixed', 'floating_ips': [],
'address': u'10.100.0.2'})], 'version': 4, 'meta': {}, 'dns': [], 'routes': [],
'cidr': u'10.100.0.0/28', 'gateway': IP({'meta': {}, 'version': 4, 'type':
'gateway', 'address': u'10.100.0.1'})})], 'meta': {'injected': False,
'tenant_id': u'e9a96753b0004178bf8dd2007b01d55f'}, 'id':
u'0c95c39b-611f-4880-9e56-cae0eb948225', 'label':
u'AttachInterfacesV3Test-1465112406-network'}), 'devname': u'tap8077e5f2-e6',
'qbh_params': None, 'meta': {}, 'address': u'fa:16:3e:83:34:a9', 'active':
True, 'type': u'ovs', 'id': u'8077e5f2-e683-42f4-b9df-9c049be25cad',
'qbg_params': None})] update_instance_cache_with_nw_info
/opt/stack/new/nova/nova/network/api.py:74
ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Port
760ab1e4-10ae-4170-b491-54ec71efd952 is not attached
TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in
_dispatch_and_reply
TRACE oslo.messaging.rpc.dispatcher incoming.message))
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in
_dispatch
TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint,
method, ctxt, args)
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in
_do_dispatch
TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint,
method)(ctxt, **new_args)
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/nova/nova/compute/manager.py", line 399, in decorated_function
TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args,
**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/nova/nova/compute/manager.py", line 4370, in detach_interface
TRACE oslo.messaging.rpc.dispatcher "attached") % port_id)
TRACE oslo.messaging.rpc.dispatcher PortNotFound: Port
760ab1e4-10ae-4170-b491-54ec71efd952 is not attached
TRACE oslo.messaging.rpc.dispatcher
ERROR oslo.messaging._drivers.common [-] Returning exception Port
760ab1e4-10ae-4170-b491-54ec71efd952 is not attached to caller
ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call
last):\n', ' File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in
_dispatch_and_reply\n incoming.message))\n', ' File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in
_dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', '
File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line
122, in _do_dispatch\n result = getattr(endpoint, method)(ctxt,
**new_args)\n', ' File "/opt/stack/new/nova/nova/compute/manager.py", line
399, in decorated_function\n return function(self, context, *args,
**kwargs)\n', ' File "/opt/stack/new/nova/nova/compute/manager.py", line 4370,
in detach_interface\n "attached") % port_id)\n', 'PortNotFound: Port
760ab1e4-10ae-4170-b491-54ec71efd952 is not attached\n']
http://logs.openstack.org/22/82922/2/gate/gate-tempest-dsvm-
neutron/87c67c5/logs/screen-n-cpu.txt.gz#_2014-03-26_00_44_04_250
It looks like when nova does 'curl -i
http://127.0.0.1:9696/v2.0/ports.json?network_id=0c95c39b-611f-4880-9e56-cae0eb948225&device_owner=network%3Adhcp'
neutron is returning an empty list.
logstash query: message:"PortNotFound: Port" AND message:"is not attached" AND
filename:logs*screen-n-cpu.txt
** Affects: neutron
Importance: Undecided
Status: New
** Affects: nova
Importance: Undecided
Status: New
** Also affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1297992
Title:
deleting an interface hits a race condition and fails
Status in OpenStack Neutron (virtual network service):
New
Status in OpenStack Compute (Nova):
New
Bug description:
We are seeing nova REST calls to delete an attached interface fail to work:
DELETE http://127.0.0.1:8774/v3/servers/UUID/os-attach-interfaces/UUID
DELETE http://127.0.0.1:8774/v2/UUID/servers/UUID/os-interface/UUID
In nova-cpu we see
DEBUG neutronclient.client [-] RESP:{'status': '200', 'content-length': '13',
'content-location':
'http://127.0.0.1:9696/v2.0/ports.json?network_id=0c95c39b-611f-4880-9e56-cae0eb948225&device_owner=network%3Adhcp',
'date': 'Wed, 26 Mar 2014 00:44:04 GMT', 'content-type': 'application/json;
charset=UTF-8', 'x-openstack-request-id':
'req-59333b8f-072d-4bce-b5cc-03d9af48dcc8'} {"ports": []}
/python-neutronclient/neutronclient/common/utils.py:179
nova.network.api [req-61feb3de-431b-4dcc-81c7-8a8e5b6b03ed
AttachInterfacesV3Test-673399961 AttachInterfacesV3Test-295185211] Updating
cache with info: [VIF({'ovs_interfaceid':
u'530319a7-b4d7-4289-80f8-2a1ec39c642a', 'network': Network({'bridge':
'br-int', 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'version': 4,
'type': 'fixed', 'floating_ips': [], 'address': u'10.100.0.4'})], 'version': 4,
'meta': {}, 'dns': [], 'routes': [], 'cidr': u'10.100.0.0/28', 'gateway':
IP({'meta': {}, 'version': 4, 'type': 'gateway', 'address': u'10.100.0.1'})})],
'meta': {'injected': False, 'tenant_id': u'e9a96753b0004178bf8dd2007b01d55f'},
'id': u'0c95c39b-611f-4880-9e56-cae0eb948225', 'label':
u'AttachInterfacesV3Test-1465112406-network'}), 'devname': u'tap530319a7-b4',
'qbh_params': None, 'meta': {}, 'address': u'fa:16:3e:e8:72:f4', 'active':
True, 'type': u'ovs', 'id': u'530319a7-b4d7-4289-80f8-2a1ec39c642a',
'qbg_params': None}), VIF({'ovs_interfaceid':
u'8077e5f2-e683-42f4-b9df-9c049be25ca
d', 'network': Network({'bridge': 'br-int', 'subnets': [Subnet({'ips':
[FixedIP({'meta': {}, 'version': 4, 'type': 'fixed', 'floating_ips': [],
'address': u'10.100.0.2'})], 'version': 4, 'meta': {}, 'dns': [], 'routes': [],
'cidr': u'10.100.0.0/28', 'gateway': IP({'meta': {}, 'version': 4, 'type':
'gateway', 'address': u'10.100.0.1'})})], 'meta': {'injected': False,
'tenant_id': u'e9a96753b0004178bf8dd2007b01d55f'}, 'id':
u'0c95c39b-611f-4880-9e56-cae0eb948225', 'label':
u'AttachInterfacesV3Test-1465112406-network'}), 'devname': u'tap8077e5f2-e6',
'qbh_params': None, 'meta': {}, 'address': u'fa:16:3e:83:34:a9', 'active':
True, 'type': u'ovs', 'id': u'8077e5f2-e683-42f4-b9df-9c049be25cad',
'qbg_params': None})] update_instance_cache_with_nw_info
/opt/stack/new/nova/nova/network/api.py:74
ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling:
Port 760ab1e4-10ae-4170-b491-54ec71efd952 is not attached
TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in
_dispatch_and_reply
TRACE oslo.messaging.rpc.dispatcher incoming.message))
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in
_dispatch
TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint,
method, ctxt, args)
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in
_do_dispatch
TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint,
method)(ctxt, **new_args)
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/nova/nova/compute/manager.py", line 399, in decorated_function
TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args,
**kwargs)
TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/new/nova/nova/compute/manager.py", line 4370, in detach_interface
TRACE oslo.messaging.rpc.dispatcher "attached") % port_id)
TRACE oslo.messaging.rpc.dispatcher PortNotFound: Port
760ab1e4-10ae-4170-b491-54ec71efd952 is not attached
TRACE oslo.messaging.rpc.dispatcher
ERROR oslo.messaging._drivers.common [-] Returning exception Port
760ab1e4-10ae-4170-b491-54ec71efd952 is not attached to caller
ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call
last):\n', ' File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in
_dispatch_and_reply\n incoming.message))\n', ' File
"/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in
_dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', '
File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line
122, in _do_dispatch\n result = getattr(endpoint, method)(ctxt,
**new_args)\n', ' File "/opt/stack/new/nova/nova/compute/manager.py", line
399, in decorated_function\n return function(self, context, *args,
**kwargs)\n', ' File "/opt/stack/new/nova/nova/compute/manager.py", line 4370,
in detach_interface\n "attached") % port_id)\n', 'PortNotFound: Port
760ab1e4-10ae-4170-b491-54ec71efd952 is not attached\n']
http://logs.openstack.org/22/82922/2/gate/gate-tempest-dsvm-
neutron/87c67c5/logs/screen-n-cpu.txt.gz#_2014-03-26_00_44_04_250
It looks like when nova does 'curl -i
http://127.0.0.1:9696/v2.0/ports.json?network_id=0c95c39b-611f-4880-9e56-cae0eb948225&device_owner=network%3Adhcp'
neutron is returning an empty list.
logstash query: message:"PortNotFound: Port" AND message:"is not attached"
AND filename:logs*screen-n-cpu.txt
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1297992/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
