Reviewed: https://review.openstack.org/558898 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ed7dd4dae0cf80f2314d9e113995e63a362b8295 Submitter: Zuul Branch: master
commit ed7dd4dae0cf80f2314d9e113995e63a362b8295 Author: Swaminathan Vasudevan <[email protected]> Date: Wed Apr 4 11:33:56 2018 -0700 DVR: Check for item_allocator key before releasing item_allocator class maintains the allocation of items(link local address) from a given pool. There may be cases where the key has been already released or not being remembered while trying to release it. So it makes sense to lookup for the key before releasing the key. Change-Id: I34765dd3efa6c6742caeb7f6ae2ff14009dce6dd Closes-Bug: #1761260 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1761260 Title: DVR: Add a check for the item_allocator IP before trying to release it, since we see a KeyError sometimes, when the item is not there anymore. Status in neutron: Fix Released Bug description: We have seen this Traceback in Pike based installation, while trying to cleanup a gateway with DVR routers. 2018-04-03 20:30:10.081 9672 DEBUG neutron.agent.l3.dvr_fip_ns [-] Delete FIP link interfaces for router: e415276a-4f37-4ee0-ba48-12d3909153c7 delete_rtr_2_fip_link /opt/stack/venv/neutron-20180328T152147Z/lib/python2.7/site-pac kages/neutron/agent/l3/dvr_fip_ns.py:364 2018-04-03 20:30:10.082 9672 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-e415276a-4f37-4ee0-ba48-12d3909153c7', 'ip', '-o', 'link', 'show', 'rfp-e415276a-4'] execute_ro otwrap_daemon /opt/stack/venv/neutron-20180328T152147Z/lib/python2.7/site-packages/neutron/agent/linux/utils.py:108 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info [-] u'e415276a-4f37-4ee0-ba48-12d3909153c7': KeyError: u'e415276a-4f37-4ee0-ba48-12d3909153c7' 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info Traceback (most recent call last): 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info File "/opt/stack/venv/neutron-20180328T152147Z/lib/python2.7/site-packages/neutron/common/utils.py", line 186, in call 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info return func(*args, **kwargs) 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info File "/opt/stack/venv/neutron-20180328T152147Z/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 1118, in process_delete 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info self._process_external_on_delete() 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info File "/opt/stack/venv/neutron-20180328T152147Z/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 890, in _process_external_on_delete 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info self._process_external_gateway(ex_gw_port) 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info File "/opt/stack/venv/neutron-20180328T152147Z/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 799, in _process_external_gateway 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info self.external_gateway_removed(self.ex_gw_port, interface_name) 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info File "/opt/stack/venv/neutron-20180328T152147Z/lib/python2.7/site-packages/neutron/agent/l3/dvr_local_router.py", line 513, in external_gateway_removed 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info self.fip_ns.delete_rtr_2_fip_link(self) 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info File "/opt/stack/venv/neutron-20180328T152147Z/lib/python2.7/site-packages/neutron/agent/l3/dvr_fip_ns.py", line 402, in delete_rtr_2_fip_link 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info self.local_subnets.release(ri.router_id) 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info File "/opt/stack/venv/neutron-20180328T152147Z/lib/python2.7/site-packages/neutron/agent/l3/item_allocator.py", line 116, in release 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info self.pool.add(self.allocations.pop(key)) 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info KeyError: u'e415276a-4f37-4ee0-ba48-12d3909153c7' 2018-04-03 20:30:10.179 9672 ERROR neutron.agent.l3.router_info Probably a check to make sure if the Key exists before release would be a good idea. We might also see if we can reproduce this in the master branch. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1761260/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
