Reviewed: https://review.opendev.org/711623 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2baeae75199405d46927c073daf772b2a4144294 Submitter: Zuul Branch: master
commit 2baeae75199405d46927c073daf772b2a4144294 Author: Slawek Kaplonski <[email protected]> Date: Mon Mar 2 13:56:21 2020 +0100 Clean dvr fip gateway entry when fip agent gw port is deleted In [1] there was introduced new db table which stored information about which DVR L3 agent has got already floating ip gateway port. It was to avoid race conditions and ensure that there is always only one such port per network and per agent (host). Unfortunately in [1] there was no added removal of correct record from this db table so it was causing problems when such port had to be recreated after it was already on the host and was deleted. This patch adds removal of such entry from db when needed. Closes-Bug: #1866336 [1] https://review.opendev.org/#/c/702547/ Change-Id: I56efd1b9f09c0449ce531a185fcf4db353f99fe1 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1866336 Title: Binding of floating ip agent gateway port and agent_id isn't removed Status in neutron: Fix Released Bug description: In patch https://review.opendev.org/#/c/702547/ new db level lock was introduced to ensure that there is always max. 1 FIP gateway port for network on the host. But unfortunately I missed to clean entry in this new table when such gateway port is removed. That causes issues with DVR routers when gateway is plug to the router. Steps to reproduce issue: 1. Create dvr router 2. Attach some network with vm to the router, 3. Attach external gateway network to the router, 4. It will create fip- namespace on compute, 5. remove this router, fip- namespace will be cleaned and agent's gateway port will be removed from neutron db, 6. repeat steps 1-5 - it will now fail with error like: Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: ERROR neutron.agent.l3.router_info [-] 'NoneType' object has no attribute 'get' Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: Traceback (most recent call last): Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: res = self.dispatcher.dispatch(message) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: return self._do_dispatch(endpoint, method, ctxt, args) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: result = func(ctxt, **new_args) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/neutron_lib/db/api.py", line 139, in wrapped Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: setattr(e, '_RETRY_EXCEEDED', True) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: self.force_reraise() Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: six.reraise(self.type_, self.value, self.tb) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: raise value Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/neutron_lib/db/api.py", line 135, in wrapped Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: return f(*args, **kwargs) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_db/api.py", line 154, in wrapper Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: ectxt.value = e.inner_exc Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: self.force_reraise() Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: six.reraise(self.type_, self.value, self.tb) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: raise value Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_db/api.py", line 142, in wrapper Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: return f(*args, **kwargs) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/neutron_lib/db/api.py", line 183, in wrapped Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: LOG.debug("Retry wrapper got retriable exception: %s", e) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: self.force_reraise() Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: six.reraise(self.type_, self.value, self.tb) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: raise value Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/usr/local/lib/python3.6/dist-packages/neutron_lib/db/api.py", line 179, in wrapped Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: return f(*dup_args, **dup_kwargs) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py", line 319, in get_agent_gateway_port Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: admin_ctx, network_id, host) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/opt/stack/neutron/neutron/db/l3_dvr_db.py", line 1078, in create_fip_agent_gw_port_if_not_exists Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: self._populate_mtu_and_subnets_for_ports(context, [agent_port]) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/opt/stack/neutron/neutron/db/l3_db.py", line 1762, in _populate_mtu_and_subnets_for_ports Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: for p in self._each_port_having_fixed_ips(ports)] Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/opt/stack/neutron/neutron/db/l3_db.py", line 1761, in <listcomp> Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: network_ids = [p['network_id'] Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: File "/opt/stack/neutron/neutron/db/l3_db.py", line 1709, in _each_port_having_fixed_ips Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: fixed_ips = port.get('fixed_ips', []) Mar 02 11:42:59 devstack-ubuntu-ovs neutron-l3-agent[23633]: AttributeError: 'NoneType' object has no attribute 'get' To solve this issue You need to manually remove corresponding row from "dvr_fip_gateway_port_network" db table. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1866336/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
