Reviewed: https://review.openstack.org/367679 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9d24490da8552542309dc7b9d6cbc695af4c6de6 Submitter: Jenkins Branch: master
commit 9d24490da8552542309dc7b9d6cbc695af4c6de6 Author: Kevin Benton <[email protected]> Date: Thu Sep 8 15:19:06 2016 -0700 Handle racey teardowns in DHCP agent Capture port not found exceptions from port updates of DHCP ports that no longer exist. The DHCP agent already checks the return value for None in case any of the other things went missing (e.g. Subnet, Network), so checking for ports disappearing makes sense. The corresponding agent-side log message for this has also been downgraded to debug since this is a normal occurrence. This also cleans up log noise from calling reload_allocations on networks that have already been torn down due to all of the subnets being removed. Closes-Bug: #1621650 Change-Id: I495401d225c664b8f1cf7b3d51747f3b47c24fc0 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1621650 Title: PortNotFound in DHCP agent logs Status in neutron: Fix Released Bug description: The DHCP agent can call update_dhcp_port, in which case a PortNotFound exception can be thrown. This currently goes uncaught and leads to lots of log noise. http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Remote%20error%3A%20PortNotFound%5C%22 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server [req-cadd9638-976b-4c45-8a67-b8e027c31b07 - -] Exception during message handling 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server ectxt.value = e.inner_exc 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server self.force_reraise() 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/opt/stack/new/neutron/neutron/db/api.py", line 82, in wrapped 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server traceback.format_exc()) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server self.force_reraise() 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/opt/stack/new/neutron/neutron/db/api.py", line 77, in wrapped 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/opt/stack/new/neutron/neutron/api/rpc/handlers/dhcp_rpc.py", line 274, in update_dhcp_port 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server return self._port_action(plugin, context, port, 'update_port') 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/opt/stack/new/neutron/neutron/api/rpc/handlers/dhcp_rpc.py", line 100, in _port_action 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server return plugin.update_port(context, port['id'], port) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/opt/stack/new/neutron/neutron/common/utils.py", line 618, in inner 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kwargs) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1364, in update_port 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server raise exc.PortNotFound(port_id=id) 2016-09-08 07:23:35.069 13627 ERROR oslo_messaging.rpc.server PortNotFound: Port 3671e6b0-b969-4949-a486-d708e43aa60c could not be found. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1621650/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
