Hello Jake: This is a duplicate of 1899141.
Please check you code has https://review.opendev.org/#/q/I55a0f4341b328af52ea3bd758a72f633fbe3abcb Regards. ** Changed in: neutron Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1903835 Title: linuxbridge-agent ERRORS with pyroute2.netlink.exceptions.NetlinkError if ipv6 link local address exist Status in neutron: Invalid Bug description: Hi all. We are running neutron train. After a minor version upgrade recently, linuxbridge-agent started throwing errors like this: 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent [req-947d5add-cdd0-437d-a2eb-1c17d2556cd3 - - - - -] Error in agent loop. Devices info: {'current': {'tap25db63b7-9b', 'tapa6658716-0e', 'tapeae0251d-48', 'tapfeeae322-76', 'tap2dcf0e81-2d', 'tapd908490e-6f', 'tap4208ea46-9f', 'tape4a832ef 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 465, in daemon_loop 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent sync = self.process_network_devices(device_info) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/train/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent result = f(*args, **kwargs) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 214, in process_network_devices 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent resync_a = self.treat_devices_added_updated(devices_added_updated) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/train/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent result = f(*args, **kwargs) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 231, in treat_devices_added_updated 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent self._process_device_if_exists(device_details) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 258, in _process_device_if_exists 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent device, device_details['device_owner']) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 588, in plug_interface 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent network_segment.mtu) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 524, in add_tap_interface 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent return False 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/train/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent self.force_reraise() 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/train/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent six.reraise(self.type_, self.value, self.tb) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/train/lib/python3.6/site-packages/six.py", line 693, in reraise 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise value 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 516, in add_tap_interface 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent tap_device_name, device_owner, mtu) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 549, in _add_tap_interface 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent mtu): 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 500, in ensure_physical_in_bridge 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent physical_interface) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 288, in ensure_flat_bridge 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent if self.ensure_bridge(bridge_name, physical_interface): 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 458, in ensure_bridge 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent self.update_interface_ip_details(bridge_name, interface) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 420, in update_interface_ip_details 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent gateway) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 396, in _update_interface_ip_details 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent dst_device.addr.add(cidr=ip['cidr']) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/agent/linux/ip_lib.py", line 528, in add 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent add_broadcast) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/agent/linux/ip_lib.py", line 802, in add_ip_address 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent device, namespace, scope, broadcast) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/neutron/neutron/privileged/agent/linux/ip_lib.py", line 73, in sync_inner 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent return input_func(*args, **kwargs) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/train/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 245, in _wrap 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent return self.channel.remote_call(name, args, kwargs) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/train/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 224, in remote_call 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise exc_type(*result[2]) 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent pyroute2.netlink.exceptions.NetlinkError: (13, 'Permission denied') 2020-11-11 21:13:50.524 6088 ERROR neutron.plugins.ml2.drivers.agent._common_agent I think I've traced it down to the (backported) change https://review.opendev.org/#/c/747627/ It happens because the linuxbridge-agent interface has a link-local ipv6 address. Prior to this patch, linuxbridge-agent works fine. With this patch, it errors. Removing the link-local ipv6 address will solve the problem. It may be the correct way is to not have ipv6 link-local addresses on linuxbridge interface; we are working towards removing them. Just thought that I might put this up as a bug, in case operators and developers run into the same problem. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1903835/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
