I actually broke this with my update, I'll explain. Although we do correctly disable accept_ra in the dhcp namespace now, on the grenade upgrade job we are restarting a new dhcp agent on an existing installation. This means that the old dhcp namespace could have auto-configured an IPv6 address already, in which case we fall on our face:
'ip', 'netns', 'exec', 'qdhcp-38b38c7e-d336-4955-a140-ef83a4410c2a', 'ip', '-6', 'addr', 'add', 'fdb9:d744:18da:0:f816:3eff:fe61:c32b/64', 'scope', 'global', 'dev', 'tap62476185-f1' ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: File exists http://logs.openstack.org/43/406243/1/check/gate-grenade-dsvm-neutron- dvr-multinode-ubuntu- xenial/6cee38c/logs/new/screen-q-dhcp.txt.gz#_2016-12-02_17_40_30_861 Looking a few lines up we can see: Reusing existing device: tap8fb1bc28-fa That's the clue that we might have a conflict. In init_l3() we filter the IP(v6) addresses with the "permanent" flag, but that will filter any previous IPv6 addresses created via SLAAC on receipt of an RA. Just dropping the "permanent" filter seems like a quick fix, but that will leave the SLAAC-based address alone, which will eventually be removed as it's lifetime expires, leading to other problems. The best thing to do would be to detect the old address (it has a "dynamic" flag) and remove it, allowing us to correctly add the new "permanent" one. I'll work on a fix. ** Changed in: neutron Status: Fix Released => Confirmed -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1627902 Title: DHCP agent conflicting with dynamic IPv6 addresses Status in neutron: Confirmed Bug description: Below in gate logs. Completely breaks the DHCP for that network because it's trying to add an address that conflicts with one given to it via RA. Cause is the merge of d86f1b87f01c53c3e0b085086133b311e5bf3ab5 which allowed the agent to be configured with stateless v6 addresses to serve metadata correctly. http://logs.openstack.org/12/343312/5/gate/gate-tempest-dsvm-neutron- full-ubuntu- xenial/c11b933/logs/screen-q-dhcp.txt.gz?level=TRACE#_2016-09-26_21_45_40_604 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.linux.utils [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: File exists 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent [-] Unable to enable dhcp for 81d252a2-8207-4e8c-a286-07fb3494a3ec. 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent Traceback (most recent call last): 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent File "/opt/stack/new/neutron/neutron/agent/dhcp/agent.py", line 114, in call_driver 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent getattr(driver, action)(**action_kwargs) 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 212, in enable 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent interface_name = self.device_manager.setup(self.network) 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 1396, in setup 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent namespace=network.namespace) 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent File "/opt/stack/new/neutron/neutron/agent/linux/interface.py", line 129, in init_l3 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent device.addr.add(ip_cidr) 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 577, in add 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent self._as_root([net.version], tuple(args)) 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 364, in _as_root 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent use_root_namespace=use_root_namespace) 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 95, in _as_root 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent log_fail_as_error=self.log_fail_as_error) 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 104, in _execute 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent log_fail_as_error=log_fail_as_error) 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent File "/opt/stack/new/neutron/neutron/agent/linux/utils.py", line 138, in execute 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent raise RuntimeError(msg) 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent RuntimeError: Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: File exists 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent 2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1627902/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
