Reviewed: https://review.openstack.org/472454 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3c1a25d9683263505b00cb0d6b04add1831f0ca0 Submitter: Jenkins Branch: master
commit 3c1a25d9683263505b00cb0d6b04add1831f0ca0 Author: Kevin Benton <[email protected]> Date: Thu Jun 8 14:56:19 2017 -0700 Make HA deletion attempt on RouterNotFound race The L3 HA RPC code that creates HA interfaces can race with an HA router deletion on the server side. The L3 HA code ends up creating a port on the HA network while the server side is deleting the router and the HA network. This stops the L3 HA network from being deleted because it has a new port without a bound segment, which leaves the HA network in a segmentless condition and no ports after the L3 RPC code cleans up its port. This adjusts the L3 RPC logic to attempt an HA network cleanup whenever it encounters the concurrent router deletion case to ensure that the HA network gets cleaned up. To make this more robust in the future, we may need the L3 HA code to recognize when an HA network has no segments and automatically create a new one. Change-Id: Idd301f6df92e9bc37187e8ed8ec00004e67da928 Closes-Bug: #1696537 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1696537 Title: test_keepalived_multiple_sighups_does_not_forfeit_mastership fails when neutron-server tries to bind with Linuxbridge driver (agent not enabled) Status in neutron: Fix Released Bug description: This happens locally and in gate. Gate example: http://logs.openstack.org/59/471059/2/check/gate-neutron-dsvm- fullstack-ubuntu-xenial/df11b90/testr_results.html.gz Traceback (most recent call last): File "neutron/tests/base.py", line 118, in func return f(self, *args, **kwargs) File "neutron/tests/fullstack/test_l3_agent.py", line 252, in test_keepalived_multiple_sighups_does_not_forfeit_mastership tenant_id, '13.37.0.0/24', network['id'], router['id']) File "neutron/tests/fullstack/test_l3_agent.py", line 61, in _create_and_attach_subnet router_interface_info['port_id']) File "neutron/tests/fullstack/test_l3_agent.py", line 51, in block_until_port_status_active common_utils.wait_until_true(lambda: is_port_status_active(), sleep=1) File "neutron/common/utils.py", line 685, in wait_until_true raise WaitTimeout("Timed out after %d seconds" % timeout) neutron.common.utils.WaitTimeout: Timed out after 60 seconds This is not 100% failure rate, depends on which driver server picks to bind ports: ovs or linuxbridge. If the latter, it just spins attempting to bind with it over and over, until bails out. It never tries to switch to ovs. In server log, we see this: http://logs.openstack.org/59/471059/2/check/gate-neutron-dsvm- fullstack-ubuntu-xenial/df11b90/logs/dsvm-fullstack- logs/TestHAL3Agent.test_keepalived_multiple_sighups_does_not_forfeit_mastership /neutron-server--2017-06-05-- 21-41-34-957535.txt.gz#_2017-06-05_21_42_13_400 2017-06-05 21:42:13.400 12566 DEBUG neutron.plugins.ml2.drivers.mech_agent [req-6618e950-5260-404d-a511-e314408542f5 - - - - -] Port 4f8dcf10-6f91-4860-b239-6b04460244a3 on network 155ebfd5-20cf-44bc-9cb5-bc885b8d2eae not bound, no agent of type Linux bridge agent registered on host host-745fd526 bind_port /opt/stack/new/neutron/neutron/plugins/ml2/drivers/mech_agent.py:103 2017-06-05 21:42:13.401 12566 ERROR neutron.plugins.ml2.managers [req-6618e950-5260-404d-a511-e314408542f5 - - - - -] Failed to bind port 4f8dcf10-6f91-4860-b239-6b04460244a3 on host host-745fd526 for vnic_type normal using segments [] 2017-06-05 21:42:13.401 12566 INFO neutron.plugins.ml2.plugin [req-6618e950-5260-404d-a511-e314408542f5 - - - - -] Attempt 2 to bind port 4f8dcf10-6f91-4860-b239-6b04460244a3 ... 2017-06-05 21:42:13.822 12566 ERROR neutron.plugins.ml2.managers [req-6618e950-5260-404d-a511-e314408542f5 - - - - -] Failed to bind port 4f8dcf10-6f91-4860-b239-6b04460244a3 on host host-745fd526 for vnic_type normal using segments [] The fullstack test case configures both ml2 drivers. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1696537/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
