Reviewed: https://review.openstack.org/523319 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=37bd42e4f5d1be49689032822aca339523cfda33 Submitter: Zuul Branch: master
commit 37bd42e4f5d1be49689032822aca339523cfda33 Author: Jens Harbott <[email protected]> Date: Tue Nov 28 07:39:04 2017 +0000 Fix error when using protocol number in security groups When the support of protocol numbers in security groups was fixed in [1], it introduced two deficiencies in the iptables code: - it was missing some protocols, for example, 'icmp', 'tcp' and 'udp', so when rules were added by number we did not use their name as iptables expects - it used a dictionary to map numbers to names, but protocol numbers are stored as strings (i.e. '1' != 1) Updated the iptables number mapping dict to have all currently-known values, even those that are already well-known and should have been using a string instead of a number. Also changed the iptables number mapping dict to use strings as the keys instead of numbers, since that's what will be passed from the security group code. Removed IPTABLES_PROTOCOL_MAP as it lives in neutron-lib, and accidentally snuck-in in [1]. [1] I5895250b47ddf664d214cf085be693c3897e0c87 Change-Id: I6b7575eb531b4f35579960c3feb47000cd259b86 Closes-Bug: 1719711 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1719711 Title: iptables failed to apply when binding a port with AGENT.debug_iptables_rules enabled Status in neutron: Fix Released Bug description: http://logs.openstack.org/21/504021/2/check/gate-tempest-dsvm-neutron- scenario-linuxbridge-ubuntu-xenial-nv/e47a3f3/testr_results.html.gz Traceback (most recent call last): File "/opt/stack/new/neutron/neutron/tests/tempest/scenario/test_security_groups.py", line 127, in test_two_sec_groups num_servers=1, security_groups=security_groups_list) File "/opt/stack/new/neutron/neutron/tests/tempest/scenario/test_security_groups.py", line 54, in create_vm_testing_sec_grp const.SERVER_STATUS_ACTIVE) File "tempest/common/waiters.py", line 76, in wait_for_server_status server_id=server_id) tempest.exceptions.BuildErrorException: Server e1120d99-f0eb-43eb-a38b-847843a838b5 failed to build and is in ERROR status Details: {u'message': u'Build of instance e1120d99-f0eb-43eb-a38b-847843a838b5 aborted: Failed to allocate the network(s), not rescheduling.', u'code': 500, u'created': u'2017-09-26T09:23:42Z'} In linuxbridge agent log: http://logs.openstack.org/21/504021/2/check /gate-tempest-dsvm-neutron-scenario-linuxbridge-ubuntu-xenial- nv/e47a3f3/logs/screen-q-agt.txt.gz?level=TRACE#_Sep_26_09_16_30_623747 Sep 26 09:16:30.623747 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.agent.linux.iptables_manager [None req-78fc6bc1-a089-4d5f-91d8-e5191e45978c None None] IPTables Rules did not converge. Diff: # Generated by iptables_manager Sep 26 09:16:30.623936 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: *filter Sep 26 09:16:30.624117 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -D neutron-linuxbri-ibc1a22b9-e 6 Sep 26 09:16:30.624316 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -I neutron-linuxbri-ibc1a22b9-e 6 -p 1 -j RETURN Sep 26 09:16:30.624482 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: COMMIT Sep 26 09:16:30.624955 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: # Completed by iptables_manager Sep 26 09:16:30.635308 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent [None req-78fc6bc1-a089-4d5f-91d8-e5191e45978c None None] Error in agent loop. Devices info: {'current': set(['tapbc1a22b9-ef', 'tapc9488f0f-ae', 'tape2d2e245-96', 'tap93881b27-41', 'tapb265ee77-37', 'tapbadc6b64-69', 'tapa813220a-1d', 'tapa376782a-75', 'tap395ccf4d-c9', 'tapca94a412-e7', 'tap58f740f2-aa', 'tapb2444941-9f']), 'timestamps': {'tap93881b27-41': 56, 'tapc9488f0f-ae': 62, 'tape2d2e245-96': 11, 'tapbc1a22b9-ef': 68, 'tapb265ee77-37': 9, 'tapbadc6b64-69': 55, 'tapa813220a-1d': 66, 'tapa376782a-75': 65, 'tap395ccf4d-c9': 67, 'tapca94a412-e7': 6, 'tap58f740f2-aa': 59, 'tapb2444941-9f': 10}, 'removed': set([]), 'added': set([]), 'updated': set([])}: IpTablesApplyException: IPTables Rules did not converge. Diff: # Generated by iptables_manager Sep 26 09:16:30.636316 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: *filter Sep 26 09:16:30.636510 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -D neutron-linuxbri-ibc1a22b9-e 6 Sep 26 09:16:30.636700 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -I neutron-linuxbri-ibc1a22b9-e 6 -p 1 -j RETURN Sep 26 09:16:30.636898 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: COMMIT Sep 26 09:16:30.637075 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: # Completed by iptables_manager Sep 26 09:16:30.637269 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): Sep 26 09:16:30.637683 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 453, in daemon_loop Sep 26 09:16:30.637962 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent sync = self.process_network_devices(device_info) Sep 26 09:16:30.638211 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 157, in wrapper Sep 26 09:16:30.638373 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent result = f(*args, **kwargs) Sep 26 09:16:30.638538 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 200, in process_network_devices Sep 26 09:16:30.638728 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent device_info.get('updated')) Sep 26 09:16:30.639034 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 256, in setup_port_filters Sep 26 09:16:30.639220 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent self.refresh_firewall(updated_devices) Sep 26 09:16:30.639702 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 110, in decorated_function Sep 26 09:16:30.639993 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent *args, **kwargs) Sep 26 09:16:30.640390 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 209, in refresh_firewall Sep 26 09:16:30.640671 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent self._apply_port_filter(device_ids, update_filter=True) Sep 26 09:16:30.640925 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 145, in _apply_port_filter Sep 26 09:16:30.641159 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent self.firewall.process_trusted_ports(trusted_devices) Sep 26 09:16:30.641466 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ Sep 26 09:16:30.641926 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent self.gen.next() Sep 26 09:16:30.642208 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/firewall.py", line 145, in defer_apply Sep 26 09:16:30.642454 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent self.filter_defer_apply_off() Sep 26 09:16:30.642701 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/linux/iptables_firewall.py", line 852, in filter_defer_apply_off Sep 26 09:16:30.642990 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent self.iptables.defer_apply_off() Sep 26 09:16:30.643349 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 429, in defer_apply_off Sep 26 09:16:30.643647 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent self._apply() Sep 26 09:16:30.643920 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 454, in _apply Sep 26 09:16:30.644170 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent raise n_exc.IpTablesApplyException(msg) Sep 26 09:16:30.644519 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent IpTablesApplyException: IPTables Rules did not converge. Diff: # Generated by iptables_manager Sep 26 09:16:30.644842 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent *filter Sep 26 09:16:30.645208 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent -D neutron-linuxbri-ibc1a22b9-e 6 Sep 26 09:16:30.645479 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent -I neutron-linuxbri-ibc1a22b9-e 6 -p 1 -j RETURN Sep 26 09:16:30.645847 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent COMMIT Sep 26 09:16:30.646182 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent # Completed by iptables_manager Sep 26 09:16:30.646457 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent This happens in scenario job when iptables manager is used. Despite the fact that it doesn't happen in ovs flavor of the job, it's not clear whether it affects ovs setups using iptables, because ovs scenario job uses 'openvswitch' flow based firewall driver instead of iptables. This happens on a patch that adds a new scenario test case targeting security groups, so may be related: https://review.openstack.org/#/c/504021/ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1719711/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
