Public bug reported: I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall.
I run into situations, where, the iptables SG chains/rules are inconsistent with ovs-ports, and system interfaces - see below for an example. In these situations, when I restart neutron-ovs-agent, I expect that such inconsistencies are cleaned up. Arguably, agent should not allow such situations to happen (but thats expecting code to be perfect) By design, the neutron-ovs-agent must audit hw-resources (iptables in this case) during startup , and cleanup inconsistencies/deltas between whats in iptables vs whats in the control plane (ports) The audit can look something like this: 1. IptablesManager recovers iptables chains during startup and marks all of them as "To Be Audited". 2. NeutronOvsAgent programs firewall rules for devices during startup 3. As chains are programmed, Iptables clears the "to be audited" state 4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed. ----- Inconsistency between OVS ports, System Interfaces and Iptables [root@rhel7-25 agent]# iptables -S -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N neutron-filter-top -N neutron-openvswi-FORWARD -N neutron-openvswi-INPUT -N neutron-openvswi-OUTPUT -N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage -N neutron-openvswi-ie3cb2e38-a -N neutron-openvswi-local -N neutron-openvswi-o76845da1-5 -N neutron-openvswi-oe3cb2e38-a -N neutron-openvswi-s76845da1-5 -N neutron-openvswi-se3cb2e38-a -N neutron-openvswi-sg-chain -N neutron-openvswi-sg-fallback [root@rhel7-25 agent]# ovs-vsctl show ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697 Bridge br-int fail_mode: secure Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port br-int Interface br-int type: internal Bridge br-tun fail_mode: secure Port "vxlan-0a0a0a1e" Interface "vxlan-0a0a0a1e" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"} Port "vxlan-0a0a0a17" Interface "vxlan-0a0a0a17" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"} Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "vxlan-0a0a0a15" Interface "vxlan-0a0a0a15" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"} Port br-tun Interface br-tun type: internal ovs_version: "2.3.0" [root@rhel7-25 agent]# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000 link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff 4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff 5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff 6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff 7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff 8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000 link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff 29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff 29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff 20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff ** Affects: neutron Importance: Undecided Status: New ** Description changed: - I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall. - I run into situations, where, the iptables SG chains/rules are inconsistent with ovs-ports see below for an example. - In these situations, when I restart neutron-ovs-agent, I expect that such inconsistencies are cleaned up. - Arguably, such situations should never arise (but thats expecting code to be perfect) + I run into situations, where, the iptables SG chains/rules are + inconsistent with ovs-ports, and system interfaces - see below for an + example. In these situations, when I restart neutron-ovs-agent, I expect + that such inconsistencies are cleaned up. Arguably, such situations + should never arise (but thats expecting code to be perfect) By design, the neutron-ovs-agent must audit hw-resources (iptables in this case) during startup , and cleanup inconsistencies/deltas. The audit can look something like this: - 1. IptablesManager recovers iptables chains during startup and marks all them as "To Be Audited". - 2. NeutronOvsAgent programs firewall rules for devices during startup - 3. As chains are programmed, Iptables clears the "to be audited" state - 4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed. + 1. IptablesManager recovers iptables chains during startup and marks all them as "To Be Audited". + 2. NeutronOvsAgent programs firewall rules for devices during startup + 3. As chains are programmed, Iptables clears the "to be audited" state + 4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed. ----- Inconsistency between OVS ports, System Interfaces and Iptables [root@rhel7-25 agent]# iptables -S -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N neutron-filter-top -N neutron-openvswi-FORWARD -N neutron-openvswi-INPUT -N neutron-openvswi-OUTPUT -N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage -N neutron-openvswi-ie3cb2e38-a -N neutron-openvswi-local -N neutron-openvswi-o76845da1-5 -N neutron-openvswi-oe3cb2e38-a -N neutron-openvswi-s76845da1-5 -N neutron-openvswi-se3cb2e38-a -N neutron-openvswi-sg-chain -N neutron-openvswi-sg-fallback - [root@rhel7-25 agent]# ovs-vsctl show ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697 - Bridge br-int - fail_mode: secure - Port patch-tun - Interface patch-tun - type: patch - options: {peer=patch-int} - Port br-int - Interface br-int - type: internal - Bridge br-tun - fail_mode: secure - Port "vxlan-0a0a0a1e" - Interface "vxlan-0a0a0a1e" - type: vxlan - options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"} - Port "vxlan-0a0a0a17" - Interface "vxlan-0a0a0a17" - type: vxlan - options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"} - Port patch-int - Interface patch-int - type: patch - options: {peer=patch-tun} - Port "vxlan-0a0a0a15" - Interface "vxlan-0a0a0a15" - type: vxlan - options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"} - Port br-tun - Interface br-tun - type: internal - ovs_version: "2.3.0" + Bridge br-int + fail_mode: secure + Port patch-tun + Interface patch-tun + type: patch + options: {peer=patch-int} + Port br-int + Interface br-int + type: internal + Bridge br-tun + fail_mode: secure + Port "vxlan-0a0a0a1e" + Interface "vxlan-0a0a0a1e" + type: vxlan + options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"} + Port "vxlan-0a0a0a17" + Interface "vxlan-0a0a0a17" + type: vxlan + options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"} + Port patch-int + Interface patch-int + type: patch + options: {peer=patch-tun} + Port "vxlan-0a0a0a15" + Interface "vxlan-0a0a0a15" + type: vxlan + options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"} + Port br-tun + Interface br-tun + type: internal + ovs_version: "2.3.0" [root@rhel7-25 agent]# ip link - 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000 - link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff + link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 - link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff + link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff 4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 - link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff + link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff 5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 - link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff + link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff 6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 - link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff + link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff 7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 - link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff + link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff 8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000 - link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff - 29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT - link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff - 29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT - link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff - 20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT - link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff + link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff + 29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT + link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff + 29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT + link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff + 20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT + link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff ** Description changed: I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall. I run into situations, where, the iptables SG chains/rules are inconsistent with ovs-ports, and system interfaces - see below for an example. In these situations, when I restart neutron-ovs-agent, I expect that such inconsistencies are cleaned up. Arguably, such situations should never arise (but thats expecting code to be perfect) By design, the neutron-ovs-agent must audit hw-resources (iptables in - this case) during startup , and cleanup inconsistencies/deltas. + this case) during startup , and cleanup inconsistencies/deltas between + whats in iptables vs whats in the control plane (ports) The audit can look something like this: 1. IptablesManager recovers iptables chains during startup and marks all them as "To Be Audited". 2. NeutronOvsAgent programs firewall rules for devices during startup 3. As chains are programmed, Iptables clears the "to be audited" state 4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed. ----- Inconsistency between OVS ports, System Interfaces and Iptables [root@rhel7-25 agent]# iptables -S -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N neutron-filter-top -N neutron-openvswi-FORWARD -N neutron-openvswi-INPUT -N neutron-openvswi-OUTPUT -N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage -N neutron-openvswi-ie3cb2e38-a -N neutron-openvswi-local -N neutron-openvswi-o76845da1-5 -N neutron-openvswi-oe3cb2e38-a -N neutron-openvswi-s76845da1-5 -N neutron-openvswi-se3cb2e38-a -N neutron-openvswi-sg-chain -N neutron-openvswi-sg-fallback [root@rhel7-25 agent]# ovs-vsctl show ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697 Bridge br-int fail_mode: secure Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port br-int Interface br-int type: internal Bridge br-tun fail_mode: secure Port "vxlan-0a0a0a1e" Interface "vxlan-0a0a0a1e" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"} Port "vxlan-0a0a0a17" Interface "vxlan-0a0a0a17" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"} Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "vxlan-0a0a0a15" Interface "vxlan-0a0a0a15" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"} Port br-tun Interface br-tun type: internal ovs_version: "2.3.0" [root@rhel7-25 agent]# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000 link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff 4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff 5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff 6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff 7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff 8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000 link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff 29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff 29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff 20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff ** Description changed: I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall. I run into situations, where, the iptables SG chains/rules are inconsistent with ovs-ports, and system interfaces - see below for an example. In these situations, when I restart neutron-ovs-agent, I expect that such inconsistencies are cleaned up. Arguably, such situations should never arise (but thats expecting code to be perfect) By design, the neutron-ovs-agent must audit hw-resources (iptables in this case) during startup , and cleanup inconsistencies/deltas between whats in iptables vs whats in the control plane (ports) The audit can look something like this: - 1. IptablesManager recovers iptables chains during startup and marks all them as "To Be Audited". + 1. IptablesManager recovers iptables chains during startup and marks all of them as "To Be Audited". 2. NeutronOvsAgent programs firewall rules for devices during startup 3. As chains are programmed, Iptables clears the "to be audited" state 4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed. ----- Inconsistency between OVS ports, System Interfaces and Iptables [root@rhel7-25 agent]# iptables -S -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N neutron-filter-top -N neutron-openvswi-FORWARD -N neutron-openvswi-INPUT -N neutron-openvswi-OUTPUT -N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage -N neutron-openvswi-ie3cb2e38-a -N neutron-openvswi-local -N neutron-openvswi-o76845da1-5 -N neutron-openvswi-oe3cb2e38-a -N neutron-openvswi-s76845da1-5 -N neutron-openvswi-se3cb2e38-a -N neutron-openvswi-sg-chain -N neutron-openvswi-sg-fallback [root@rhel7-25 agent]# ovs-vsctl show ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697 Bridge br-int fail_mode: secure Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port br-int Interface br-int type: internal Bridge br-tun fail_mode: secure Port "vxlan-0a0a0a1e" Interface "vxlan-0a0a0a1e" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"} Port "vxlan-0a0a0a17" Interface "vxlan-0a0a0a17" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"} Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "vxlan-0a0a0a15" Interface "vxlan-0a0a0a15" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"} Port br-tun Interface br-tun type: internal ovs_version: "2.3.0" [root@rhel7-25 agent]# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000 link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff 4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff 5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff 6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff 7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff 8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000 link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff 29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff 29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff 20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1494039 Title: Must audit SG chains on ovs-agent restart Status in neutron: New Bug description: I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall. I run into situations, where, the iptables SG chains/rules are inconsistent with ovs-ports, and system interfaces - see below for an example. In these situations, when I restart neutron-ovs-agent, I expect that such inconsistencies are cleaned up. Arguably, agent should not allow such situations to happen (but thats expecting code to be perfect) By design, the neutron-ovs-agent must audit hw-resources (iptables in this case) during startup , and cleanup inconsistencies/deltas between whats in iptables vs whats in the control plane (ports) The audit can look something like this: 1. IptablesManager recovers iptables chains during startup and marks all of them as "To Be Audited". 2. NeutronOvsAgent programs firewall rules for devices during startup 3. As chains are programmed, Iptables clears the "to be audited" state 4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed. ----- Inconsistency between OVS ports, System Interfaces and Iptables [root@rhel7-25 agent]# iptables -S -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N neutron-filter-top -N neutron-openvswi-FORWARD -N neutron-openvswi-INPUT -N neutron-openvswi-OUTPUT -N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage -N neutron-openvswi-ie3cb2e38-a -N neutron-openvswi-local -N neutron-openvswi-o76845da1-5 -N neutron-openvswi-oe3cb2e38-a -N neutron-openvswi-s76845da1-5 -N neutron-openvswi-se3cb2e38-a -N neutron-openvswi-sg-chain -N neutron-openvswi-sg-fallback [root@rhel7-25 agent]# ovs-vsctl show ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697 Bridge br-int fail_mode: secure Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port br-int Interface br-int type: internal Bridge br-tun fail_mode: secure Port "vxlan-0a0a0a1e" Interface "vxlan-0a0a0a1e" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"} Port "vxlan-0a0a0a17" Interface "vxlan-0a0a0a17" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"} Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "vxlan-0a0a0a15" Interface "vxlan-0a0a0a15" type: vxlan options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"} Port br-tun Interface br-tun type: internal ovs_version: "2.3.0" [root@rhel7-25 agent]# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000 link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff 4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff 5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff 6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff 7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000 link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff 8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000 link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff 29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff 29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff 20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1494039/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp