Public bug reported: Hello,
I've been playing with Rocky on CentOS 7 with DVR / DVR-SNAT / BGP. Our provider network uses private IP-space for scalability. Our tenants run in public IP-space. Steps: openstack network create --share --provider-network-type vlan --external --provider-physical-network EXTERNAL NET-EXT-VLAN625-UPLINK --provider-segment 625 openstack subnet create --network NET-EXT-VLAN625-UPLINK --subnet-range 192.0.2.0/24 SUBNET-EXT-VLAN625-UPLINK openstack bgp speaker add network BGP-REFLECTOR-SHARED NET-EXT-VLAN625-UPLINK openstack address scope create --share --ip-version 4 SCOPE-SHARED-VFOUR openstack subnet pool create --pool-prefix 93.115.169.128/25 --address-scope SCOPE-SHARED-VFOUR POOL-SHARED-EXTERNAL openstack subnet pool set --default-prefix-length 28 --min-prefix-length 26 POOL-SHARED-EXTERNAL openstack network create --share NET-INT-PUBLIC-SHARED openstack subnet create --network NET-INT-PUBLIC-SHARED --subnet-pool POOL-SHARED-EXTERNAL --prefix-length 28 SUBNET-INT-PUBLIC-SHARED openstack bgp speaker add network BGP-REFLECTOR-SHARED NET-INT-PUBLIC-SHARED openstack router create ROUTER-SHARED openstack router set ROUTER-SHARED --disable-snat --external-gateway NET-EXT-VLAN625-UPLINK openstack router add subnet ROUTER-SHARED SUBNET-INT-PUBLIC-SHARED openstack bgp speaker list advertised routes BGP-REFLECTOR-SHARED Result: +-------------------+-----------+ | Destination | Nexthop | +-------------------+-----------+ | 93.115.169.128/28 | 192.0.2.6 | +-------------------+-----------+ Where 192.0.2.6 is the IP in the SNAT instance on the DVR-SNAT network node. So far things seem good however it seems my egress traffic is blocked. I'm not 100% sure it's not caused by misconfiguration but I found the two lines to get traffic flowing in my config: https://github.com/openstack/neutron/blob/stable/rocky/neutron/agent/l3/router_info.py#L1093 was rewritten to "dont_block_external = (ip_version == lib_constants.IP_VERSION_4 and external_port)" -> This would allow traffic to go egress into the provider network https://github.com/openstack/neutron/blob/stable/rocky/neutron/agent/l3/router_info.py#L1122 if-return statement was commented out. -> I need that connmark/mark restore rule to have ingress replies match the existing connections. Validate: ip netns exec $(ip netns | grep snat | awk '{ print $1}') iptables -t mangle -L neutron-l3-agent-POSTROUTING -nv Chain neutron-l3-agent-POSTROUTING (1 references) pkts bytes target prot opt in out source destination 508 27174 CONNMARK all -- * qg-b075d908-66 0.0.0.0/0 0.0.0.0/0 connmark match 0x0/0xffff0000 CONNMARK save mask 0xffff0000 1 entry ip netns exec $(ip netns | grep snat | awk '{ print $1}') iptables -t filter -L neutron-l3-agent-scope -nv Chain neutron-l3-agent-scope (1 references) pkts bytes target prot opt in out source destination 6588 350K DROP all -- * sg-c46c9df8-06 0.0.0.0/0 0.0.0.0/0 mark match ! 0x4010000/0xffff0000 1 entry (instead of two before) ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1839004 Title: Rocky DVR-SNAT seems missing entries for conntrack marking Status in neutron: New Bug description: Hello, I've been playing with Rocky on CentOS 7 with DVR / DVR-SNAT / BGP. Our provider network uses private IP-space for scalability. Our tenants run in public IP-space. Steps: openstack network create --share --provider-network-type vlan --external --provider-physical-network EXTERNAL NET-EXT-VLAN625-UPLINK --provider-segment 625 openstack subnet create --network NET-EXT-VLAN625-UPLINK --subnet-range 192.0.2.0/24 SUBNET-EXT-VLAN625-UPLINK openstack bgp speaker add network BGP-REFLECTOR-SHARED NET-EXT-VLAN625-UPLINK openstack address scope create --share --ip-version 4 SCOPE-SHARED-VFOUR openstack subnet pool create --pool-prefix 93.115.169.128/25 --address-scope SCOPE-SHARED-VFOUR POOL-SHARED-EXTERNAL openstack subnet pool set --default-prefix-length 28 --min-prefix-length 26 POOL-SHARED-EXTERNAL openstack network create --share NET-INT-PUBLIC-SHARED openstack subnet create --network NET-INT-PUBLIC-SHARED --subnet-pool POOL-SHARED-EXTERNAL --prefix-length 28 SUBNET-INT-PUBLIC-SHARED openstack bgp speaker add network BGP-REFLECTOR-SHARED NET-INT-PUBLIC-SHARED openstack router create ROUTER-SHARED openstack router set ROUTER-SHARED --disable-snat --external-gateway NET-EXT-VLAN625-UPLINK openstack router add subnet ROUTER-SHARED SUBNET-INT-PUBLIC-SHARED openstack bgp speaker list advertised routes BGP-REFLECTOR-SHARED Result: +-------------------+-----------+ | Destination | Nexthop | +-------------------+-----------+ | 93.115.169.128/28 | 192.0.2.6 | +-------------------+-----------+ Where 192.0.2.6 is the IP in the SNAT instance on the DVR-SNAT network node. So far things seem good however it seems my egress traffic is blocked. I'm not 100% sure it's not caused by misconfiguration but I found the two lines to get traffic flowing in my config: https://github.com/openstack/neutron/blob/stable/rocky/neutron/agent/l3/router_info.py#L1093 was rewritten to "dont_block_external = (ip_version == lib_constants.IP_VERSION_4 and external_port)" -> This would allow traffic to go egress into the provider network https://github.com/openstack/neutron/blob/stable/rocky/neutron/agent/l3/router_info.py#L1122 if-return statement was commented out. -> I need that connmark/mark restore rule to have ingress replies match the existing connections. Validate: ip netns exec $(ip netns | grep snat | awk '{ print $1}') iptables -t mangle -L neutron-l3-agent-POSTROUTING -nv Chain neutron-l3-agent-POSTROUTING (1 references) pkts bytes target prot opt in out source destination 508 27174 CONNMARK all -- * qg-b075d908-66 0.0.0.0/0 0.0.0.0/0 connmark match 0x0/0xffff0000 CONNMARK save mask 0xffff0000 1 entry ip netns exec $(ip netns | grep snat | awk '{ print $1}') iptables -t filter -L neutron-l3-agent-scope -nv Chain neutron-l3-agent-scope (1 references) pkts bytes target prot opt in out source destination 6588 350K DROP all -- * sg-c46c9df8-06 0.0.0.0/0 0.0.0.0/0 mark match ! 0x4010000/0xffff0000 1 entry (instead of two before) To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1839004/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
