Re: [ovs-discuss] [ovn] OVN asymmetric routing with conntrack
On Tue, Aug 24, 2021 at 3:02 PM Vladislav Odintsov wrote: > > > Regards, > Vladislav Odintsov > > On 25 Aug 2021, at 00:47, Numan Siddique wrote: > > On Mon, Aug 23, 2021 at 11:22 AM Vladislav Odintsov > wrote: > > > Hi, > > we’ve faced an issue where asymmetric-routed traffic is used. Please help > understand what options do we have to allow such traffic. > > Topology is next: > > client lsp (10.0.0.1/24) > | >ls-external >/ \ > lsp router vm1 eth0: 10.0.0.2/24 lsp router vm2 eth0: 10.0.0.3/24 > lsp router vm1 eth1: 192.168.0.1/24 lsp router vm2 eth1: 192.168.0.2/24 >\ / >ls-internal > | >server lsp (192.168.0.10/24) > > > All LSPs have port_security configured with " 0.0.0.0/0 ::/0" and > belong to port group pg1. > > There are two ACLs within this PG: > from-lport 0.0.0.0/0 allow-related > to-lport 0.0.0.0/0 allow-related > > The problem is when traffic from client to server goes through router vm1 > and returns through router vm2, there is no connectivity. I see reply > traffic on the server interface, which is going to router vm2 mac address, > but I don't see it on the router vm2 interface. > I guess the reason for this is that conntrack first time sees packet for > the connection and ACK+SYN flags are set and treats this packet as invalid, > right? > > > I think so. > > > If yes, is there any option how to use asymmetric-routed topologies inside > OVN with stateful ACLs? > I found there is an ability to replace ct.inv field check: > https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4 > Is it good idea to use this option to solve the issue or this is intended > specifically to use with smart NICs without invalid state support and can > be removed in future? > > I am afraid disabling ct.inv will not help here. In your use case the connections won't become established in conntrack, which means stateful ACL wouldn't work. This will be the case even with physical stateful FWs. I'd suggest either disable ACLs or use stateless ACLs (i.e. allow/allow-stateless instead of allow-related). > > I do not understand your use case completely. I'm not quite clear > from the diagram which all resources are external > and which all are part of OVN. Have you tried using the ECMP routes > feature ? > > Logical Routers are not used in this topology. Only two logical switches - > ls-external and ls-internal. > > In OVN we have: > 1. LS "ls-external" with 3 LSPs: "lsp-router-vm1-eth0" (10.0.0.2/24), > "lsp-router-vm2-eth0" (10.0.0.3/24) and "lsp-client" (10.0.0.1/24) > 2. LS "ls-internal" with 2 LSPs: "lsp-router-vm1-eth1" (192.168.0.1/24), > "lsp-router-vm2-eth1" (192.168.0.2/24) and "lsp-server" (192.168.0.10/24) > 3. Port group pg1 with mentioned above LSPs > 4. Ingress and egress ACLs 0.0.0.0/0 with allow-related action. > > External: > Linux client VM (10.0.0.1), which has static ECMP route: > 192.168.0.0/24 via nexthop 10.0.0.2 via nexthop 10.0.0.3 > > sends tcp syn from ip 10.0.0.1 to 192.168.0.10 (server). > Traffic to server can go either through router VM1 (10.0.0.2) or through > router VM2 (10.0.0.3). > > Server also has static ECMP route to reverse direction: > 10.0.0.0/24 via nexthop 192.168.0.1 via nexthop 192.168.0.2 > > So, when traffic in both directions goes through same router VM, it passes > normally. > When traffic goes from client to server through one router VM and returns > back through another - SYN-ACK is blocked somewhere in OVS/conntrack. On > router VM’s interface we don’t see SYN+ACK. > > So, OVN-based ECMP feature is not relevant for this case since it doesn’t > invoke logical routers. > > Regarding the ct.inv flag, does it work when you disable the usage of > ct.inv ? > > Not yet. First wanted to understand options. > > Thanks > Numan > > > If these routes are configured in the logical router, then > > Thanks. > > Regards, > Vladislav Odintsov > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] [ovn] OVN asymmetric routing with conntrack
Regards, Vladislav Odintsov > On 25 Aug 2021, at 00:47, Numan Siddique wrote: > > On Mon, Aug 23, 2021 at 11:22 AM Vladislav Odintsov wrote: >> >> Hi, >> >> we’ve faced an issue where asymmetric-routed traffic is used. Please help >> understand what options do we have to allow such traffic. >> >> Topology is next: >> >> client lsp (10.0.0.1/24) >> | >>ls-external >>/ \ >> lsp router vm1 eth0: 10.0.0.2/24 lsp router vm2 eth0: 10.0.0.3/24 >> lsp router vm1 eth1: 192.168.0.1/24 lsp router vm2 eth1: 192.168.0.2/24 >>\ / >>ls-internal >> | >>server lsp (192.168.0.10/24) >> >> >> All LSPs have port_security configured with " 0.0.0.0/0 ::/0" and >> belong to port group pg1. >> >> There are two ACLs within this PG: >> from-lport 0.0.0.0/0 allow-related >> to-lport 0.0.0.0/0 allow-related >> >> The problem is when traffic from client to server goes through router vm1 >> and returns through router vm2, there is no connectivity. I see reply >> traffic on the server interface, which is going to router vm2 mac address, >> but I don't see it on the router vm2 interface. >> I guess the reason for this is that conntrack first time sees packet for the >> connection and ACK+SYN flags are set and treats this packet as invalid, >> right? > > I think so. > >> >> If yes, is there any option how to use asymmetric-routed topologies inside >> OVN with stateful ACLs? >> I found there is an ability to replace ct.inv field check: >> https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4 >> Is it good idea to use this option to solve the issue or this is intended >> specifically to use with smart NICs without invalid state support and can be >> removed in future? >> > > I do not understand your use case completely. I'm not quite clear > from the diagram which all resources are external > and which all are part of OVN. Have you tried using the ECMP routes feature ? > Logical Routers are not used in this topology. Only two logical switches - ls-external and ls-internal. In OVN we have: 1. LS "ls-external" with 3 LSPs: "lsp-router-vm1-eth0" (10.0.0.2/24), "lsp-router-vm2-eth0" (10.0.0.3/24) and "lsp-client" (10.0.0.1/24) 2. LS "ls-internal" with 2 LSPs: "lsp-router-vm1-eth1" (192.168.0.1/24), "lsp-router-vm2-eth1" (192.168.0.2/24) and "lsp-server" (192.168.0.10/24) 3. Port group pg1 with mentioned above LSPs 4. Ingress and egress ACLs 0.0.0.0/0 with allow-related action. External: Linux client VM (10.0.0.1), which has static ECMP route: 192.168.0.0/24 via nexthop 10.0.0.2 via nexthop 10.0.0.3 sends tcp syn from ip 10.0.0.1 to 192.168.0.10 (server). Traffic to server can go either through router VM1 (10.0.0.2) or through router VM2 (10.0.0.3). Server also has static ECMP route to reverse direction: 10.0.0.0/24 via nexthop 192.168.0.1 via nexthop 192.168.0.2 So, when traffic in both directions goes through same router VM, it passes normally. When traffic goes from client to server through one router VM and returns back through another - SYN-ACK is blocked somewhere in OVS/conntrack. On router VM’s interface we don’t see SYN+ACK. So, OVN-based ECMP feature is not relevant for this case since it doesn’t invoke logical routers. > Regarding the ct.inv flag, does it work when you disable the usage of ct.inv ? > Not yet. First wanted to understand options. > Thanks > Numan > > > If these routes are configured in the logical router, then >> Thanks. >> >> Regards, >> Vladislav Odintsov >> >> ___ >> discuss mailing list >> disc...@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] [ovn] OVN asymmetric routing with conntrack
On Mon, Aug 23, 2021 at 11:22 AM Vladislav Odintsov wrote: > > Hi, > > we’ve faced an issue where asymmetric-routed traffic is used. Please help > understand what options do we have to allow such traffic. > > Topology is next: > > client lsp (10.0.0.1/24) > | > ls-external > / \ > lsp router vm1 eth0: 10.0.0.2/24 lsp router vm2 eth0: 10.0.0.3/24 > lsp router vm1 eth1: 192.168.0.1/24 lsp router vm2 eth1: 192.168.0.2/24 > \ / > ls-internal > | > server lsp (192.168.0.10/24) > > > All LSPs have port_security configured with " 0.0.0.0/0 ::/0" and belong > to port group pg1. > > There are two ACLs within this PG: > from-lport 0.0.0.0/0 allow-related > to-lport 0.0.0.0/0 allow-related > > The problem is when traffic from client to server goes through router vm1 and > returns through router vm2, there is no connectivity. I see reply traffic on > the server interface, which is going to router vm2 mac address, but I don't > see it on the router vm2 interface. > I guess the reason for this is that conntrack first time sees packet for the > connection and ACK+SYN flags are set and treats this packet as invalid, right? I think so. > > If yes, is there any option how to use asymmetric-routed topologies inside > OVN with stateful ACLs? > I found there is an ability to replace ct.inv field check: > https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4 > Is it good idea to use this option to solve the issue or this is intended > specifically to use with smart NICs without invalid state support and can be > removed in future? > I do not understand your use case completely. I'm not quite clear from the diagram which all resources are external and which all are part of OVN. Have you tried using the ECMP routes feature ? Regarding the ct.inv flag, does it work when you disable the usage of ct.inv ? Thanks Numan If these routes are configured in the logical router, then > Thanks. > > Regards, > Vladislav Odintsov > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] SFPs & ovs-vsctl get interface
Apologies for the newbie question but could someone kindly refresh my memory as to the correct ovs-vsctl args for inspecting SFPs ? (i.e. light levels etc). I tried the man page for `ovs-vsctl` but there's not much detail in there on the args for "Interface". Thanks! Laura ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] ovsdb relay server active connection probe interval do not work
hi，all
the default inactivity probe interval of ovsdb relay server to nb/sb ovsdb
server is 5000ms.
I set an active connection as follow，set inactivity probe interval to 12ms
：
_uuid : 5ddab5a4-a267-42b4-9dd4-76d55855a109
external_ids: {}
inactivity_probe: 12
is_connected: true
max_backoff : []
other_config: {}
status : {sec_since_connect="0", state=ACTIVE}
target : "tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641"
ovn-ovsdb-nb.openstack.svc.cluster.local is a vip
but the inactivity probe is still 5000
the follow is log of ovsdb relay server
2021-08-24T12:34:17.313Z|04924|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 120225 ms, sending inactivity probe
2021-08-24T12:36:17.759Z|05854|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 120446 ms, sending inactivity probe
2021-08-24T12:37:06.326Z|06145|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 6853 ms, sending inactivity probe
2021-08-24T12:37:11.330Z|06155|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5004 ms, sending inactivity probe
2021-08-24T12:37:16.334Z|06165|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5003 ms, sending inactivity probe
2021-08-24T12:37:21.339Z|06175|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5005 ms, sending inactivity probe
2021-08-24T12:37:33.850Z|06226|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 6681 ms, sending inactivity probe
2021-08-24T12:37:38.855Z|06236|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5003 ms, sending inactivity probe
2021-08-24T12:37:43.859Z|06246|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5004 ms, sending inactivity probe
2021-08-24T12:37:48.864Z|06256|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5004 ms, sending inactivity probe
2021-08-24T12:37:53.870Z|06266|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5006 ms, sending inactivity probe
2021-08-24T12:37:58.876Z|06276|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5006 ms, sending inactivity probe
2021-08-24T12:38:08.882Z|06293|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 6299 ms, sending inactivity probe
2021-08-24T12:38:13.887Z|06303|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5003 ms, sending inactivity probe
2021-08-24T12:38:18.890Z|06313|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 121131 ms, sending inactivity probe
2021-08-24T12:38:18.891Z|06316|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5004 ms, sending inactivity probe
2021-08-24T12:38:23.895Z|06330|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5004 ms, sending inactivity probe
2021-08-24T12:38:28.901Z|06340|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5005 ms, sending inactivity probe
2021-08-24T12:38:33.905Z|06350|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5004 ms, sending inactivity probe
2021-08-24T12:38:38.909Z|06360|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5003 ms, sending inactivity probe
2021-08-24T12:38:43.913Z|06370|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5003 ms, sending inactivity probe
2021-08-24T12:38:48.922Z|06380|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5009 ms, sending inactivity probe
2021-08-24T12:38:53.926Z|06390|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5003 ms, sending inactivity probe
2021-08-24T12:38:58.930Z|06400|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5003 ms, sending inactivity probe
2021-08-24T12:39:03.934Z|06410|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5003 ms, sending inactivity probe
2021-08-24T12:39:08.938Z|06420|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5004 ms, sending inactivity probe
2021-08-24T12:39:13.941Z|06430|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5002 ms, sending inactivity probe
2021-08-24T12:39:18.946Z|06440|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5004 ms, sending inactivity probe
2021-08-24T12:39:23.951Z|06452|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5005 ms, sending inactivity probe
2021-08-24T12:39:28.956Z|06462|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5004 ms, sending inactivity probe
2021-08-24T12:39:33.962Z|06472|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
idle 5006 ms, sending inactivity probe
best regards，Wentao Jia
___
discuss mailing list
disc...@openvswitch.org
[ovs-discuss] [OVN][CT] nf_conntrack limits - silent drop / fill out
Hi, I'm stressing OVN environment by generating millions of flows. What surprises me is when I fill out conntrack table on hypervisor I don't see any errors/warning from kernel nf_conntrack or openvswitch modules. When I increase conntrack table size, it is immediately filled out again so I'm for sure generating more flows than nf_conntrack is capable to store. Is it a normal behavior? Is the OVS kernel module in some kind "soft" in filling conntrack, for example monitoring its capacity and not adding more entries when it is full? What happens with new OVS flows when conntrack is full? I suppose they are dropped because they don't match to any existing entry. Should I see any warnings/erros anywhere in kernel or openvswitch logs? Thanks ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] 答复: discuss Digest, Vol 146, Issue 36
5? Create Loadbalancer lb1,lb backen is vm1?vm2?vm3?vm4?VIP is 100.7.33.152 ovn-nbctl lb-add lb1 100.7.33.152:1234 21.21.21.21:3000,21.21.21.22:3000,22. 22.22.21:3000,22.22.22.22:3000 ovn-nbctl --wait=sb -- --id=@hc create Load_Balancer_Health_Check vip="100.7.33.152\:1234" -- add Load_Balancer . health_check @hc ovn-nbctl --wait=sb set load_balancer . ip_port_mappings: 21.21.21.21=lsp-vm1:21.21.21.2 ovn-nbctl --wait=sb set load_balancer . ip_port_mappings: 21.21.21.22=lsp-vm2:21.21.21.2 ovn-nbctl --wait=sb set load_balancer . ip_port_mappings: 22.22.22.21=lsp-vm3:22.22.22.2 ovn-nbctl --wait=sb set load_balancer . ip_port_mappings: 22.22.22.22=lsp-vm4:22.22.22.2 check service_monitor all backend are online 6? Add lb1 to LS1?LS2 and LR1 ovn-nbctl ls-lb-add sw1 lb1 ovn-nbctl lr-lb-add lr1 lb1 my topo is: My config: When I run curl http://100.7.33.152:1234 from Client,it works,But all traffic pass from NODE2 (SNAT chassis),it?s not distributed! Is there anything I mis-config? By the way,where can I found examples of how virtual port works with loadbalance? -- next part -- An HTML attachment was scrubbed... URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210824/f47b 1c30/attachment.html> -- next part -- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 22178 bytes Desc: not available URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210824/f47b 1c30/attachment.png> -- next part -- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 61094 bytes Desc: not available URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210824/f47b 1c30/attachment.jpg> -- next part -- A non-text attachment was scrubbed... Name: image003.jpg Type: image/jpeg Size: 30808 bytes Desc: not available URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210824/f47b 1c30/attachment-0001.jpg> -- next part -- A non-text attachment was scrubbed... Name: image004.jpg Type: image/jpeg Size: 27461 bytes Desc: not available URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210824/f47b 1c30/attachment-0002.jpg> -- next part -- A non-text attachment was scrubbed... Name: image005.jpg Type: image/jpeg Size: 19245 bytes Desc: not available URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210824/f47b 1c30/attachment-0003.jpg> -- next part -- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3600 bytes Desc: not available URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210824/f47b 1c30/attachment.p7s> -- Subject: Digest Footer ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss -- End of discuss Digest, Vol 146, Issue 36 smime.p7s Description: S/MIME cryptographic signature ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
