[ovs-discuss] OVS Killed By OOM
Hello, I am having trouble that ovs process is being killed by OOM. I am using using openstack compute with OVN / OVS and using kernel datapath. [Sat Feb 19 03:46:26 2022] Memory cgroup out of memory: Killed process 2080898 (ovs-vswitchd) total-vm:9474284kB, anon-rss:1076384kB, file-rss:11700kB, shmem-rss:0kB, UID:0 pgtables:2776kB oom_score_adj:0 [Sat Feb 19 03:47:01 2022] Memory cgroup out of memory: Killed process 2081218 (ovs-vswitchd) total-vm:9475332kB, anon-rss:1096988kB, file-rss:11700kB, shmem-rss:0kB, UID:0 pgtables:2780kB oom_score_adj:0 [Sat Feb 19 03:47:06 2022] Memory cgroup out of memory: Killed process 2081616 (ovs-vswitchd) total-vm:9473252kB, anon-rss:1073052kB, file-rss:11700kB, shmem-rss:0kB, UID:0 pgtables:2784kB oom_score_adj:0 [Sat Feb 19 03:47:16 2022] Memory cgroup out of memory: Killed process 2081940 (ovs-vswitchd) total-vm:9471236kB, anon-rss:1070920kB, file-rss:11700kB, shmem-rss:0kB, UID:0 pgtables:2776kB oom_score_adj:0 Anyone please advise how to increase the memory limit for ovs ? Ammad ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] OVN /OVS openvswitch: ovs-system: deferred action limit reached, drop recirc action
Hello All,
Is this issue fixed in any new ovn release ?
[Mon Jan 17 10:05:30 2022] openvswitch: ovs-system: deferred action limit
reached, drop recirc action
[Mon Jan 17 10:05:31 2022] openvswitch: ovs-system: deferred action limit
reached, drop recirc action
[Mon Jan 17 10:05:31 2022] openvswitch: ovs-system: deferred action limit
reached, drop recirc action
[Mon Jan 17 10:05:31 2022] openvswitch: ovs-system: deferred action limit
reached, drop recirc action
Ammad
On Tue, Nov 2, 2021 at 11:59 AM Ammad Syed wrote:
> Hi,
>
> I just upgraded by ovn and ovs to the latest releases i.e ovn 21.09 and
> ovs 2.16.0. Still getting the same messages in my dmesg logs.
>
> The issue can be reproduced by below steps.
>
> - Add neutron router
> - Set its external gateway.
> - Add a local network subnet with a router. In my case geneve is a tenant
> network and vlan is provider external network.
> - Now try to access SNAT public / external IP that is assigned to the
> router via any means (you can just put that IP in your web browser and
> enter) you will see below logs in dmesg.
> - The logs can only be seen on external gateway chassis.
>
> [Tue Nov 2 11:48:12 2021] openvswitch: ovs-system: deferred action limit
> reached, drop recirc action
> [Tue Nov 2 11:48:19 2021] openvswitch: ovs-system: deferred action limit
> reached, drop recirc action
> [Tue Nov 2 11:48:39 2021] openvswitch: ovs-system: deferred action limit
> reached, drop recirc action
>
> - Ammad
>
>
> On Thu, Sep 9, 2021 at 7:25 PM Odintsov Vladislav
> wrote:
>
>> Hi Han,
>>
>> I’ll try answer first question to move this discussion forward.
>>
>> Next is the output of the ovs-appctl ofproto/trace | ovn-detrace
>> for my topology.
>> There is a part of last stages of lr egress pipeline and jump to lr
>> ingress.
>> The full output is in attachment.
>> Hope this can help.
>>
>>
>> 25. metadata=0x4, priority 0, cookie 0xb4d0917
>> resubmit(,26)
>> * Logical datapaths:
>> * "lr0-edge" (c55eb989-eda9-47b9-8b34-e898dc1c6be2) [ingress]
>> * "lr0" (f0acad28-1531-4c32-98f1-6e95c528c2a5) [ingress]
>> * Logical flow: table=17 (lr_in_larger_pkts), priority=0, match=(1),
>> actions=(next;)
>> 26. reg15=0x1,metadata=0x4, priority 50, cookie 0x1d634149
>> set_field:0x2->reg15
>> resubmit(,27)
>> * Logical datapaths:
>> * "lr0-edge" (c55eb989-eda9-47b9-8b34-e898dc1c6be2) [ingress]
>> * Logical flow: table=18 (lr_in_gw_redirect), priority=50,
>> match=(outport == "lr0-wan), actions=(outport = "cr-lr0-wan"; next;)
>>* Logical Router Port: lr0-wan mac 0e:01:aa:29:41:03 networks ['
>> 172.16.0.1/32'] ipv6_ra_configs {}
>> 27. metadata=0x4, priority 0, cookie 0x433abe7d
>> resubmit(,37)
>> * Logical datapaths:
>> * "lr0-edge" (c55eb989-eda9-47b9-8b34-e898dc1c6be2) [ingress]
>> * "lr0" (f0acad28-1531-4c32-98f1-6e95c528c2a5) [ingress]
>> * Logical flow: table=19 (lr_in_arp_request), priority=0, match=(1),
>> actions=(output;)
>> 37. priority 0
>> resubmit(,38)
>> 38. reg15=0x2,metadata=0x4, priority 100, cookie 0xf7faafb5
>> set_field:0x1->reg15
>> set_field:0x9->reg11
>> set_field:0xb->reg12
>> resubmit(,39)
>> * Logical datapath: "lr0-edge" (c55eb989-eda9-47b9-8b34-e898dc1c6be2)
>> * Port Binding: logical_port "cr-lr0-wan", tunnel_key 2, chassis-name
>> "ai10", chassis-str "ai10.ai315t.int.c2.croc.ru"
>> 39. priority 0
>> set_field:0->reg0
>> set_field:0->reg1
>> set_field:0->reg2
>> set_field:0->reg3
>> set_field:0->reg4
>> set_field:0->reg5
>> set_field:0->reg6
>> set_field:0->reg7
>> set_field:0->reg8
>> set_field:0->reg9
>> resubmit(,40)
>> 40. ip,metadata=0x4, priority 50, cookie 0x851809e6
>> set_field:0x1/0x1->reg10
>> ct(table=41,zone=NXM_NX_REG11[0..15],nat)
>> nat
>> -> A clone of the packet is forked to recirculate. The forked pipeline
>> will be resumed at table 41.
>> -> Sets the packet to an untracked state, and clears all the conntrack
>> fields.
>> * Logical datapaths:
>> * "lr0-edge" (c55eb989-eda9-47b9-8b34-e898dc1c6be2) [egress]
>> * Logical flow: table=0 (lr_out_undnat), priority=50, match=(ip),
>> actions=(flags.loopback = 1; ct_dnat;)
>>
>> Final flow:
>> recirc_id=0x3223,eth,tcp,reg10=0x1,reg11=0x9,reg12=0xb,reg14=0x1,reg15=0x1,metadata=0x4,in_port=132,vlan_tci=0x,dl_src=0e:01:aa:29:41:03
Re: [ovs-discuss] Segmentation ID should be lower or equal to 4095
Hi Mikahil, I have used openstack xena release with ovn 21.09 installed from UCA repo from canonical, it works fine. # ovn-nbctl get NB_Global . options:max_tunid "16711680" I have set below options for geneve, [ml2_type_geneve] vni_ranges = 1:65536 max_header_size = 38 [ml2_type_vlan] network_vlan_ranges = vlannet1:3500:3600 Though I am not using vxlan. Ammad On Tue, Nov 16, 2021 at 4:53 PM Frode Nordahl wrote: > On Fri, Nov 12, 2021 at 6:13 PM Mikhail Okhrimenko > wrote: > > > > Hi, > > > > I am trying to set up OVN using Openstack. > > > > OVN 21.06 > > Openstack wallaby > > > > When creating a network in openstack, we get an error: > > > > "Segmentation ID should be lower or equal to 4095." > > > > ml2 config: > > > > # ML2 general > > [ml2] > > type_drivers = flat,vlan,vxlan,geneve > > tenant_network_types = geneve > > mechanism_drivers = ovn > > extension_drivers = port_security,dns > > physical_network_mtus = vlan:1500,floating:1500 > > path_mtu = 1558 > > overlay_ip_version = 4 > > > > # ML2 geneve networks > > [ml2_type_geneve] > > vni_ranges = 5000:6000 > > max_header_size = 38 > > > > # ML2 VLAN networks > > [ml2_type_vlan] > > network_vlan_ranges = vlan:3900:3999 > > > > # ML2 VXLAN networks > > [ml2_type_vxlan] > > vni_ranges = 5000:6000 > > > > > > # Security groups > > [securitygroup] > > enable_security_group = True > > firewall_driver = > neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver > > > > [ovn] > > ovn_nb_connection = tcp::6641 > > ovn_sb_connection = tcp::6642 > > ovn_l3_scheduler = leastloaded > > ovn_metadata_enabled = True > > isolated_metadata = True > > > > > > When I execute the command > > $ ovn-nbctl get NB_Global. options:max_tunid > > or > > $ ovn-sbctl get SB_Global. options:max_tunid > > I get "4095" > > > > If I manually try to change max_tunid using the command > > $ ovn-nbctl set NB_Global. options:max_tunid="16777215" > > or > > $ ovn-sbctl set SB_Global. options:max_tunid="16777215" > > then no change occurs. > > > > How can I change max_tunid from 4095 to 16777215 for supporting geneve > or vxlan networks in Openstack. > > This bug report should probably be directed at OpenStack Neutron, I > believe they manage bugs using Launchpad [0]. > > However, I can also provide some insights on the topic. For overlay > type networks, Neutron will actually not provide OVN with the > segmentation ID, the VNI will be allocated by OVN itself. So to solve > your concrete problem now you can just set your vni_ranges option to a > value below 4095 and unless you are going to create 4096+ networks > today this should work fine. > > OVN does not create datapaths with overlapping VNIs, so you should not > have issues with overlaps regardless of what Neutron's internal > representation of the VNI is. > > 0: https://bugs.launchpad.net/neutron/+filebug > > -- > Frode Nordahl > > > > > -- > > --- > > > > Mikhail Okhrimenko > > > > System administrator at "PS Internet Company" > > > > ___ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > -- Regards, Syed Ammad Ali ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] [ovn] Upgrade from 20.12 to 21.09
Hi, I have upgraded ovn controllers on compute nodes and ovn central on network node. The package has been upgraded and everything works fine. How to check to check ovnsb and ovnnb databases has upgraded successfully ? I am seeing ovnnb version in ovnnb_db.db "name":"OVN_Northbound","version":"5.32.1" and for ovnsb version in ovnsb_db.db "name":"OVN_Southbound","version":"20.20.0" in the start of the db file. I can also see below db files created after upgrade. -rw-r- 1 root root 151488 Nov 2 14:23 ovnnb_db.db -rw-r- 1 root root 78946 Nov 2 10:34 *ovnnb_db.db.backup5.28.0-610359755* -rw-r- 1 root root 969484 Nov 2 14:23 ovnsb_db.db -rw-r- 1 root root 391183 Nov 2 10:34 *ovnsb_db.db.backup20.12.0-3969471120 * How to ensure that db upgrade has been performed successfully ? Ammad ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] OVN /OVS openvswitch: ovs-system: deferred action limit reached, drop recirc action
(lr_in_learn_neighbor), priority=100, > match=(reg9[2] == 1), actions=(next;) > > > Regards, > Vladislav Odintsov > > On 4 Aug 2021, at 21:02, Han Zhou wrote: > > > > On Wed, Aug 4, 2021 at 6:41 AM Numan Siddique wrote: > > > > On Wed, Aug 4, 2021 at 4:17 AM Krzysztof Klimonda > > wrote: > > > > > > Hi Ammad, > > > > > > (Re-adding ovs-discuss@openvswitch.org to CC to keep track of the > discussion) > > > > > > Thanks for testing it with SNAT enabled/disabled and verifying that it > seems to be related. > > > > > > As for the impact of this bug I have to say I'm unsure. I have > theorized that this could the cause for (or at least connected to) BFD > sessions being dropped between gateway chassises, but I couldn't really > validate it. > > > > > > My linked patch is pretty old and no longer applies cleanly on master, > but I'd be interested in getting some feedback from developers on whether > I'm even fixing the right thing. > > > > Hi Krzysztof, > > > > Your patch is in the "change requested" stage. I see from the comment > > that the ddlog part of the code is missing. > > > > Seems like a valid case to me. The issue is seen when the packet is > > destined to the router port IP right ? > > > > In the case of ovn-kubernetes, the router port IP is also used as a > > load balancer backend IP. > > > > Will your patch have any impact if the logical router has this load > > balancer configured ? (for the system test case you've added ) > > > > ovn-nbctl lb-add lb1 172.16.1.254:90 192.168.1.100:90 > > ovn-nbctl lr-lb-add R1 lb1 > > > > Can you please repost the patch for further review. It would be great > > if you can add ddlog code. Or you can repost the patch > > and the ddlog part can be added if the reviewers are fine with the patch. > > > > Thanks > > Numan > > > > Thanks Krzysztof, this is interesting. Could you share more on the root > cause since you debugged it - how did the loop happen? When a packet > destined to the SNAT IP hits the router ingress pipeline, what's the next > hop? How the L2 dst is populated for the dst IP and how is the packet > forwarded back to the router pipeline? How /32 IP (instead of a subnet) on > the SNAT config made a difference? > > > > > > > Regards, > > > Krzysztof > > > > > > On Wed, Aug 4, 2021, at 09:02, Ammad Syed wrote: > > > > I am able to reproduce this issue with snat enabled network and > > > > accessing the snat IP from external network can reproduce this issue > . > > > > If I keep snat disable, then I didn't see these logs in syslog. > > > > > > > > Ammad > > > > > > > > On Tue, Aug 3, 2021 at 6:39 PM Ammad Syed > wrote: > > > > > Thanks. Let me try to reproduce it with this way. > > > > > > > > > > Can you please advise if this will cause any trouble if we have > this bug in production? Any workaround to avoid this issue? > > > > > > > > > > Ammad > > > > > > > > > > On Tue, Aug 3, 2021 at 5:56 PM Krzysztof Klimonda < > kklimo...@syntaxhighlighted.com> wrote: > > > > >> Hi, > > > > >> > > > > >> To reproduce it (on openstack. although the issue does not seem > to be openstack-specific) I've created a network with SNAT enabled (which > is default) and set its external gateway to my external network. Next, I've > tried establishing TCP session from the outside to IP address assigned to > the router and checked dmesg on the chassis that the port is assigned to > for "ovs-system: deferred action limit reached, drop recirc action" > messages. > > > > >> > > > > >> Best Regards, > > > > >> Krzysztof > > > > >> > > > > >> On Tue, Aug 3, 2021, at 09:05, Ammad Syed wrote: > > > > >> > Hi Krzysztof, > > > > >> > > > > > >> > Yes I might be stuck in this issue. How can I check if there is > any > > > > >> > loop in lflow-list ? > > > > >> > > > > > >> > Ammad > > > > >> > > > > > >> > On Tue, Aug 3, 2021 at 2:14 AM Krzysztof Klimonda > > > > >> > wrote: > > > > >> > > Hi, > > > > >> > > > > > > >> > > Not sure if it's related, but I've seen this bug
Re: [ovs-discuss] MAC_Binding\" table to have identical values
Hi Numan,
I tried to reproduce the issue with your way but didn't get the warning
again by restarting the controller of one chassis. Then I reviewed your
last response and seen the warning log.
The mac_binding entry that it tries to create is learned from external as
seen in the last warning log. I have then applied below patch of neutron.
https://opendev.org/openstack/neutron/commit/a278c5ba789c014ec777a75fc9538179d6707202
The patch is making always_learn_from_arp_request to false while creating
the router. This has reduced the SB DB size as well.
On Thu, Oct 21, 2021 at 1:29 AM Numan Siddique wrote:
> On Mon, Oct 18, 2021 at 7:55 AM Daniel Alvarez Sanchez
> wrote:
> >
> >
> >
> > On Mon, Oct 18, 2021 at 1:12 PM Ammad Syed
> wrote:
> >>
> >> Hi Brendan,
> >>
> >> Not sure but this could be related to the patch below in neutron that
> was recently released.
> >>
> >>
> https://opendev.org/openstack/neutron/commit/f6c35527698119ee6f73a6a3613c9beebb563840
> >
> >
> > Not really, as this commit that you refer to is to reduce the memory
> footprint in the neutron-server process itself. Neutron never inserts
> anything into the MAC_Binding table.
> > However, we landed another patch [0] that will likely minimize this
> issue as it will reduce the MAC_Binding insertions, limiting them only to
> MAC addresses that need to be reached out by the overlay.
> >
> > Thanks!
> > daniel
> >
> > [0] https://review.opendev.org/c/openstack/neutron/+/813610
> >
> >
> >>
> >>
> >> Ammad
> >>
> >> On Mon, Oct 18, 2021 at 3:40 PM Brendan Doyle
> wrote:
> >>>
> >>>
> >>> I too am seeing many entries in the ovn-controller log like these:
> >>>
> >>>
> ovn/ovn-controller.log:2021-10-18T09:50:46.984Z|00164|ovsdb_idl|WARN|transaction
> error: {"details":"Transaction causes multiple rows in \"MAC_Binding\"
> table to have identical values (lr_vcn6727324-ls_vcn6727324_external_ugw
> and \"253.255.80.18\") for index on columns \"logical_port\" and \"ip\".
> First row, with UUID 614509d1-5dfc-4268-94b7-6190c1bd8c58, was inserted by
> this transaction. Second row, with UUID
> 5b56044f-7e62-4e6b-b559-2bca31ad5ab0, existed in the database before this
> transaction and was not modified by the transaction.","error":"constraint
> violation"}
> >>>
> >>>
> >>> So with my limited understanding of the MAC_Binding table I'm
> wondering are these benign?
> >
> >
> > I think it's benign
> >>>
> >>>
> >>> In that is it that we for what ever reason have decided that we need
> to do an ARP to discover
> >>> a MAC, then ovn-controller tries to add the discovered MAC to the
> MAC_Binding only to find
> >>> that it is already there. In which case the question is why was an ARP
> request sent when
> >>> we already had a MAC binding.
>
> FYI, ovn-controller also learns from the ARP requests received from
> external which can be
> turned off if desired with the option - always_learn_from_arp_request
> set in the logical router's options
> column.
>
> I don't think ovn-controller will generate an ARP request if its
> already learnt. Looks to me
> multiple ovn-controller try to learn from an ARP request originated
> from external and ofcourse
> only one will be able to write to the mac_binding entry and others would
> fail.
>
> Thanks
> Numan
>
>
>
> >>>
> >>> For me these are only ever seen on Distributed router Port Gateways.
> >>>
> >>>
> >>> On 12/10/2021 13:06, Ammad Syed wrote:
> >>>
> >>> Hi,
> >>>
> >>> I am using openstack with ml2/ovn. I have two gateway chassis whenever
> I shutdown one chassis and bring it back online, I see below error in
> ovn-controller logs.
> >>>
> >>> Can you please advise the way to fix it ?
> >>>
> >>>
> 2021-10-12T11:49:22.124Z|00511|binding|INFO|cr-lrp-4c882760-15b1-4cf4-b680-53e0e928:
> Claiming fa:16:3e:55:12:5a 101.53.244.97/24
> >>> 2021-10-12T11:49:22.126Z|00512|binding|INFO|Releasing lport
> cr-lrp-4c882760-15b1-4cf4-b680-53e0e928 from this chassis.
> >>> 2021-10-12T11:50:32.032Z|00513|ovsdb_idl|WARN|transaction error:
> {"details":"Transaction causes multiple rows in \"MAC_Binding\" table to
> have identical values (lrp-4c882760-15b1-4cf4-b680-53e0e928 and
> \"fe80::6a4f:64ff:fef7:8c0\
[ovs-discuss] OVN Error Logs
Hi,
I am using openstack with ml2/ovn. I have two gateway chassis whenever I
shutdown one chassis and bring it back online, I see below error in
ovn-controller logs.
Can you please advise the way to fix it ?
2021-10-12T11:49:22.124Z|00511|binding|INFO|cr-lrp-4c882760-15b1-4cf4-b680-53e0e928:
Claiming fa:16:3e:55:12:5a 101.53.244.97/24
2021-10-12T11:49:22.126Z|00512|binding|INFO|Releasing lport
cr-lrp-4c882760-15b1-4cf4-b680-53e0e928 from this chassis.
2021-10-12T11:50:32.032Z|00513|ovsdb_idl|WARN|transaction error:
{"details":"Transaction causes multiple rows in \"MAC_Binding\" table to
have identical values (lrp-4c882760-15b1-4cf4-b680-53e0e928 and
\"fe80::6a4f:64ff:fef7:8c0\") for index on columns \"logical_port\" and
\"ip\". First row, with UUID cbc847e6-75e4-4d73-9906-4fef221cad38, was
inserted by this transaction. Second row, with UUID
bcac2e3e-5f32-411a-afcf-3249b85700f4, existed in the database before this
transaction and was not modified by the transaction.","error":"constraint
violation"}
Ammad
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] [ovn] logical router port failover
Hi,
I am using openstack neutron and using distributed floating IP with ovn
backend. The VMs that have floating IP NATed have distributed traffic and
its traffic goes out directly from compute node uplink.
While the traffic of SNAT goes from lrp that is scheduled on the gateway
chassis. Routers are scheduled on HA node.
# ovn-nbctl list logical_router_port f3ab4336-559a-4081-b975-cf3c9a1dd6ad
_uuid : f3ab4336-559a-4081-b975-cf3c9a1dd6ad
enabled : []
external_ids:
{"neutron:network_name"=neutron-1ce2354f-7a83-45ab-80f5-1e9dc1c16be9,
"neutron:revision_number"="2341",
"neutron:router_name"="ed24659b-6345-4ea7-b651-b8a0af875b5e",
"neutron:subnet_ids"="d3c02380-17d3-4381-ac42-f1260e7f8b79"}
gateway_chassis : [3bae14ea-6fe2-4c43-a8bb-1a99504662ef,
42762997-1f91-404c-a170-3f6547b2e0a3, eb4f0959-d4b5-4129-8c02-6d48e15d3a68]
ha_chassis_group: []
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:c5:c9:64"
name: lrp-95b381bb-8dd7-47ba-a673-6724e8163c12
networks: ["x.x.x.x/x"]
options : {}
peer: []
How can I check that the router is currently scheduled on which chassis ?
and how to migrate manually gateway router (lrp) from one gateway chassis
to another gateway chassis ?
--
Ammad Ali
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] OVN disable conntrack for UDP ACL
Refer the xena release notes of neutron here. https://docs.openstack.org/releasenotes/neutron/unreleased.html Ammad On Sat, Sep 11, 2021 at 7:45 AM Satish Patel wrote: > Thank you for reply, > > That does make sense, if xena has support then i can wait for it, I > believe it's about to release. > > On Fri, Sep 10, 2021 at 10:29 PM Ammad Syed wrote: > > > > I think stateless acl with ovn backend is currently not supported in > openstack. The feature is planned and will be available in next openstack > release i.e xena. > > > > Ammad > > On Sat, Sep 11, 2021 at 1:23 AM Satish Patel > wrote: > >> > >> Thank you, i am trying the following but look like it doesn't like it, > >> Openstack Doc saying it should work. (i am running latest openstack) > >> > >> # openstack security group create --stateless foo_sg > >> Error while executing command: BadRequestException: 400, Unrecognized > >> attribute(s) 'stateful' > >> > >> On Fri, Sep 10, 2021 at 4:05 PM Odintsov Vladislav > wrote: > >> > > >> > I’m not an openstack user, so leave this question to somebody > >> > from openstack guys. > >> > > >> > Regards, > >> > Vladislav Odintsov > >> > > >> > On 10 Sep 2021, at 23:00, Satish Patel wrote: > >> > > >> > Thank you for your reply, > >> > > >> > Glad to know there is a workaround, i am little noob to OVN, could you > >> > explain how to set higher priority ACL using "openstack security group > >> > rule" command, because most of my users using terrafrom to deploy vms > >> > and play with security-group and how do i tell allow-stateless when > >> > create group using openstack clients? > >> > > >> > On Fri, Sep 10, 2021 at 3:54 PM Odintsov Vladislav < > vlodint...@croc.ru> wrote: > >> > > >> > > >> > Hi, > >> > > >> > with OVN 21.06+ you can create overriding ACLs with higher priority > >> > than you currently have, with special "allow-stateless" verb, which > ensures > >> > packets bypassing conntrack. > >> > > >> > Regards, > >> > Vladislav Odintsov > >> > > >> > On 10 Sep 2021, at 22:49, Satish Patel wrote: > >> > > >> > Folk, > >> > > >> > We are a large shop of UDP applications so trying to find a way to > >> > disable the conntrack for the entire UDP protocol stack, I did google > >> > and dig into some ovn documentation but did not find any workaround > >> > which allows disabling a conntrack on UDP protocol. > >> > > >> > Or another option i was thinking of is to disable ACL in OVS entirely > >> > and then i will use iptables on vm because that way i can disable > >> > conntrack using iptables. > >> > > >> > Anyone have any idea what to do if possible? > >> > ___ > >> > discuss mailing list > >> > disc...@openvswitch.org > >> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > >> > > >> > > >> > > >> ___ > >> discuss mailing list > >> disc...@openvswitch.org > >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > > -- > > Regards, > > > > > > Syed Ammad Ali > -- Regards, Syed Ammad Ali ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] OVN disable conntrack for UDP ACL
I think stateless acl with ovn backend is currently not supported in openstack. The feature is planned and will be available in next openstack release i.e xena. Ammad On Sat, Sep 11, 2021 at 1:23 AM Satish Patel wrote: > Thank you, i am trying the following but look like it doesn't like it, > Openstack Doc saying it should work. (i am running latest openstack) > > # openstack security group create --stateless foo_sg > Error while executing command: BadRequestException: 400, Unrecognized > attribute(s) 'stateful' > > On Fri, Sep 10, 2021 at 4:05 PM Odintsov Vladislav > wrote: > > > > I’m not an openstack user, so leave this question to somebody > > from openstack guys. > > > > Regards, > > Vladislav Odintsov > > > > On 10 Sep 2021, at 23:00, Satish Patel wrote: > > > > Thank you for your reply, > > > > Glad to know there is a workaround, i am little noob to OVN, could you > > explain how to set higher priority ACL using "openstack security group > > rule" command, because most of my users using terrafrom to deploy vms > > and play with security-group and how do i tell allow-stateless when > > create group using openstack clients? > > > > On Fri, Sep 10, 2021 at 3:54 PM Odintsov Vladislav > wrote: > > > > > > Hi, > > > > with OVN 21.06+ you can create overriding ACLs with higher priority > > than you currently have, with special "allow-stateless" verb, which > ensures > > packets bypassing conntrack. > > > > Regards, > > Vladislav Odintsov > > > > On 10 Sep 2021, at 22:49, Satish Patel wrote: > > > > Folk, > > > > We are a large shop of UDP applications so trying to find a way to > > disable the conntrack for the entire UDP protocol stack, I did google > > and dig into some ovn documentation but did not find any workaround > > which allows disabling a conntrack on UDP protocol. > > > > Or another option i was thinking of is to disable ACL in OVS entirely > > and then i will use iptables on vm because that way i can disable > > conntrack using iptables. > > > > Anyone have any idea what to do if possible? > > ___ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > > > > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > -- Regards, Syed Ammad Ali ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] OVN /OVS openvswitch: ovs-system: deferred action limit reached, drop recirc action
Hello, I am using openstack with OVN 20.12 and OVS 2.15.0 on ubuntu 20.04. I am using geneve tenant network and vlan provider network. I am continuously getting below messages in my dmesg logs continuously on compute node 1 only the other two compute nodes have no such messages. [275612.826698] openvswitch: ovs-system: deferred action limit reached, drop recirc action [275683.750343] openvswitch: ovs-system: deferred action limit reached, drop recirc action [276102.200772] openvswitch: ovs-system: deferred action limit reached, drop recirc action [276161.575494] openvswitch: ovs-system: deferred action limit reached, drop recirc action [276210.262524] openvswitch: ovs-system: deferred action limit reached, drop recirc action I have tried by reinstalling (OS everything) compute node 1 but still having same errors. Need your advise. -- Regards, Syed Ammad Ali ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] [ovn/ovs] hit and miss
Hi, I am using ovn with openstack and using geneve tunnel as an overlay. I have two compute nodes and I am trying to test network performance via iperf3 between VMs. I have configured 9000 on tunnel interface and VM has 1500 mtu. The network performance with 1500mtu is 3.5Gbps with --bidir switch and with 8942 I am having 7.5Gbps with --bidir between VMs on different compute hosts. I am net using dpdk and or any other offloading. I have checked ovs-dpctl stats. Node01# ovs-dpctl show system@ovs-system: lookups: hit:24147954 missed:294831 lost:4 flows: 21 masks: hit:161013147 total:4 hit/pkt:6.59 port 0: ovs-system (internal) port 1: br-int (internal) port 2: tapeee623a9-24 port 3: tap18ca5a79-10 port 4: br-vlan (internal) port 5: bond0 port 6: genev_sys_6081 (geneve: packet_type=ptap) port 8: tap0181d519-c0 port 9: tap630cd3eb-61 Node02# ovs-dpctl show system@ovs-system: lookups: hit:23287526 missed:1140779 lost:5 flows: 19 masks: hit:196158405 total:11 hit/pkt:8.03 port 0: ovs-system (internal) port 1: br-int (internal) port 2: br-vlan (internal) port 3: bond0 port 4: genev_sys_6081 (geneve: packet_type=ptap) port 5: tap58e445c4-75 port 6: tap0181d519-c0 Are above stats ok or do I need to perform any other tuning on kernel or at ovs level ? Ammad ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] [ovn/ovs] OVS hardware offloading
Hi, I am using OVN with openstack neutron on ubuntu 20.04. I want to enable hardware offloading on ovs. I have used below command to to enable offload. ovs-vsctl set Open_vSwitch . other_config:hw-offload=true By reviewing below URL. https://www.openvswitch.org/support/ovscon2019/day2/0951-hw_offload_ovs_con_19-Oz-Mellanox.pdf I just want to enable kernel offload using tc. I am using Broadcom BCM57504 NetXtreme-E and below firmware and driver details. driver: bnxt_en version: 1.10.0 firmware-version: 218.0.169.2/pkg 21.80.16.95 expansion-rom-version: bus-info: :63:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: no supports-priv-flags: no I have checked with ethtool -k with on NIC, hw-tc-offload: on is on. When I have enabled offloading. I have started seeing below errors. [ 1197.506839] bnxt_en :63:00.0 eth0-tor1: hwrm req_type 0x103 seq id 0x1282 error 0x2 [ 1197.507086] bnxt_en :63:00.0 eth0-tor1: Error: bnxt_tc_add_flow: cookie=0x9fe730da8000 error=-22 [ 1197.516042] bnxt_en :63:00.1 eth0-tor2: hwrm req_type 0x103 seq id 0x127f error 0x2 [ 1197.525717] bnxt_en :63:00.1 eth0-tor2: Error: bnxt_tc_add_flow: cookie=0x9fe730da8000 error=-22 [ 1227.522630] bnxt_en :63:00.0 eth0-tor1: hwrm req_type 0x103 seq id 0x12f7 error 0x2 [ 1227.531524] bnxt_en :63:00.0 eth0-tor1: Error: bnxt_tc_add_flow: cookie=0x9fe731a28800 error=-22 [ 1227.540403] bnxt_en :63:00.1 eth0-tor2: hwrm req_type 0x103 seq id 0x12f4 error 0x2 [ 1227.548676] bnxt_en :63:00.1 eth0-tor2: Error: bnxt_tc_add_flow: cookie=0x9fe731a28800 error=-22 [ 1257.563517] bnxt_en :63:00.0 eth0-tor1: hwrm req_type 0x103 seq id 0x136c error 0x2 [ 1257.572898] bnxt_en :63:00.0 eth0-tor1: Error: bnxt_tc_add_flow: cookie=0x9fe731a28800 error=-22 [ 1257.583457] bnxt_en :63:00.1 eth0-tor2: hwrm req_type 0x103 seq id 0x1369 error 0x2 [ 1257.591647] bnxt_en :63:00.1 eth0-tor2: Error: bnxt_tc_add_flow: cookie=0x9fe731a28800 error=-22 Please advise how to fix it. -- Regards, Ammad ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] [ovn/ovs] OVS hardware offloading
Hi, I am using OVN with openstack neutron on ubuntu 20.04. I want to enable hardware offloading on ovs. I have used below command to to enable offload. ovs-vsctl set Open_vSwitch . other_config:hw-offload=true By reviewing below URL. https://www.openvswitch.org/support/ovscon2019/day2/0951-hw_offload_ovs_con_19-Oz-Mellanox.pdf I just want to enable kernel offload using tc. I am using Broadcom BCM57504 NetXtreme-E and below firmware and driver details. driver: bnxt_en version: 1.10.0 firmware-version: 218.0.169.2/pkg 21.80.16.95 expansion-rom-version: bus-info: :63:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: no supports-priv-flags: no I have checked with ethtool -k with on NIC, hw-tc-offload: on is on. When I have enabled offloading. I have started seeing below errors. [ 1197.506839] bnxt_en :63:00.0 eth0-tor1: hwrm req_type 0x103 seq id 0x1282 error 0x2 [ 1197.507086] bnxt_en :63:00.0 eth0-tor1: Error: bnxt_tc_add_flow: cookie=0x9fe730da8000 error=-22 [ 1197.516042] bnxt_en :63:00.1 eth0-tor2: hwrm req_type 0x103 seq id 0x127f error 0x2 [ 1197.525717] bnxt_en :63:00.1 eth0-tor2: Error: bnxt_tc_add_flow: cookie=0x9fe730da8000 error=-22 [ 1227.522630] bnxt_en :63:00.0 eth0-tor1: hwrm req_type 0x103 seq id 0x12f7 error 0x2 [ 1227.531524] bnxt_en :63:00.0 eth0-tor1: Error: bnxt_tc_add_flow: cookie=0x9fe731a28800 error=-22 [ 1227.540403] bnxt_en :63:00.1 eth0-tor2: hwrm req_type 0x103 seq id 0x12f4 error 0x2 [ 1227.548676] bnxt_en :63:00.1 eth0-tor2: Error: bnxt_tc_add_flow: cookie=0x9fe731a28800 error=-22 [ 1257.563517] bnxt_en :63:00.0 eth0-tor1: hwrm req_type 0x103 seq id 0x136c error 0x2 [ 1257.572898] bnxt_en :63:00.0 eth0-tor1: Error: bnxt_tc_add_flow: cookie=0x9fe731a28800 error=-22 [ 1257.583457] bnxt_en :63:00.1 eth0-tor2: hwrm req_type 0x103 seq id 0x1369 error 0x2 [ 1257.591647] bnxt_en :63:00.1 eth0-tor2: Error: bnxt_tc_add_flow: cookie=0x9fe731a28800 error=-22 Please advise how to fix it. -- Regards, Ammad ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
