Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-08-11 Thread Dumitru Ceara
On 8/10/23 18:56, Dumitru Ceara wrote: > On 8/10/23 18:15, Han Zhou wrote: >> Thanks Vladislav and Dumitru for reporting and fixing the issue. >> The impact of the issue is more than just the memory spikes in >> ovn-controller. More importantly, it incurs much higher load on SB DB >> because the

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-08-10 Thread Dumitru Ceara
On 8/10/23 18:15, Han Zhou wrote: > Thanks Vladislav and Dumitru for reporting and fixing the issue. > The impact of the issue is more than just the memory spikes in > ovn-controller. More importantly, it incurs much higher load on SB DB > because the conditions are flooded with all the localnet

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-08-10 Thread Han Zhou
Thanks Vladislav and Dumitru for reporting and fixing the issue. The impact of the issue is more than just the memory spikes in ovn-controller. More importantly, it incurs much higher load on SB DB because the conditions are flooded with all the localnet ports regardless of whether they belong to

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-08-01 Thread Vladislav Odintsov
Hi Dumitru! I’ve performed test on the host, where ovn-controller (22.09.x) without your patch and without any local datapaths consumed 3+GiB of ram after start and the process start took 100% CPU during ~40 seconds after start. With your patch ovn-controller starts during ~5 seconds with max

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-31 Thread Vladislav Odintsov
Thanks Dumitru! I’ll test this patch in a few days. > On 28 Jul 2023, at 14:36, Dumitru Ceara wrote: > > Hi Vladislav, > > After quite some time trying to implement the IDL API change to allow > setting a different default monitor condition and mostly struggling with > ovn-controller using

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-28 Thread Dumitru Ceara
Hi Vladislav, After quite some time trying to implement the IDL API change to allow setting a different default monitor condition and mostly struggling with ovn-controller using that properly I kind of gave up and decided to approach this in a different way. We have guidelines about supported

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-25 Thread Vladislav Odintsov
Many thanks for the information! > On 25 Jul 2023, at 11:14, Dumitru Ceara wrote: > > On 7/24/23 21:10, Vladislav Odintsov wrote: >> Hi Dumitru, >> > > Hi Vladislav, > >> I just wanted to ask wether you need any help (maybe, testing) in this? >> I’m ready to check this on my dataset if you

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-25 Thread Dumitru Ceara
On 7/24/23 21:10, Vladislav Odintsov wrote: > Hi Dumitru, > Hi Vladislav, > I just wanted to ask wether you need any help (maybe, testing) in this? > I’m ready to check this on my dataset if you were successful to > implement a fix. > Thanks for offering to help. I didn't get the chance to

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-24 Thread Vladislav Odintsov
Hi Dumitru, I just wanted to ask wether you need any help (maybe, testing) in this? I’m ready to check this on my dataset if you were successful to implement a fix. > On 12 Jul 2023, at 12:15, Dumitru Ceara wrote: > > On 7/12/23 00:01, Ilya Maximets wrote: >> On 7/11/23 19:01, Dumitru Ceara

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-12 Thread Dumitru Ceara
On 7/12/23 00:01, Ilya Maximets wrote: > On 7/11/23 19:01, Dumitru Ceara wrote: >> On 7/11/23 18:33, Vladislav Odintsov wrote: >>> Hi Dumitru, >>> >>> The system on which I reproduced this issue is running 22.09.x version. >>> I’ve tried to upgrade ovn-controller to main branch + your patch.

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-11 Thread Ilya Maximets
On 7/11/23 19:01, Dumitru Ceara wrote: > On 7/11/23 18:33, Vladislav Odintsov wrote: >> Hi Dumitru, >> >> The system on which I reproduced this issue is running 22.09.x version. I’ve >> tried to upgrade ovn-controller to main branch + your patch. Please, note >> that it has test error: [1]. >>

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-11 Thread Dumitru Ceara
On 7/11/23 18:33, Vladislav Odintsov wrote: > Hi Dumitru, > > The system on which I reproduced this issue is running 22.09.x version. I’ve > tried to upgrade ovn-controller to main branch + your patch. Please, note > that it has test error: [1]. > After two minutes after upgrade it still

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-11 Thread Vladislav Odintsov
Hi Dumitru, The system on which I reproduced this issue is running 22.09.x version. I’ve tried to upgrade ovn-controller to main branch + your patch. Please, note that it has test error: [1]. After two minutes after upgrade it still consumed 3.3G. I tried to backport your patch to 22.09, it

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-11 Thread Dumitru Ceara
On 7/10/23 22:20, Vladislav Odintsov wrote: > Hi Dumitru, > > thanks for digging into this! I highly appreciate your help! > No worries, my pleasure! :) > Please, see my answers inline. > >> On 10 Jul 2023, at 15:28, Dumitru Ceara wrote: >> >> On 7/10/23 12:57, Dumitru Ceara wrote: >>> On

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-10 Thread Vladislav Odintsov
Hi Dumitru, thanks for digging into this! I highly appreciate your help! Please, see my answers inline. > On 10 Jul 2023, at 15:28, Dumitru Ceara wrote: > > On 7/10/23 12:57, Dumitru Ceara wrote: >> On 7/6/23 13:00, Vladislav Odintsov wrote: >>> >>> On 5 Jul 2023, at 20:07, Vladislav

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-10 Thread Dumitru Ceara
On 7/10/23 12:57, Dumitru Ceara wrote: > On 7/6/23 13:00, Vladislav Odintsov wrote: >> >> >>> On 5 Jul 2023, at 20:07, Vladislav Odintsov wrote: >>> >>> Hi Dumitru, >>> >>> thanks for the quick response! >>> On 5 Jul 2023, at 19:53, Dumitru Ceara wrote: On 7/5/23 17:14, Vladislav

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-10 Thread Dumitru Ceara
On 7/6/23 13:00, Vladislav Odintsov wrote: > > >> On 5 Jul 2023, at 20:07, Vladislav Odintsov wrote: >> >> Hi Dumitru, >> >> thanks for the quick response! >> >>> On 5 Jul 2023, at 19:53, Dumitru Ceara wrote: >>> >>> On 7/5/23 17:14, Vladislav Odintsov wrote: Hi, >>> >>> Hi

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-06 Thread Vladislav Odintsov
> On 5 Jul 2023, at 20:07, Vladislav Odintsov wrote: > > Hi Dumitru, > > thanks for the quick response! > >> On 5 Jul 2023, at 19:53, Dumitru Ceara wrote: >> >> On 7/5/23 17:14, Vladislav Odintsov wrote: >>> Hi, >>> >> >> Hi Vladislav, >> >>> we’ve noticed there is a huge ovn-controller

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-05 Thread Vladislav Odintsov
Hi Dumitru, thanks for the quick response! > On 5 Jul 2023, at 19:53, Dumitru Ceara wrote: > > On 7/5/23 17:14, Vladislav Odintsov wrote: >> Hi, >> > > Hi Vladislav, > >> we’ve noticed there is a huge ovn-controller memory consumption introduced >> with [0] comparing to version without its

Re: [ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-05 Thread Dumitru Ceara
On 7/5/23 17:14, Vladislav Odintsov wrote: > Hi, > Hi Vladislav, > we’ve noticed there is a huge ovn-controller memory consumption introduced > with [0] comparing to version without its changes in ovn-controller.c part > (just OVS submodule bump without ovn-controller changes doesn’t trigger

[ovs-dev] [ovn] ovn-controller high memory consumption with sbrec_server_has_##table##_table - enabled codepath

2023-07-05 Thread Vladislav Odintsov
Hi, we’ve noticed there is a huge ovn-controller memory consumption introduced with [0] comparing to version without its changes in ovn-controller.c part (just OVS submodule bump without ovn-controller changes doesn’t trigger such behaviour). On an empty host connected to working cluster