On 7/5/23 17:14, Vladislav Odintsov wrote: > Hi, > Hi Vladislav,
> we’ve noticed there is a huge ovn-controller memory consumption introduced > with [0] comparing to version without its changes in ovn-controller.c part > (just OVS submodule bump without ovn-controller changes doesn’t trigger such > behaviour). > Thanks for reporting this! > On an empty host connected to working cluster ovn-controller normally > consumes ~175 MiB RSS, and the same host updated to a version with commit [0] > consumes 3.3 GiB RSS. The start time and CPU consumption during process start > of an affected version is higher that for the normal version. > > Before upgrade (normal functioning): > > #ovn-appctl -t ovn-controller memory/show ;ps-mem -p $(pidof ovn-controller)| > grep ovn > idl-cells-OVN_Southbound:343855 idl-cells-Open_vSwitch:14131 > ofctrl_desired_flow_usage-KB:76 ofctrl_installed_flow_usage-KB:60 > ofctrl_sb_flow_ref_usage-KB:18 > 174.2 MiB + 327.5 KiB = 174.5 MiB ovn-controller > > After upgrade to an affected version I’ve checked memory report while > ovn-controller was starting and at some time there was a bigger amount of > OVN_Southbound cells comparing to "after start" time: > > during start: > > # ovn-appctl -t ovn-controller memory/show ;ps-mem -p $(pidof > ovn-controller)| grep ovn > idl-cells-OVN_Southbound:11388742 idl-cells-Open_vSwitch:14131 > idl-outstanding-txns-Open_vSwitch:1 > 3.3 GiB + 327.0 KiB = 3.3 GiB ovn-controller > > after start: > > # ovn-appctl -t ovn-controller memory/show ;ps-mem -p $(pidof > ovn-controller)| grep ovn > idl-cells-OVN_Southbound:343896 idl-cells-Open_vSwitch:14131 > idl-outstanding-txns-Open_vSwitch:1 > 3.3 GiB + 327.0 KiB = 3.3 GiB ovn-controller > > > cells during start: > 11388742 > > cells after start: > 343896 > Are you running with ovn-monitor-all=true on this host? I guess it's unlikely but I'll try just in case: would it be possible to share the SB database? > I guess it could be connected with this problem. Can anyone look at this and > comment please? > Does the memory usage persist after SB is upgraded too? I see the number of SB idl-cells goes down eventually which means that eventually the periodic malloc_trim() call would free up memory. We trim on idle since https://github.com/ovn-org/ovn/commit/b4c593d23bd959e98fcc9ada4a973ac933579ded Are you using a different allocator? E.g., jemalloc. > > 0: > https://github.com/ovn-org/ovn/commit/1b0dbde940706e5de6e60221be78a278361fa76d > > > > Regards, > Vladislav Odintsov > Regards, Dumitru _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
