We tested ovs from both 2.15 and 2.17 branches. The leak is not that remarkable, about 26KB memory leak detected by memleak-bpfss when creating and deleting 800 ports.
Our environment is complex with ovs, ovn, kubernetes and kube-ovn etc. I will try to reproduce it in a minimal setup. On Tue, 20 Sept 2022 at 20:40, Aaron Conole <[email protected]> wrote: > > Mengxin Liu <[email protected]> writes: > > > Sorry for the incomplete information. This patch resolved the problem > > in my environment, > > but I'm not familiar with the recycling logic and if it will affect > > the recycling. Here is the > > description of how we found and resolved the leak, hope it helps you > > further analyze this issue. > > > > The leak was found in a Kubernetes cluster that runs thousands of ci > > jobs everyday and kube-ovn is the CNI. > > We found the vswitchd memory keeps increasing even if the jobs have > > finished and all related ovs ports have > > been deleted. > > Strange - I have a test case going now - adding and deleting ports, so > far no memory issues? Can you generate a small reproducer? For > reference, I'm doing something like: > > for I in $(seq 0 1000); do > ip tuntap add mode tap p$I > sleep 1 > ovs-vsctl add-port br-int p$I > sleep 5 > ovs-vsctl del-port p$I > ip tuntap del mode tap p$I > done > > And this doesn't seem to show any change to virt/res memory on my > system. I will switch to master branch as well (I'm on 2.14 branch for > the moment) once the test completes. > > > Then we used memleak-bpfcc to find the potential memory leak with command > > "memleak-bpfcc -o 300000 -p 2099300 3600" and kept creating and > > deleting ovs ports during the > > detection period. After the detection it shows that the ofproto_usage > > objects are not released, and the > > number of the existing objects equals to the ports we created and deleted. > > > > After this patch, the memleak-bpfcc cannot find any leak and the > > memory will not always increase. > > What version of OVS are you using? > > > On Mon, 19 Sept 2022 at 21:57, Aaron Conole <[email protected]> wrote: > >> > >> Mengxin Liu <[email protected]> writes: > >> > >> > Signed-off-by: Mengxin Liu <[email protected]> > >> > --- > >> > ofproto/ofproto.c | 2 +- > >> > 1 file changed, 1 insertion(+), 1 deletion(-) > >> > > >> > diff --git a/ofproto/ofproto.c b/ofproto/ofproto.c > >> > index 3a527683c..7c6cb1c56 100644 > >> > --- a/ofproto/ofproto.c > >> > +++ b/ofproto/ofproto.c > >> > @@ -2429,7 +2429,7 @@ static void > >> > dealloc_ofp_port(struct ofproto *ofproto, ofp_port_t ofp_port) > >> > { > >> > if (ofp_to_u16(ofp_port) < ofproto->max_ports) { > >> > - ofport_set_usage(ofproto, ofp_port, time_msec()); > >> > + ofport_remove_usage(ofproto, ofp_port); > >> > } > >> > } > >> > > >> > -- > >> > >> This interferes with ofp port recycling code. Can you describe the > >> leak? I don't really understand where / what the leak is, and the > >> commit message is very sparse here. Please give details under when the > >> memory leaks, how you found it, and whether/how this changes the > >> recycling code. > >> > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
