On Fri, Nov 4, 2022 at 10:22 AM Ilya Maximets <[email protected]> wrote: > > On 11/2/22 05:09, Han Zhou wrote: > > When a row is deleted, if the row has weak references to other rows, the > > weak reference nodes attached to the destination rows (through > > weak->dst_node hmap) are not destroyed. > > > > Deleting weak references is properly handled when a row is modified. The > > removed references are taken care by: > > 1. assess_weak_refs() figures out the deleted references from the row > > and add them to txn_row->deleted_refs. > > 2. before commit, in ovsdb_txn_update_weak_refs() it finds the > > destination row for each item in txn_row->deleted_refs (from step 1), > > and destroy the corresponding weak references of the destionation row. > > > > However, when the row is deleted, the step 1 in assess_weak_refs() is > > missing. It directly returns without adding the deleted references to > > txn_row->deleted_refs. So, the detination nodes will keep those weak > > references although the source side of the references are already > > deleted. When such rows that originating weak references are created > > and deleted, more and more such useless weak reference structures > > accumulate in the memory, and can stay there until the destination rows > > are deleted. It is possible that the detination row is never deleted, > > and in such case the ovsdb-server memory keeps growing (although it is > > not strictly memory leak, because the structures are still referenced). > > > > This problem has an impact to applications like OVN SB DB - the memory > > grows very fast in long-runing deployments and finally causes OOM. > > > > This patch fixes it by generating deleted_refs for deleted rows in > > assess_weak_refs(). > > > > Fixes: 4dbff9f0a685 ("ovsdb: transaction: Incremental reassessment of weak refs.") > > Signed-off-by: Han Zhou <[email protected]> > > --- > > ovsdb/transaction.c | 11 +++++++++-- > > 1 file changed, 9 insertions(+), 2 deletions(-) > > Hi, Han. > > I reproduced the issue by continuously adding and removing logical > switches in OVN setup that already has some other logical switches > pre-created. Number of weak reference objects grows without bound. > > The patch fixes the problem and looks good to me. > > I also ran some of our scale tests with ovn-heater and observed > a bit lower memory consumption overall with the fix applied (These > tests do not really remove a lot of resources). > > Good catch! Thanks! > > I slightly re-named the 'area' part of the patch subject just to > be make the look similar to other commits in that area. > > With that, applied and backported down to 2.17. > > Best regards, Ilya Maximets.
Thanks Ilya! _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
