Reviewed: https://review.opendev.org/c/openstack/neutron/+/812805 Committed: https://opendev.org/openstack/neutron/commit/f6c35527698119ee6f73a6a3613c9beebb563840 Submitter: "Zuul (22348)" Branch: master
commit f6c35527698119ee6f73a6a3613c9beebb563840 Author: Daniel Alvarez Sanchez <[email protected]> Date: Thu Oct 7 09:23:21 2021 +0200 [ovn] Stop monitoring the SB MAC_Binding table to reduce mem footprint The MAC_Binding table in the SB database may grow indefinitely (due to a lack of an aging mechanism of its entries) and eventually lead to OOM killers for neutron-server which maintains an in-memory copy of the database. In order to stop monitoring this table, this patch is invoking the ovsdb-client tool to remove the entries associated to Floating IPs that have just been detached. The execution of this tool is really fast as it will just invoke a JSON-RPC transact command which doesn't require downloading the database contents. In a scale test, the memory consumption of neutron-server dropped from 75GB to 7GB with this patch. Closes-Bug: #1946318 Signed-off-by: Daniel Alvarez Sanchez <[email protected]> Change-Id: Id84bf17953527c415d611bfc198038fb6f811de3 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1946318 Title: [ovn] Memory consumption grows over time due to MAC_Binding entries in SB database Status in neutron: Fix Released Bug description: MAC_Binding entries are used in OVN as a mechanism to learn MAC addresses on logical ports and avoid sending ARP requests to the network. There is no aging mechanism for these entries [0] and the table can grow indefinitely. In environments with for example large (eg. /16) external networks; OVN may learn a considerable amount of addresses growing the size of the db a lot. Today, Neutron monitors this table to workaround the lack of aging mechanism and remove the MAC_Binding entries associated to Floating IPs and each neutron-server worker will keep an in-memory copy of such table increasing its memory footprint to several Gigabytes, eventually leading to OOM killers. [0] https://mail.openvswitch.org/pipermail/ovs- discuss/2019-June/048936.html To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1946318/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
