On Wed, Feb 17, 2021, at 17:34, Gaetan Rivet wrote: > Conntracks are executed within the datapath. Locks along this path are crucial > and their critical section should be minimal. The global 'ct_lock' is > necessary > before any action taken on connection states. This lock is needed for many > operations on the conntrack, slowing down the datapath. > > The cleanup thread 'ct_clean' will take it to do its job. As it can hold it a > long time, the thread is limited in amount of connection cleaned per round, > and calls are rate-limited. > > * Timeout policies locking is contrived to avoid deadlock. > Anytime a connection state is updated, during its update it is unlocked, > 'ct_lock' is taken, then the connection is locked again. Then the reverse > is done for unlock. > > * Scalability is poor. The global ct_lock needs to be taken before applying > any change to a conn object. This is backward: local changes to smaller > objects should be independent, then the global lock should only be taken > once > the rest of the work is done, the goal being to have the smallest possible > critical section. > > It can be improved. Using RCU-friendly structures for connections, zone limits > and timeout policies, read-first workload is improved and the precedence of > the > global 'ct_lock' and local 'conn->lock' can be inversed. > > Running the conntrack benchmark we see these changes: > ./tests/ovstest test-conntrack benchmark <N> 3000000 32 > > code \ N 1 2 4 8 > Before 2310 2766 6117 19838 (ms) > After 2072 2084 2653 4541 (ms) > > One thread in the benchmark executes the task of a PMD, while the > 'ct_clean' thread > runs in background as well. > > Github actions: https://github.com/grivet/ovs/actions/runs/574446345 > > Gaetan Rivet (9): > conntrack: Use rcu-lists to store conn expirations > conntrack: Use a cmap to store zone limits > conntrack: Init hash basis first at creation > conntrack-tp: Use a cmap to store timeout policies > conntrack: Inverse conn and ct lock precedence > conntrack: Do not schedule zero ms timers > conntrack: Do not rate limit ct-sweep > conntrack: Do not log empty ct-sweep > conntrack: Use an atomic conn expiration value > > lib/conntrack-private.h | 96 +++++++++++++------ > lib/conntrack-tp.c | 137 +++++++++++++++----------- > lib/conntrack.c | 206 +++++++++++++++++++++++++--------------- > lib/conntrack.h | 4 +- > lib/dpif-netdev.c | 5 +- > 5 files changed, 280 insertions(+), 168 deletions(-) > > -- > 2.30.0 > > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev >
The v2 of this series has been sent: https://patchwork.ozlabs.org/project/openvswitch/list/?series=240241 -- Gaetan Rivet _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
