When RTNL is contended, network-related control-plane operations can be
delayed.
In such scenario, if OVS control-plane operations (such as vport
creation) take a bit longer, it's acceptable. However, flow installation
operations happen as part of upcall processing, executed in the context
of of handler threads. If they get delayed, it affects the data-plane
and can even result in packet drops.
Because flow operations also use ovs_mutex for concurrency protection and
given RTNL can nest under ovs_mutex, contention can be easily transferred
from RTNL to ovs_mutex, causing delay in flow operations.
In order to protect flow operations from RTNL delays, this series
decouples them from ovs_mutex. First, the flow_table is converted into an
rcu-protected pointer. Then two locking mechanisms are introduced: a
per-table mutex and a refcount.
The mutex protects the flow_table against concurrent modifications,
while the refcount is used to extend the lifetime of the flow_table
beyond the rcu read-protected region used to dereference it.
The following is an example of how concurrent datapath deletion and
flow modifcation would work.
Datapath deletion:
ovs_lock();
table = rcu_dereference_protected(dp->table, ...);
rcu_assign_pointer(dp->table, NULL);
ovs_flow_tbl_put(table);
ovs_unlock();
Flow modification:
rcu_read_lock();
dp = get_dp(...);
table = rcu_dereference(dp->table);
ovs_flow_tbl_get(table);
rcu_read_unlock();
mutex_lock(&table->lock);
/* Perform modifications on the flow_table */
mutex_unlock(&table->lock);
ovs_flow_tbl_put(table);
v3:
- Split in 2 patches (Paolo)
- Improve locking in get_dp_stats (Paolo and Sashiko)
- Use __always_unused in lockdep stubs (Paolo)
- Use READ_ONCE/WRITE_ONCE for table->count (Aaron)
- Take a reference in ovs_dp_masks_rebalance (Aaron)
v2: Fix argument in ovs_flow_tbl_put (sparse)
Remove rcu checks in ovs_dp_masks_rebalance
Adrian Moreno (2):
net: openvswitch: make flow_table an rcu pointer
net: openvswitch: decouple flow_table from ovs_mutex
net/openvswitch/datapath.c | 293 ++++++++++++++++++++++++-----------
net/openvswitch/datapath.h | 2 +-
net/openvswitch/flow.c | 13 +-
net/openvswitch/flow.h | 9 +-
net/openvswitch/flow_table.c | 194 ++++++++++++++---------
net/openvswitch/flow_table.h | 56 ++++++-
6 files changed, 396 insertions(+), 171 deletions(-)
--
2.53.0
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
