With the current OVS offload design, when an offload-device fails to add a flow rule and returns an error, OVS adds the rule to the kernel datapath. The flow gets processed by the kernel datapath for the entire life of that flow. This is fine when an error is returned by the device due to lack of support for certain keys or actions.
But when an error is returned due to temporary conditions such as lack of resources to add a flow rule, the flow continues to be processed by kernel even when resources become available later. That is, those flows never get offloaded again. This problem becomes more pronounced when a flow that has been initially offloaded may have a smaller packet rate than a later flow that could not be offloaded due to lack of resources. This leads to inefficient use of HW resources and wastage of host CPU cycles. This patch-set addresses this issue by providing a way to detect temporary offload resource constraints (Out-Of-Resource or OOR condition) and to selectively and dynamically offload flows with a higher packets-per-second (pps) rate. This dynamic rebalancing is done periodically on netdevs that are in OOR state until resources become available to offload all pending flows. The patch-set involves the following changes at a high level: 1. Detection of Out-Of-Resources (OOR) condition on an offload-capable netdev. 2. Gathering flow offload selection criteria for all flows on an OOR netdev; i.e, packets-per-second (pps) rate of flows for offloaded and non-offloaded (pending) flows. 3. Dynamically replacing offloaded flows with a lower pps-rate, with non-offloaded flows with a higher pps-rate, on an OOR netdev. 4. A new OpenvSwitch configuration option - "offload-rebalancing" to enable this policy. ****** v3-->v4: - Updated parse_flow_put() with the following changes: - Fixed outdev memory leak with multiple output actions - Moved variables closer to their first use - Removed outdev check while setting oor, since indev is sufficient v2-->v3: - Removed some VLOG_DBG() in patches 2 and 3 - Reworded a few VLOG_DBG() in patch 3 - Fixed a comment line in patch 3 v1-->v2: - Fixed build errors reported by 0-day robot - Updated patch prefixes with relevant subsystem names ****** Sriharsha Basavapatna (4): dpif-netlink: Detect Out-Of-Resource condition on a netdev revalidator: Gather packets-per-second rate of flows revalidator: Rebalance offloaded flows based on the pps rate netdev: Add a configuration option to enable dynamic rebalancing of flows lib/dpif-netdev.c | 3 +- lib/dpif-netlink.c | 25 +- lib/dpif-provider.h | 8 +- lib/dpif.c | 20 +- lib/dpif.h | 20 +- lib/flow.c | 27 ++ lib/flow.h | 1 + lib/netdev-provider.h | 8 + lib/netdev.c | 45 +++ lib/netdev.h | 2 + ofproto/ofproto-dpif-upcall.c | 571 +++++++++++++++++++++++++++++++++- vswitchd/vswitch.xml | 22 ++ 12 files changed, 732 insertions(+), 20 deletions(-) -- 2.18.0.rc1.1.g6f333ff _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev