This RFC patch series introduces a major architectural
refactoring of Open vSwitch's hardware offload
infrastructure. It replaces the tightly coupled
`netdev-offload` implementation with a new, modular
`dpif-offload-provider` framework.
MOTIVATION
-------------------------------------------------------------
The existing `netdev-offload` API tightly couples datapath
implementations (like `dpif-netdev`) with specific offload
technologies (DPDK's rte_flow). This design has several
limitations:
- Rigid Architecture: It creates complex dependencies,
making the code difficult to maintain and extend.
- Limited Flexibility: Supporting multiple offload backends
simultaneously or adding new ones is cumbersome.
- Inconsistent APIs: The logic for handling different
offload types is scattered, leading to an inconsistent
and hard-to-follow API surface.
This refactoring aims to resolve these issues by creating a
clean separation of concerns, improving modularity, and
establishing a clear path for future hardware offload
integrations.
PROPOSED SOLUTION: THE `DPIF-OFFLOAD-PROVIDER` FRAMEWORK
-------------------------------------------------------------
This series introduces the `dpif-offload-provider`
framework, which functions similarly to the existing
`dpif-provider` pattern. It treats hardware offload as a
distinct layer with multiple, dynamically selectable
backends.
Key features of the new framework include:
1. Modular Architecture: A clean separation between the
generic datapath interface and specific offload
provider implementations (e.g., `dpif-offload-tc`,
`dpif-offload-dpdk`). `dpif` layers are now generic
clients of the offload API.
2. Provider-based System: Allows multiple offload backends
to coexist.
3. Unified and Asynchronous API: Establishes a consistent
API across all offload providers. For userspace
datapaths, the API is extended to support asynchronous
flow operations with callbacks, making `dpif-netdev` a
more efficient client.
4. Enhanced Configuration: Provides granular control over
offload provider selection through a global and per-port
priority system (`hw-offload-priority`), allowing
fine-tuned policies for different hardware.
5. Improved Testing: Includes a new test framework
specifically for validating DPDKs rte_flow offloads,
enhancing long-term maintainability.
PATCH SERIES ORGANIZATION
-------------------------------------------------------------
This large series is organized logically to facilitate
review:
1. Framework Foundation: The initial patches establish the
core `dpif-offload-provider` framework, including the
necessary APIs for port management, flow mark
allocation, configuration, and a dummy provider for
testing.
2. Provider Implementation: These patches introduce the new
`dpif-offload-tc` and `dpif-offload-dpdk` providers,
building out their specific implementations on top of
the new framework.
3. API Migration and Decoupling: The bulk of the series
systematically migrates functionality from the legacy
`netdev-offload` layer to the new providers. Key
commits here decouple `dpif-netdev` and, crucially,
`dpif-netlink` from their hardware offload
entanglements.
4. Cleanup: The final patches remove the now-redundant
global APIs and structures from `netdev-offload`,
completing the transition.
BACKWARD COMPATIBILITY
-------------------------------------------------------------
This refactoring maintains full API compatibility from a
user's perspective. All existing `ovs-vsctl` and
`ovs-appctl` commands continue to function as before. The
changes are primarily internal architectural improvements
designed to make OVS more robust and extensible.
REQUEST FOR COMMENTS
-------------------------------------------------------------
This is a significant architectural change that affects
core OVS infrastructure. We welcome feedback on:
- The overall architectural approach and the
`dpif-offload-provider` concept.
- The API design, particularly the new asynchronous model
for `dpif-netdev`.
- The migration strategy and any potential backward
compatibility concerns.
- Performance implications of the new framework.
-------------------------------------------------------------
¯Changes from v1:
- Fixed issues reported by Aaron and my AI experiments.
- See individual patches for specific changes.
Eelco Chaudron (41):
dpif-offload-provider: Add dpif-offload-provider implementation.
dpif-offload: Add provider for tc offload.
dpif-offload: Add provider for dpdk (rte_flow).
dpif-offload: Allow configuration of offload provider priority.
dpif-offload: Move hw-offload configuration to dpif-offload.
dpif-offload: Add offload provider set_config API.
dpif-offload: Add port registration and management APIs.
dpif-offload-tc: Add port management framework.
dpif-offload-dpdk: Add port management framework.
dpif-offload: Validate mandatory port class callbacks on registration.
dpif-offload: Allow per-port offload provider priority config.
dpif-offload: Introduce provider debug information API.
dpif-offload: Call flow-flush netdev-offload APIs via dpif-offload.
dpif-offload: Call meter netdev-offload APIs via dpif-offload.
dpif-offload: Move the flow_get_n_flows() netdev-offload API to dpif.
dpif-offload: Move hw_miss_packet_postprocess netdev API to dpif.
dpif-offload: Add flow dump APIs to dpif-offload.
dpif-offload: Move the tc flow dump netdev APIs to dpif-offload.
dpif-netlink: Remove netlink-offload integration.
dpif-netlink: Add API to get offloaded netdev from port_id.
dpif-offload: Add API to find offload implementation type.
dpif-offload: Add operate implementation to dpif-offload.
netdev-offload: Temporarily move thread-related APIs to dpif-netdev.
dpif-offload: Add port dump APIs to dpif-offload.
dpif-netdev: Remove indirect DPDK netdev offload API calls.
dpif: Add dpif_get_features() API.
dpif-offload: Add flow operations to dpif-offload-tc.
dpif-netlink: Remove entangled hardware offload.
dpif-offload-tc: Remove netdev-offload dependency.
netdev_dummy: Remove hardware offload override.
dpif-offload: Move the netdev_any_oor() API to dpif-offload.
netdev-offload: Remove the global netdev-offload API.
dpif-offload: Add inline flow APIs for userspace datapaths.
dpif_netdev: Fix nullable memcpy in queue_netdev_flow_put().
dpif-offload: Move offload_stats_get() API to dpif-offload.
dpif-netdev: Add full name to the dp_netdev structure.
dpif-offload-dpdk: Abstract rte_flow implementation from dpif-netdev.
dpif-offload-dummy: Add flow add/del/get APIs.
netdev-offload: Fold netdev-offload APIs and files into dpif-offload.
tests: Fix NSH decap header test for real Ethernet devices.
tests: Add a simple DPDK rte_flow test framework.
Documentation/topics/testing.rst | 19 +
include/openvswitch/json.h | 1 +
include/openvswitch/netdev.h | 1 +
lib/automake.mk | 17 +-
lib/dp-packet.h | 1 +
lib/dpctl.c | 50 +-
lib/dpdk.c | 2 -
lib/dpif-netdev-avx512.c | 4 +-
lib/dpif-netdev-private-flow.h | 9 +-
lib/dpif-netdev.c | 1246 +++---------
lib/dpif-netlink.c | 557 +----
...load-dpdk.c => dpif-offload-dpdk-netdev.c} | 592 ++++--
lib/dpif-offload-dpdk-private.h | 73 +
lib/dpif-offload-dpdk.c | 1182 +++++++++++
lib/dpif-offload-dummy.c | 882 ++++++++
lib/dpif-offload-provider.h | 421 ++++
...-offload-tc.c => dpif-offload-tc-netdev.c} | 262 ++-
lib/dpif-offload-tc-private.h | 76 +
lib/dpif-offload-tc.c | 872 ++++++++
lib/dpif-offload.c | 1789 +++++++++++++++++
lib/dpif-offload.h | 221 ++
lib/dpif-provider.h | 65 +-
lib/dpif.c | 145 +-
lib/dpif.h | 14 +-
lib/dummy.h | 9 +
lib/json.c | 7 +
lib/netdev-dpdk.c | 9 +-
lib/netdev-dpdk.h | 2 +-
lib/netdev-dummy.c | 199 +-
lib/netdev-linux.c | 3 +-
lib/netdev-offload-provider.h | 148 --
lib/netdev-offload.c | 910 ---------
lib/netdev-offload.h | 169 --
lib/netdev-provider.h | 10 +-
lib/netdev.c | 70 +-
lib/netdev.h | 22 +
lib/tc.c | 2 +-
ofproto/ofproto-dpif-upcall.c | 47 +-
ofproto/ofproto-dpif.c | 90 +-
tests/.gitignore | 3 +
tests/automake.mk | 24 +
tests/dpif-netdev.at | 21 +-
tests/ofproto-dpif.at | 170 ++
tests/ofproto-macros.at | 17 +-
tests/sendpkt.py | 12 +-
tests/system-dpdk-offloads-macros.at | 236 +++
tests/system-dpdk-offloads-testsuite.at | 28 +
tests/system-dpdk-offloads.at | 223 ++
tests/system-dpdk.at | 35 +
tests/system-kmod-macros.at | 5 +
tests/system-offloads-testsuite-macros.at | 5 +
tests/system-offloads-traffic.at | 48 +
tests/system-traffic.at | 9 +-
tests/system-userspace-macros.at | 5 +
vswitchd/bridge.c | 7 +-
vswitchd/vswitch.xml | 43 +
56 files changed, 7753 insertions(+), 3336 deletions(-)
rename lib/{netdev-offload-dpdk.c => dpif-offload-dpdk-netdev.c} (83%)
create mode 100644 lib/dpif-offload-dpdk-private.h
create mode 100644 lib/dpif-offload-dpdk.c
create mode 100644 lib/dpif-offload-dummy.c
create mode 100644 lib/dpif-offload-provider.h
rename lib/{netdev-offload-tc.c => dpif-offload-tc-netdev.c} (94%)
create mode 100644 lib/dpif-offload-tc-private.h
create mode 100644 lib/dpif-offload-tc.c
create mode 100644 lib/dpif-offload.c
create mode 100644 lib/dpif-offload.h
delete mode 100644 lib/netdev-offload-provider.h
delete mode 100644 lib/netdev-offload.c
delete mode 100644 lib/netdev-offload.h
create mode 100644 tests/system-dpdk-offloads-macros.at
create mode 100644 tests/system-dpdk-offloads-testsuite.at
create mode 100644 tests/system-dpdk-offloads.at
--
2.50.1
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
