This RFC patch series introduces a major architectural refactoring of Open vSwitch's hardware offload infrastructure. It replaces the tightly coupled `netdev-offload` implementation with a new, modular `dpif-offload-provider` framework.
MOTIVATION ------------------------------------------------------------- The existing `netdev-offload` API tightly couples datapath implementations (like `dpif-netdev`) with specific offload technologies (rte_flow). This design has several limitations: - Rigid Architecture: It creates complex dependencies, making the code difficult to maintain and extend. - Limited Flexibility: Supporting multiple offload backends simultaneously or adding new ones is cumbersome. - Inconsistent APIs: The logic for handling different offload types is scattered, leading to an inconsistent and hard-to-follow API surface. This refactoring aims to resolve these issues by creating a clean separation of concerns, improving modularity, and establishing a clear path for future hardware offload integrations. PROPOSED SOLUTION: THE `DPIF-OFFLOAD-PROVIDER` FRAMEWORK ------------------------------------------------------------- This series introduces the `dpif-offload-provider` framework, which functions similarly to the existing `dpif-provider` pattern. It treats hardware offload as a distinct layer with multiple, dynamically selectable backends. Key features of the new framework include: 1. Modular Architecture: A clean separation between the generic datapath interface and specific offload provider implementations (e.g., `dpif-offload-tc`, `dpif-offload-rte_flow`). `dpif` layers are now generic clients of the offload API. 2. Provider-based System: Allows multiple offload backends to coexist. 3. Unified and Asynchronous API: Establishes a consistent API across all offload providers. For userspace datapaths, the API is extended to support asynchronous flow operations with callbacks, making `dpif-netdev` a more efficient client. 4. Enhanced Configuration: Provides granular control over offload provider selection through a global and per-port priority system (`hw-offload-priority`), allowing fine-tuned policies for different hardware. 5. Improved Testing: Includes a new test framework specifically for validating RTE_Flow offloads, enhancing long-term maintainability. PATCH SERIES ORGANIZATION ------------------------------------------------------------- This large series is organized logically to facilitate review: 1. Framework Foundation: The initial patches establish the core `dpif-offload-provider` framework, including the necessary APIs for port management, flow mark allocation, configuration, and a dummy provider for testing. 2. Provider Implementation: These patches introduce the new `dpif-offload-tc` and `dpif-offload-rte_flow` providers, building out their specific implementations on top of the new framework. 3. API Migration and Decoupling: The bulk of the series systematically migrates functionality from the legacy `netdev-offload` layer to the new providers. Key commits here decouple `dpif-netdev` and, crucially, `dpif-netlink` from their hardware offload entanglements. 4. Cleanup: The final patches remove the now-redundant global APIs and structures from `netdev-offload`, completing the transition. BACKWARD COMPATIBILITY ------------------------------------------------------------- This refactoring maintains full API compatibility from a user's perspective. All existing `ovs-vsctl` and `ovs-appctl` commands continue to function as before. The changes are primarily internal architectural improvements designed to make OVS more robust and extensible. REQUEST FOR COMMENTS ------------------------------------------------------------- This is a significant architectural change that affects core OVS infrastructure. We welcome feedback on: - The overall architectural approach and the `dpif-offload-provider` concept. - The API design, particularly the new asynchronous model for `dpif-netdev`. - The migration strategy and any potential backward compatibility concerns. - Performance implications of the new framework. ------------------------------------------------------------- v2: - Fixed some minor AI review comments (see individual patches) - Fix some Coverity issues reported on this patch set. - Fixed and investigated some rte_flow unit test. Eelco Chaudron (41): dpif-offload-provider: Add dpif-offload-provider implementation. dpif-offload: Add provider for tc offload. dpif-offload: Add provider for rte_flow. dpif-offload: Allow configuration of offload provider priority. dpif-offload: Move hw-offload configuration to dpif-offload. dpif-offload: Add offload provider set_config API. dpif-offload: Add port registration and management APIs. dpif-offload-tc: Add port management framework. dpif-offload-rte_flow: Add port management framework. dpif-offload: Validate mandatory port class callbacks on registration. dpif-offload: Allow per-port offload provider priority config. dpif-offload: Introduce provider debug information API. dpif-offload: Call flow-flush netdev-offload APIs via dpif-offload. dpif-offload: Call meter netdev-offload APIs via dpif-offload. dpif-offload: Move the flow_get_n_flows() netdev-offload API to dpif. dpif-offload: Move the hw_miss_packet_recover netdev API to dpif. dpif-offload: Add flow dump APIs to dpif-offload. dpif-offload: Move the tc flow dump netdev APIs to dpif-offload. dpif-netlink: Remove netlink-offload integration. dpif-netlink: Add API to get offloaded netdev from port_id. dpif-offload: Add API to find offload implementation type. dpif-offload: Add operate implementation to dpif-offload. netdev-offload: Temporarily move thread-related APIs to dpif-netdev. dpif-offload: Add port dump APIs to dpif-offload. dpif-netdev: Remove indirect DPDK netdev offload API calls. dpif: Add dpif_get_features() API. dpif-offload: Add flow operations to dpif-offload-tc. dpif-netlink: Remove entangled hardware offload. dpif-offload-tc: Remove netdev-offload dependency. netdev_dummy: Remove hardware offload override. dpif-offload: Move the netdev_any_oor() API to dpif-offload. netdev-offload: Remove the global netdev-offload API. dpif-offload: Add flow mark allocation API. dpif-offload: Add inline flow APIs for userspace datapaths. dpif_netdev: Fix nullable memcpy in queue_netdev_flow_put(). dpif-offload: Move offload_stats_get() API to dpif-offload. dpif-netdev: Add full name to the dp_netdev structure. dpif-offload-rte: Abstract rte_flow implementation from dpif-netdev. dpif-offload-dummy: Add flow add/del/get APIs. tests: Fix NSH decap header test for real Ethernet devices. tests: Add a simple rte_flow test framework. Documentation/topics/testing.rst | 19 + include/openvswitch/json.h | 1 + include/openvswitch/netdev.h | 1 + lib/automake.mk | 12 +- lib/dpctl.c | 50 +- lib/dpdk.c | 2 - lib/dpif-netdev-avx512.c | 3 +- lib/dpif-netdev-private-flow.h | 5 - lib/dpif-netdev.c | 1090 +++---------- lib/dpif-netlink.c | 556 +------ lib/dpif-offload-dummy.c | 642 ++++++++ lib/dpif-offload-provider.h | 399 +++++ lib/dpif-offload-rte_flow-private.h | 34 + lib/dpif-offload-rte_flow.c | 1228 ++++++++++++++ lib/dpif-offload-tc.c | 873 ++++++++++ lib/dpif-offload.c | 1805 +++++++++++++++++++++ lib/dpif-offload.h | 214 +++ lib/dpif-provider.h | 65 +- lib/dpif.c | 139 +- lib/dpif.h | 14 +- lib/dummy.h | 9 + lib/json.c | 7 + lib/netdev-dpdk.c | 9 +- lib/netdev-dpdk.h | 2 +- lib/netdev-dummy.c | 199 +-- lib/netdev-linux.c | 3 +- lib/netdev-offload-dpdk.c | 178 +- lib/netdev-offload-dpdk.h | 51 + lib/netdev-offload-provider.h | 148 -- lib/netdev-offload-tc.c | 261 +-- lib/netdev-offload-tc.h | 76 + lib/netdev-offload.c | 838 +--------- lib/netdev-offload.h | 118 -- lib/netdev-provider.h | 9 +- lib/netdev.c | 18 +- lib/netdev.h | 1 + ofproto/ofproto-dpif-upcall.c | 47 +- ofproto/ofproto-dpif.c | 89 +- tests/.gitignore | 3 + tests/automake.mk | 24 + tests/dpif-netdev.at | 22 +- tests/ofproto-dpif.at | 164 ++ tests/ofproto-macros.at | 17 +- tests/system-dpdk.at | 35 + tests/system-kmod-macros.at | 5 + tests/system-offloads-testsuite-macros.at | 5 + tests/system-offloads-traffic.at | 44 + tests/system-rte-offloads-macros.at | 230 +++ tests/system-rte-offloads-testsuite.at | 28 + tests/system-rte-offloads.at | 111 ++ tests/system-traffic.at | 8 +- tests/system-userspace-macros.at | 5 + vswitchd/bridge.c | 6 +- vswitchd/vswitch.xml | 42 + 54 files changed, 6874 insertions(+), 3090 deletions(-) create mode 100644 lib/dpif-offload-dummy.c create mode 100644 lib/dpif-offload-provider.h create mode 100644 lib/dpif-offload-rte_flow-private.h create mode 100644 lib/dpif-offload-rte_flow.c create mode 100644 lib/dpif-offload-tc.c create mode 100644 lib/dpif-offload.c create mode 100644 lib/dpif-offload.h create mode 100644 lib/netdev-offload-dpdk.h delete mode 100644 lib/netdev-offload-provider.h create mode 100644 lib/netdev-offload-tc.h create mode 100644 tests/system-rte-offloads-macros.at create mode 100644 tests/system-rte-offloads-testsuite.at create mode 100644 tests/system-rte-offloads.at -- 2.50.1 _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
