Hi,
These patches were initially included in a joint GSO-GRO series [1]
implementing both TX and RX batching mechanisms in the ovpn data path.
Sashiko's review of v2 of that series identified fundamental flaws in
the GRO part which required a broad redesign of the new RX path. So I
decided to proceed with the GSO series separately [2] and keep this
series GRO-specific since the two mechanisms are not functionally
dependent. The series applies both on its own and on top of [2].
The redesign uses the standard SKB_GSO_UDP_L4 | SKB_GSO_FRAGLIST
representation instead of giving an ovpn aggregate tunnel-specific
completion semantics. This requires UDP to let an encapsulation callback
opt into consuming that representation, while retaining a
segment-and-redeliver fallback when the callback is absent or declines
it. It also requires detachable GRO callbacks: unlike a conventional
kernel-owned tunnel socket, ovpn attaches to a userspace socket which
can outlive its association with the driver. The first patch fixes a
pre-existing UDP frag-list GRO checksum problem on CHECKSUM_COMPLETE
devices which the new receive path would otherwise hit; it carries a
Fixes tag and does not depend on the rest of the series. The next three
provide those core contracts and make UDP_ENCAP changes retire
detachable callbacks before replacing the encapsulation state. Clearly,
these four patches are more suited to netdev than openvpn-devel, but in
order to keep this series applicable and reviewable, they are included
in this openvpn-devel-targeted series. Patch 2 explains why this opt-in
does not reopen commit 3d010c8031e3 and shows that consuming the
aggregate directly is 22.7% to 25.2% faster than letting UDP segment it
first.
On receive, the ovpn GRO callback collects compatible DATA_V2 records
from one outer transport flow, matching their opcode, key and peer
header while retaining every complete encrypted record as a separate
frag-list entry. The aggregate continues through the ordinary outer
receive stack. At the encapsulation callback, ovpn validates its layout,
ownership, headers and outer tuple before detaching the records,
propagating post-GRO metadata and authenticating each one independently.
Any aggregate which ovpn cannot accept whole, such as a foreign or
modified one, is declined, causing UDP to segment it and retry delivery
one datagram at a time. The final patch uses the resulting lookahead to
prefetch ciphertext two records before it is decrypted.
Performance was measured with the GSO series [2] applied solely to keep
transmit from limiting the receive comparison. Five interleaved
60-second iperf3 -Z single-flow AES-128-GCM runs in each direction,
after a 10-second warm-up, over two directly connected 100-Gbit/s mlx5
ports produced:
Forward Reverse
Without GRO 18.721 Gbit/s 18.777 Gbit/s
With GRO 21.542 Gbit/s 21.366 Gbit/s
With GRO + RX prefetch 25.741 Gbit/s 25.150 Gbit/s
The complete series improved throughput by 37.5% forward and 33.9%
reverse. Across the monitored data-path CPUs, mean cycles per byte fell
from 4.535 to 3.896 (-14.1%) and instructions per byte from 10.913 to
9.492 (-13.0%). Individual patch messages contain the incremental
measurements and further test details.
Thanks,
Ralf Lici
Mandelbit Srl
[1]
https://lore.kernel.org/openvpn-devel/[email protected]/
[2]
https://lore.kernel.org/openvpn-devel/[email protected]/
---
Changes since v2
https://lore.kernel.org/openvpn-devel/[email protected]/
- Point the Fixes tag of the checksum fix at the commit which stopped
converting CHECKSUM_COMPLETE, and explain why only the checksum state
can carry the GRO verdict. (Sashiko)
- Restore the pre-encapsulation checksum bypass for accepted aggregates,
since a tunnel can reset their checksum state. (Sashiko)
- Refuse records inside FOU or GUE in the ovpn GRO callback. (Sashiko)
- Rebase on current ovpn-net-next.
Changes since v1
https://lore.kernel.org/openvpn-devel/[email protected]/
- Add a UDP fix which turns CHECKSUM_COMPLETE frag-list aggregates into
CHECKSUM_UNNECESSARY at GRO completion, and drop the pre-encapsulation
checksum bypass which it makes redundant. (Sashiko)
- Run the inbound XFRM policy check and conntrack reset once for an
accepted aggregate, so that a declined one is not checked again
without that state, and count every datagram of a dropped aggregate.
(Sashiko)
- Read frag-list acceptance with acquire ordering and warn when setting
up a socket which still accepts frag-lists. (Sashiko)
- Require a detachable GRO socket to have no registered GRO callback,
and refuse to attach ovpn to a socket which another encapsulation has
configured. (Sashiko)
- Read a held aggregate's OpenVPN header without pulling it, and never
pull an aggregate before deciding whether to consume it. (Sashiko)
- Clear the redirect, classification and delivery time state which the
receive stack consumed on the aggregate head only. (Sashiko)
Ralf Lici (6):
udp: keep the GRO checksum verdict for CHECKSUM_COMPLETE frag-lists
udp: let encapsulation callbacks consume UDP frag-list GSO
udp_tunnel: support detachable GRO receive callbacks
udp: detach tunnel GRO before changing UDP encapsulation
ovpn: coalesce UDP data records with GRO
ovpn: prefetch encrypted records before GRO batch decryption
drivers/net/ovpn/udp.c | 368 +++++++++++++++++++++++++++++++++++--
include/linux/udp.h | 18 ++
include/net/udp_tunnel.h | 3 +
net/core/gro.c | 1 +
net/ipv4/udp.c | 79 ++++++--
net/ipv4/udp_offload.c | 67 ++++++-
net/ipv4/udp_tunnel_core.c | 37 +++-
net/ipv6/udp.c | 73 ++++++--
net/ipv6/udp_offload.c | 8 +
9 files changed, 608 insertions(+), 46 deletions(-)
base-commit: 7b6ab4dbc156e2f96419715130b3d5738a306c93
--
2.56.0
_______________________________________________
Openvpn-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openvpn-devel
