Hi,
This series is a respin of earlier work that combined GSO and GRO
batching across ovpn's data path. After submitting v2, Sashiko
demolished the GRO part by pointing out various lifetime and ownership
issues that both I and previous Sashiko review rounds had missed. So
this series now carries only the GSO patches, with the TX-specific
findings addressed. The GRO work will be submitted separately since
there's no functional dependency between the two batching mechanisms.
Another major difference from v2 is the exclusion of asynchronous crypto
backends from the new GSO data path. This proved necessary after
carefully re-reading the DMA API documentation and realizing that
adjacent records in the output UDP GSO skb can share cache lines.
Independently mapped asynchronous DMA operations cannot safely use that
layout on non-coherent systems. Keeping asynchronous transforms on the
existing per-record path also addresses Sabrina's concern about
preserving aggregate lifetime across asynchronous crypto completions.
This series lets GSO skbs reach ovpn, completes each inner packet's
checksum during software segmentation and, when the layout permits,
encrypts the resulting packets directly into a UDP GSO aggregate.
Ordinary packets, GSO_BY_FRAGS input, unsupported segment geometries and
traffic using asynchronous crypto retain the existing in-place
encryption path. Packet IDs for a UDP GSO batch are reserved as a single
range instead of one segment at a time. The last two patches also
construct each record's AAD directly in its output range when possible
and reuse one AEAD request and its scratch storage across the synchronous
batch.
Performance was measured with iperf3 -Z using one TCP flow over UDP ovpn
with AES-128-GCM, a 1408-byte tunnel MTU and a 1500-byte physical MTU
between two directly connected 100-Gbit/s mlx5 ports. Each 25-second run
omitted the first 5 seconds, with a 15-second perf sample covering the
data-path CPUs. Six runs of each tested commit were interleaved
round-robin. Comparing the state immediately before patch 5 with the
final series produced:
Without UDP GSO Final series
Throughput 13.431 Gbit/s 16.932 Gbit/s (+26.1%)
Cycles per byte 6.628 4.680 (-29.4%)
Instructions per byte 13.871 10.354 (-25.4%)
Once UDP GSO output was introduced, throughput remained on the same
roughly 17 Gbit/s plateau within run-to-run variation. The subsequent
patches reduced data-path cost from 4.899 to 4.680 cycles per byte
(-4.5%) and from 10.653 to 10.354 instructions per byte (-2.8%).
To verify that this was not only a hardware-segmentation win, the main
conversion was also tested in four order-balanced pairs with
tx-udp-segmentation disabled on the lower device. Mean throughput rose
from 13.647 to 16.278 Gbit/s (+19.3%) and cycles per byte fell from
6.609 to 6.199 (-6.2%), despite instructions per byte increasing from
13.858 to 14.582 (+5.2%) because the lower device segments the aggregate
in software. The patches making individual performance claims carry
their corresponding measurements.
Thanks,
Ralf Lici
Mandelbit Srl
---
Changes since v2
https://lore.kernel.org/openvpn-devel/[email protected]/
- Drop the GRO and receive-prefetch patches; they will be submitted as a
separate series.
- Drop transmit prefetch after new measurements found no repeatable
benefit.
- Restrict UDP GSO output to synchronous AEAD transforms.
- Let fixed-size frag-list GSO input use the UDP GSO output path; only
GSO_BY_FRAGS input remains on the in-place path.
- Replace asynchronous batch-completion machinery with serial
synchronous encryption, and reuse one AEAD request and its scratch
storage per batch.
- Count peer packet statistics for every segment which survives GSO
segmentation rather than once for the original GSO skb. (Sashiko)
- Transfer socket write ownership with the input skb's original
destructor so TCP Small Queues wakeups remain correct. (Sashiko)
- Preserve timestamp and ooo_okay metadata on the replacement aggregate.
- Reset the replacement skb's inner network header so tunnel byte
accounting excludes reserved headroom, and document that device
tx_packets counts output GSO skbs. (Sashiko)
- Use an unaligned-safe store when constructing AAD directly in the
output aggregate. (Sashiko)
Changes since v1
https://lore.kernel.org/openvpn-devel/[email protected]/
- Release successfully encrypted GSO source skbs with consume_skb after
their data has been transferred to the output aggregate. (Antonio)
- Complete a held GRO aggregate before delivering an unmergeable record,
and let a DATA_V2 record with a changed header start the next
aggregate. (Antonio)
- Document that RX prefetch primarily targets synchronous software AEAD;
asynchronous transforms are not expected to benefit in the same way.
- Add direct placement of record AAD in the output aggregate, retaining
the scratch-and-copy path as a fallback.
Changes since RFC v4
https://lore.kernel.org/openvpn-devel/[email protected]/
- Drop the opt-in direct GRO mode and the generic GRO ownership change
it required. The series now retains the complete outer receive stack.
- Preserve safe page-backed payload fragments during GSO segmentation
and map them directly into the AEAD source scatterlist.
- Reserve one consecutive packet-ID range for each UDP GSO batch instead
of updating the counter once per segment.
- Extend transmit prefetch to the first payload fragment of segments
encrypted into a UDP GSO aggregate.
- Refresh the GSO, GRO and prefetch measurements using two directly
connected 100-Gbit/s mlx5 ports.
Ralf Lici (8):
ovpn: advertise checksum offload for GSO packets
ovpn: accept frag-list GSO input
ovpn: refactor AEAD encryption helpers
ovpn: record asynchronous transmit crypto
ovpn: convert GSO input into UDP GSO output
ovpn: reserve packet-ID ranges for UDP GSO batches
ovpn: place GSO record AAD directly in output
ovpn: reuse one AEAD request per UDP GSO batch
drivers/net/ovpn/crypto.h | 2 +
drivers/net/ovpn/crypto_aead.c | 226 +++++++++++++++++----
drivers/net/ovpn/crypto_aead.h | 21 ++
drivers/net/ovpn/io.c | 357 ++++++++++++++++++++++++++++-----
drivers/net/ovpn/io.h | 3 +-
drivers/net/ovpn/main.c | 5 +-
drivers/net/ovpn/pktid.h | 48 +++++
drivers/net/ovpn/proto.h | 4 +
drivers/net/ovpn/stats.h | 16 +-
drivers/net/ovpn/tcp.c | 4 +-
drivers/net/ovpn/udp.c | 26 ++-
11 files changed, 608 insertions(+), 104 deletions(-)
base-commit: 734bef548a96e8774d6eef5dfa2026c6edc4e80b
--
2.56.0
_______________________________________________
Openvpn-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openvpn-devel
