On 7/31/26 11:13, Alexander Lobakin wrote:
From: Matt Vollrath <[email protected]>
Date: Fri, 31 Jul 2026 08:41:06 -0400
In a few places libeth and the iavf driver inspect which page pool is
attached to a netmem_ref. This forces an immediate load of the
netmem_desc struct for each buffer in the Rx loop.
Defer or eliminate these loads of netmem_desc from the driver fast path
by using the page_pool ref in the first cache line of iavf_ring instead.
I was thinking of this when implementing the current design, but:
Do you have any data to prove this actually helps performance?
The optimization was profile-driven, but not on an iavf. I was working on
a libeth port of e1000e, because I'm an obsessive optimizer, and this quirk
of the interface fell out. I'm working with a Xeon E3-1240 v3 and I218-V.
With a mirror of the changes in this patch series, on my platform, it's a
net zero overall change in Rx performance. This is not surprising, because
the cold read is just deferred until later. The time saved in
eth_type_trans (where the payload is first read) is instead spent in
skb_gro_receive and pool puts.
The additional thing that does make an improvement, which was omitted from
this series, is prefetching netmem_desc early in the loop. This makes about
22 cycles/packet, 3% difference.
I left it where it is because I don't have hardware to perf this on, and
didn't know if anyone would even be interested in looking at something that
I can't produce data for on the actual hardware. For iavf the prefetch
would be conditional, only when the EOP flag is set, because in this case
netmem_desc will always be needed either for GRO merge or pool put.
I can't guarantee that this will be worth 3% on an iavf platform. In any
case, it'll be a couple weeks before I can work on this again.
There are only two paths out of the Rx loop where netmem_desc needs to
be consumed:
* The "very rare" case of libeth_rx_sync_for_cpu calling
libeth_rx_recycle_slow and indicating that there was no data. This
could be similarly factored out, but not by this series.
* GRO merging a frame into an existing aggregate stream. In this case,
the cold load of netmem_desc may overlap the payload prefetch started
by iavf_build_skb, which is now no longer dependent on netmem_desc to
start.
I see now this isn't right, anything going to GRO will end up back in the
pool in the same iteration, and netmem_desc will be loaded. It just
happens later.
Thanks,
Olek
