I can't help thinking that there needs to be a more dynamic heuristic. Keep existing upper limit but only cache as much as gets used.
There are two main usage models. One is fast direct recycling which run to completion generates. The other is when pipeline is used and arrival and departure is on different cores. On Mon, Jul 20, 2026, 10:13 Bruce Richardson <[email protected]> wrote: > On Sat, Jul 18, 2026 at 04:07:05PM +0200, Morten Brørup wrote: > > > From: Kishore Padmanabha [mailto:[email protected]] > > > Sent: Friday, 17 July 2026 19.10 > > > > > > Hi Bruce, > > > > > > The below patch works fine for us. We tested all the different packet > > > sizes. > > > Thanks for the patch. Do you want to push this patch since it is not > > > changing the ABI/API? > > > > Too late in the release process. > > > > Let's postpone the discussion for DPDK 26.11, where API/ABI breakage is > allowed. > > Agreed. Let's not unnecessarily rush this. > > > > > I'm not strongly opposed to Bruce's algorithm, targeting a fill level of > 25 % from the edges and flushing/refilling up to 75 % of the cache when > necessary. It does have its advantages for some mempool access patterns > (which are not exotic). > > I just prefer the current algorithm, targeting a fill level of 50 % and > only flushing/refilling up to 50 % of the cache when necessary. It performs > better at random get/put access patterns, and the backend transactions are > smaller. > > > > For DPDK 26.11, where we can break the API/ABI, we can simply double > RTE_MEMPOOL_CACHE_MAX_SIZE to 1024, to compensate for reducing the > effective cache size from 150 % to 100 %. > > The mempool cache objs array will no longer be > [RTE_MEMPOOL_CACHE_MAX_SIZE * 2], but only [RTE_MEMPOOL_CACHE_MAX_SIZE], so > doubling RTE_MEMPOOL_CACHE_MAX_SIZE will not increase the memory footprint, > but allow using a cache size up to 1024. > > My concern with this approach is that it won't automatically fix the > problem if we have users who experience a performance regression due to the > mempool changes. While testpmd allows the mbcache size to be provided via > parameter, end applications are likely to have it hardcoded. That means > that if an app does experience a regression, the author/user has to be > either aware of the mempool changes, or has to debug it down to the mempool > and then know to increase the mempool cache size in the app. > > It's not an insurmountable problem, but one that needs to be very clearly > called out in our documentation, what the change is, how it may affect > things and how to fix it. > > On the other hand, in realworld, i.e. not just testpmd/l3fwd cases doing > little packet processesing, I would be fairly hopeful that regressions are > going to be few and very small. > > > > > Please also note that the current implementation is carefully designed > to keep the transfers to/from the mempool backend CPU cache aligned > (assuming cache->size is 2^N and large enough). > > Refer to the parameters passed to rte_mempool_ops_enqueue/dequeue_bulk(). > > E.g. with mempool cache size 256, backend transfers are 128 objects, 16 > full cache lines. > > Using CPU cache aligned transfers has a few advantages: > > - There are no cache line ownership issues across different CPU cores > repeatedly accessing the backend. > > - The mempool backend drivers can be performance optimized for > transferring full CPU cache lines. (Both source and destination addresses, > and number of objects copied, are CPU cache aligned. Assuming all transfers > go via the mempool cache.) > > > > These details should be fine tuned in the implementation, if we do > proceed with Bruce's algorithm. > > > Yep, good points. > > /Bruce >
