Hello mlx5 guys, On Fri, 12 Jun 2026 at 09:14, Xingui Yang <[email protected]> wrote: > > From: Shuaisong Yang <[email protected]> > > On Kunpeng SoC with mlx CX7, dpdk-l3fwd with intra-NUMA core pinning > under SMMU nonstrict/strict mode shows about 30% performance degradation > compared to cross-NUMA pinning. With SMMU disabled or passthrough mode, > intra-NUMA performs as expected (slightly better than cross-NUMA). > > CX7 in NUMA1 > NUMA node0 CPU(s): 0-39 > NUMA node1 CPU(s): 40-79 > > intra-NUMA: > dpdk-l3fwd -l 40-55 -n 4 -a 0000:17:00.1,mprq_en=1 -- -p 0x1 -P \ > --config='(0,0,40),(0,1,41),(0,2,42),(0,3,43),(0,4,44),\ > (0,5,45),(0,6,46),(0,7,47),(0,8,48),(0,9,49),\ > (0,10,50),(0,11,51),(0,12,52),(0,13,53),\ > (0,14,54),(0,15,55)' \ > --rx-queue-size=4096 --tx-queue-size=4096 --rx-burst=64 > > cross-NUMA: > dpdk-l3fwd -l 11-26 -n 4 -a 0000:17:00.1,mprq_en=1 -- -p 0x1 -P \ > --config='(0,0,11),(0,1,12),(0,2,13),(0,3,14),(0,4,15),\ > (0,5,16),(0,6,17),(0,7,18),(0,8,19),(0,9,20),\ > (0,10,21),(0,11,22),(0,12,23),(0,13,24),\ > (0,14,25),(0,15,26)' \ > --rx-queue-size=4096 --tx-queue-size=4096 --rx-burst=64 > > The root cause is that under SMMU enabled mode, the mempool allocated > for intra-NUMA pinning is aligned to system page size instead of > hugepage size, while cross-NUMA pinning correctly uses hugepage size > alignment. This causes high TLB miss rates under SMMU. > > Align all memory ranges to hugepage boundaries during mempool > registration to ensure hugepage_sz alignment, thereby reducing TLB > misses and fixing the intra-NUMA performance degradation. > > Fixes: 690b2a88c2f7 ("common/mlx5: add mempool registration facilities") > Cc: [email protected] > > Signed-off-by: Shuaisong Yang <[email protected]> > Signed-off-by: Xingui Yang <[email protected]>
Review please. -- David Marchand
