All even number cores are on numa 0, which also hosts all nics. It seems corelist-workers can only take maximum 8 cores.
On Mon, Nov 4, 2019 at 9:45 AM Tkachuk, Georgii <[email protected]> wrote: > Hi Chuan, are cores 20 and 22 on socket0 or socket1? If they are on > socket1, the application is crashing because the aesni_mb driver is > pointing to socket0: vdev crypto_aesni_mb0,socket_id=0. > > > > George > > > > *From:* [email protected] <[email protected]> *On Behalf Of *Chuan > Han via Lists.Fd.Io > *Sent:* Monday, November 04, 2019 10:27 AM > *To:* vpp-dev <[email protected]> > *Cc:* [email protected] > *Subject:* [vpp-dev] Is there a limit when assigning corelist-workers in > vpp? > > > > Hi, vpp experts, > > > > I am trying to allocate more cores to a phy nic. I want to allocate cores > 4,6,8,10 to eth0, and cores 12,14,16,18 to eth1. > > > > cpu { > main-core 2 > * # corelist-workers 4,6,8,10,12,14,16,18,20,22 <== This does not work. > vpp crashes when starting. * > corelist-workers 4,6,8,10,12,14,16,18 > } > > dpdk { > socket-mem 2048,0 > log-level debug > no-tx-checksum-offload > dev default{ > num-tx-desc 512 > num-rx-desc 512 > } > dev 0000:1a:00.0 { > # workers 4,6,8,10,12 > workers 4,6,8,10 > name eth0 > } > dev 0000:19:00.1 { > # workers 14,16,18,20,22 > workers 12,14,16,18 > name eth1 > } > # Use aesni mb lib. > vdev crypto_aesni_mb0,socket_id=0 > # Use qat VF pcie addresses. > # dev 0000:3d:01.0 > no-multi-seg > } > > > > Afte vpp starts, I can see eth1 got 4 cores but eth0 only got 3 cores. > > > > vpp# sh thread > ID Name Type LWP Sched Policy (Priority) > lcore Core Socket State > 0 vpp_main 10653 other (0) 2 > 0 0 > 1 vpp_wk_0 workers 10655 other (0) 4 > 1 0 > 2 vpp_wk_1 workers 10656 other (0) 6 > 4 0 > 3 vpp_wk_2 workers 10657 other (0) 8 > 2 0 > 4 vpp_wk_3 workers 10658 other (0) 10 > 3 0 > 5 vpp_wk_4 workers 10659 other (0) 12 > 8 0 > 6 vpp_wk_5 workers 10660 other (0) 14 > 13 0 > 7 vpp_wk_6 workers 10661 other (0) 16 > 9 0 > *8 vpp_wk_7 workers 10662 other (0) > 18 12 0 <=== core 18 is not used by eth0.* > vpp# sh interface rx-placement > Thread 1 (vpp_wk_0): > node dpdk-input: > eth1 queue 0 (polling) > Thread 2 (vpp_wk_1): > node dpdk-input: > eth1 queue 1 (polling) > Thread 3 (vpp_wk_2): > node dpdk-input: > eth1 queue 2 (polling) > Thread 4 (vpp_wk_3): > node dpdk-input: > eth1 queue 3 (polling) > > > > *Thread 5 (vpp_wk_4): node dpdk-input: eth0 queue 0 (polling) > eth0 queue 2 (polling)* > Thread 6 (vpp_wk_5): > node dpdk-input: > eth0 queue 3 (polling) > Thread 7 (vpp_wk_6): > node dpdk-input: > eth0 queue 1 (polling) > vpp# > > > > It seems there is a limitation on assigning cores to nic. > > 1. I cannot allocate cores after 20 to corelist-workers. > > 2. Cores after 18 cannot be allocated to nic. > > > > Is this some bug? Or, some undocumented limitation? > > > > Thanks. > > Chuan >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14491): https://lists.fd.io/g/vpp-dev/message/14491 Mute This Topic: https://lists.fd.io/mt/41334952/21656 Group Owner: [email protected] Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
