Hi Chuan, are cores 20 and 22 on socket0 or socket1? If they are on socket1, the application is crashing because the aesni_mb driver is pointing to socket0: vdev crypto_aesni_mb0,socket_id=0.
George From: [email protected] <[email protected]> On Behalf Of Chuan Han via Lists.Fd.Io Sent: Monday, November 04, 2019 10:27 AM To: vpp-dev <[email protected]> Cc: [email protected] Subject: [vpp-dev] Is there a limit when assigning corelist-workers in vpp? Hi, vpp experts, I am trying to allocate more cores to a phy nic. I want to allocate cores 4,6,8,10 to eth0, and cores 12,14,16,18 to eth1. cpu { main-core 2 # corelist-workers 4,6,8,10,12,14,16,18,20,22 <== This does not work. vpp crashes when starting. corelist-workers 4,6,8,10,12,14,16,18 } dpdk { socket-mem 2048,0 log-level debug no-tx-checksum-offload dev default{ num-tx-desc 512 num-rx-desc 512 } dev 0000:1a:00.0 { # workers 4,6,8,10,12 workers 4,6,8,10 name eth0 } dev 0000:19:00.1 { # workers 14,16,18,20,22 workers 12,14,16,18 name eth1 } # Use aesni mb lib. vdev crypto_aesni_mb0,socket_id=0 # Use qat VF pcie addresses. # dev 0000:3d:01.0 no-multi-seg } Afte vpp starts, I can see eth1 got 4 cores but eth0 only got 3 cores. vpp# sh thread ID Name Type LWP Sched Policy (Priority) lcore Core Socket State 0 vpp_main 10653 other (0) 2 0 0 1 vpp_wk_0 workers 10655 other (0) 4 1 0 2 vpp_wk_1 workers 10656 other (0) 6 4 0 3 vpp_wk_2 workers 10657 other (0) 8 2 0 4 vpp_wk_3 workers 10658 other (0) 10 3 0 5 vpp_wk_4 workers 10659 other (0) 12 8 0 6 vpp_wk_5 workers 10660 other (0) 14 13 0 7 vpp_wk_6 workers 10661 other (0) 16 9 0 8 vpp_wk_7 workers 10662 other (0) 18 12 0 <=== core 18 is not used by eth0. vpp# sh interface rx-placement Thread 1 (vpp_wk_0): node dpdk-input: eth1 queue 0 (polling) Thread 2 (vpp_wk_1): node dpdk-input: eth1 queue 1 (polling) Thread 3 (vpp_wk_2): node dpdk-input: eth1 queue 2 (polling) Thread 4 (vpp_wk_3): node dpdk-input: eth1 queue 3 (polling) Thread 5 (vpp_wk_4): node dpdk-input: eth0 queue 0 (polling) eth0 queue 2 (polling) Thread 6 (vpp_wk_5): node dpdk-input: eth0 queue 3 (polling) Thread 7 (vpp_wk_6): node dpdk-input: eth0 queue 1 (polling) vpp# It seems there is a limitation on assigning cores to nic. 1. I cannot allocate cores after 20 to corelist-workers. 2. Cores after 18 cannot be allocated to nic. Is this some bug? Or, some undocumented limitation? Thanks. Chuan
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14490): https://lists.fd.io/g/vpp-dev/message/14490 Mute This Topic: https://lists.fd.io/mt/41334952/21656 Group Owner: [email protected] Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
