Hi Team, We are planning to use VPP for throughput enhancement with our Applications. However I found a bottleneck in TX side. So our application is placed like this VPP RX Interface--> VPP Memif --> Our Application --> VPPCOM UDP session --> VPP TX Interface. The RX part we are not seeing issue and we could able to reach expected throughput. However with TX we are suffering to push same amount of traffic out. After further investigation I found, the TX sessions are only on Worker 1, These are not getting distributed among other threads, vpp# show sess verbose Connection State Rx-f Tx-f [0:0][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:7428LISTEN 0 0 [0:1][U] 2001:5b0:ffff:1150:b883:31f:678:98f8:7428LISTEN 0 0 [0:2][U] 2001:5b0:ffff:1150:b883:31f:378:98f8:7428LISTEN 0 0 [0:3][U] 2001:5b0:ffff:1150:b883:31f:278:98f8:7428LISTEN 0 0 [0:4][U] 2001:5b0:ffff:1150:b883:31f:578:98f8:7428LISTEN 0 0 [0:5][U] 2001:5b0:ffff:1150:b883:31f:478:98f8:7428LISTEN 0 0 Thread 0: active sessions 6
Connection State Rx-f Tx-f *[1:0][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:2194OPENED 0 0* *[1:1][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:2753OPENED 0 0* *[1:2][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:5163OPENED 0 0* *[1:3][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:6022OPENED 0 0* *[1:4][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:1272OPENED 0 0* *[1:5][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:1222OPENED 0 0* *[1:6][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:3080OPENED 0 0* *[1:7][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:8598OPENED 0 0* *[1:8][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:2764OPENED 0 0* *[1:9][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:1567OPENED 0 0* *[1:10][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:243OPENED 0 0* *[1:11][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:458OPENED 0 0* Thread 1: active sessions 12 Thread 2: no sessions Connection State Rx-f Tx-f [3:0][U] 2001:5b0:ffff:1150:b883:31f:178:98f8:7428OPENED 0 0 [3:1][U] 2001:5b0:ffff:1150:b883:31f:678:98f8:7428OPENED 0 0 Thread 3: active sessions 2 Connection State Rx-f Tx-f [4:0][U] 2001:5b0:ffff:1150:b883:31f:278:98f8:7428OPENED 0 0 [4:1][U] 2001:5b0:ffff:1150:b883:31f:578:98f8:7428OPENED 0 0 Thread 4: active sessions 2 Thread 5: no sessions Connection State Rx-f Tx-f [6:0][U] 2001:5b0:ffff:1150:b883:31f:378:98f8:7428OPENED 0 0 [6:1][U] 2001:5b0:ffff:1150:b883:31f:478:98f8:7428OPENED 0 0 Thread 6: active sessions 2 That is creating Bottleneck, Please find our RX placement, vpp# show interface rx-placement Thread 1 (vpp_wk_0): node memif-input: memif15/0 queue 0 (polling) memif16/0 queue 0 (polling) Thread 2 (vpp_wk_1): node memif-input: memif2/0 queue 0 (polling) memif4/0 queue 0 (polling) Thread 3 (vpp_wk_2): node memif-input: memif3/0 queue 0 (polling) memif14/0 queue 0 (polling) node dpdk-input: HundredGigabitEthernet12/0/0 queue 2 (polling) Thread 4 (vpp_wk_3): node memif-input: memif1/0 queue 0 (polling) memif12/0 queue 0 (polling) node dpdk-input: HundredGigabitEthernet12/0/0 queue 3 (polling) Thread 5 (vpp_wk_4): node memif-input: memif11/0 queue 0 (polling) memif13/0 queue 0 (polling) node dpdk-input: HundredGigabitEthernet12/0/0 queue 0 (polling) Thread 6 (vpp_wk_5): node memif-input: memif5/0 queue 0 (polling) memif6/0 queue 0 (polling) node dpdk-input: HundredGigabitEthernet12/0/0 queue 1 (polling) vpp# We made sure the RX queues are on not on worker 1 and 0, there is slight improvement but not upto the mark, So our question is, 1> How can we distribute these TX sessions among different workers? 2> Or How can we attain maximum with one thread? Thanks in advance for reading and answering
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17508): https://lists.fd.io/g/vpp-dev/message/17508 Mute This Topic: https://lists.fd.io/mt/77078709/21656 Mute #vnet:https://lists.fd.io/g/vpp-dev/mutehashtag/vnet Mute #vppcom:https://lists.fd.io/g/vpp-dev/mutehashtag/vppcom Mute #txbuff:https://lists.fd.io/g/vpp-dev/mutehashtag/txbuff Group Owner: [email protected] Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
