Hi Ian, > As you are setting all lcore and pmd core to node 1 why are you giving 1024 memory to node 0? > When processing packets for this port it means cpu is accessing data across the numa nodes which causes a performance penalty I am benchmarking performance in different settings and trying to understand the roles of OVS and DPDK in mediating core affinity of pmds and hugepage utilization. Your answer helps a lot!
> try using ‘other_config:dpdk-socket-mem=0,4096’ and see if you still see the issue. But this warning should appear regardless the socket-mem allocation right? If my understanding is correct, when OVS pmd's are pinned to 10-19 and the VM tespmd app is pinned to core 2; the OVSpmd thread running on node 1 is having to access a huge-page on node 0 which VMtestpmd happens to access as well. Thanks, Alan 2018-04-12 18:28 GMT+02:00 Stokes, Ian <[email protected]>: > Hi, > > > > I was able to reproduce the issue on my system. > > > > As you are setting all lcore and pmd core to node 1 why are you giving > 1024 memory to node 0? > > > > I saw the same issue on my system but the warning did not appear once > memory was allocated to node 1 only. > > > > I would think the VM being launched is using memory for the vhost port > from node 0, however the queue for the vhost port is assigned to core 14 > which is on node1. When processing packets for this port it means cpu is > accessing data across the numa nodes which causes a performance penalty, > hence the warning. > > > > To avoid you should ensure all memory and cores operate on the same node > where possible, try using ‘other_config:dpdk-socket-mem=0,4096’ and see > if you still see the issue. > > > > Thanks > > Ian > > > > *From:* [email protected] [mailto:ovs-discuss-bounces@ > openvswitch.org] *On Behalf Of *Alan Kayahan > *Sent:* Thursday, April 12, 2018 2:27 AM > *To:* [email protected] > *Subject:* [ovs-discuss] There's no available (non-isolated) pmd thread > on numa node 0, Expect reduced performance. > > > > Hello, > > > > On the following setup, where all cores but 0 are isolated, > > > > available: 2 nodes (0-1) > > node 0 cpus: 0 1 2 3 4 5 6 7 8 9 > > node 1 cpus: 10 11 12 13 14 15 16 17 18 19 > > > > I am trying to start OVS entirely on numa node 1 as following > > > > ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true > other_config:dpdk-lcore-mask=0x00400 other_config:pmd-cpu-mask=0xffc00 > other_config:dpdk-socket-mem=1024,4096 > > > > However when I create a vhost port SRC to attach a VNF(via virtio) on node > 0, I get the following > > > > dpif_netdev|WARN|There's no available (non-isolated) pmd thread on numa > node 0. Queue 0 on port 'SRC' will be assigned to the pmd on core 14 (numa > node 1). Expect reduced performance. > > > > Any ideas? > > > > Thanks >
_______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
