From: Levente Csikor <[email protected]>
Date: Sunday, May 7, 2017 at 6:31 AM
To: Darrell Ball <[email protected]>, "[email protected]"
<[email protected]>
Subject: Re: [ovs-discuss] OVS 2.7.0 + DPDK 17.02 + multicore
Thanks, it's working now! The 'interface' was the keyword indeed!
Do I need to do the affinity ``trick'' as well?
I mean the part similar to after '\' in the following:
ovs-vsctl set interface dpdk-p0 options:n_rxq=4 \
other_config:pmd-rxq-affinity="0:3,1:7,3:8"
I realized that without this OVS automatically assigns cores/threads to the
queues, however when I explicitly set it (like above), the output of
'ovs-appctl dpif-netdev/pmd-rxq-show' says for those threads that they are
isolated (isolated: true).
Otherwise, the value of isolated is false.
Does it have any (significant) impact on the performance or on anything else?
“Isolated” is just an attribute of a PMD and means the PMD will only poll a
specified set of queues.
Affinity could be used to enforce more predictability (in terms of throughout
and latency) for those queues.
Cheers
On 05/07/2017 01:12 AM, Darrell Ball wrote:
From:
<[email protected]><mailto:[email protected]>
on behalf of Levente Csikor <[email protected]><mailto:[email protected]>
Date: Saturday, May 6, 2017 at 3:36 AM
To: "[email protected]"<mailto:[email protected]>
<[email protected]><mailto:[email protected]>
Subject: [ovs-discuss] OVS 2.7.0 + DPDK 17.02 + multicore
Dear All,
I would like to run OVS with DPDK using multiple cores by multiple poll-mode
driver threads to fine tune the performance and achieve higher packet rate.
I realized that since version 2.5.0 the multiple core assignment changed a bit,
and I followed the instructions at
http://docs.openvswitch.org/en/latest/intro/install/dpdk/<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.openvswitch.org_en_latest_intro_install_dpdk_&d=DwMDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=L_sO_555UZCkLTLLfgsvpixjfPj6uRLy9AJhNbSppMA&s=5qBNp8XzYYYmUz6gGXR0eYxcQ3QIjEF3jU526iCJe00&e=>
Everything is properly installed and set with respect to hugepages, dpdk
drivers, etc.
Here is the relevant part of my init-script, for instance, in case of 3 cores:
sudo $OVS_PATH/utilities/ovs-vsctl --no-wait set Open_vSwitch .
other_config:dpdk-init=true
sudo $OVS_PATH/utilities/ovs-vsctl --no-wait set Open_vSwitch .
other_config:dpdk-lcore-mask=e
sudo $OVS_PATH/utilities/ovs-vsctl --no-wait set Open_vSwitch .
other_config:dpdk-socket-mem="4096,0"
sudo $OVS_PATH/utilities/ovs-vsctl --no-wait set Open_vSwitch .
other_config:dpdk-hugepage-dir="/mnt/huge"
sudo $OVS_PATH/utilities/ovs-vsctl --no-wait set Open_vSwitch .
other_config:pmd-cpu-mask=e
sudo $OVS_PATH/utilities/ovs-vsctl --no-wait set Open_vSwitch .
other_config:n-rxq=3
*************
I guess you want “options : n_rxq” in Interface table
see
http://openvswitch.org/support/dist-docs/ovs-vswitchd.conf.db.5.pdf<https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_support_dist-2Ddocs_ovs-2Dvswitchd.conf.db.5.pdf&d=DwMDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=3jDNcjtjrAEW8uHiSQEpQ8wq18Wqmzww-elLHB8A3-U&s=R2ZvfMV-uexWNNs3kOvrbfbXl_6X6N_YKhVVbtwIV4M&e=>
also, you should read
http://docs.openvswitch.org/en/latest/howto/dpdk/<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.openvswitch.org_en_latest_howto_dpdk_&d=DwMDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=3jDNcjtjrAEW8uHiSQEpQ8wq18Wqmzww-elLHB8A3-U&s=4Rtw6dRrXrG4vsf12kGPzhnYGLVbHoV4pWiqc39Q4pU&e=>
*************
sudo $OVS_PATH/vswitchd/ovs-vswitchd unix:$DB_SOCK --pidfile --detach
I also tried the older version of rx_queue setup that worked for version 2.5.0:
sudo $OVS_PATH/utilities/ovs-vsctl set Open_vSwitch . other_config:n-dpdk-rxqs=3
Open vSwitch starts, but after installing numerous flow rules, sending
different packets with different L2/L3/L4 headers (to foster RSS), it did not
scale up. It still used only 1 core.
Linux command top showed only 100% CPU load instead of 300%, or even when I
checked /proc/cpuinfo it also showed that only 1 core is scaled up to the
maximum CPU frequency.
(Furthermore, if the outputs of these programs were wrong after all, I check
the packet rate as well, but it did not change unfortunately)
Would anyone be so kind as to further assist me towards using multiple cores?
What did I miss/do wrong?
Thank you in advance,
Levi
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
