[ovs-discuss] ovs-dpdk performance
Folks,
I have configured ovs-dpdk to replace sriov deployment for bonding
support. everything good but somehow as soon as i start hitting
200kpps rate i start seeing packet drop.
I have configured CPU isolation as per documentation to assign a
dedicated pmd thread. I have assigned 8 dedicated PMD threads but
still performance is very poor.
I created an 8vCPU vm on openstack using dpdk and running some
workload using an in-house application, during the 200kpps packet rate
I noticed my all PMD cpu showing high CPU processing cycles.
$ ovs-vswitchd -V
ovs-vswitchd (Open vSwitch) 2.13.3
DPDK 19.11.7
In the following output what does these queue-id:0 to 8 and why only
the first queue is in use but not others, they are always zero. What
does this mean?
ovs-appctl dpif-netdev/pmd-rxq-show
pmd thread numa_id 0 core_id 2:
isolated : false
port: vhu1c3bf17a-01queue-id: 0 (enabled) pmd usage: 0 %
port: vhu1c3bf17a-01queue-id: 1 (enabled) pmd usage: 0 %
port: vhu6b7daba9-1aqueue-id: 2 (disabled) pmd usage: 0 %
port: vhu6b7daba9-1aqueue-id: 3 (disabled) pmd usage: 0 %
pmd thread numa_id 1 core_id 3:
isolated : false
pmd thread numa_id 0 core_id 22:
isolated : false
port: vhu1c3bf17a-01queue-id: 3 (enabled) pmd usage: 0 %
port: vhu1c3bf17a-01queue-id: 6 (enabled) pmd usage: 0 %
port: vhu6b7daba9-1aqueue-id: 0 (enabled) pmd usage: 54 %
port: vhu6b7daba9-1aqueue-id: 5 (disabled) pmd usage: 0 %
pmd thread numa_id 1 core_id 23:
isolated : false
port: dpdk1 queue-id: 0 (enabled) pmd usage: 3 %
pmd thread numa_id 0 core_id 26:
isolated : false
port: vhu1c3bf17a-01queue-id: 2 (enabled) pmd usage: 0 %
port: vhu1c3bf17a-01queue-id: 7 (enabled) pmd usage: 0 %
port: vhu6b7daba9-1aqueue-id: 1 (disabled) pmd usage: 0 %
port: vhu6b7daba9-1aqueue-id: 4 (disabled) pmd usage: 0 %
pmd thread numa_id 1 core_id 27:
isolated : false
pmd thread numa_id 0 core_id 46:
isolated : false
port: dpdk0 queue-id: 0 (enabled) pmd usage: 27 %
port: vhu1c3bf17a-01queue-id: 4 (enabled) pmd usage: 0 %
port: vhu1c3bf17a-01queue-id: 5 (enabled) pmd usage: 0 %
port: vhu6b7daba9-1aqueue-id: 6 (disabled) pmd usage: 0 %
port: vhu6b7daba9-1aqueue-id: 7 (disabled) pmd usage: 0 %
pmd thread numa_id 1 core_id 47:
isolated : false
$ ovs-appctl dpif-netdev/pmd-stats-clear && sleep 10 && ovs-appctl
dpif-netdev/pmd-stats-show | grep "processing cycles:"
processing cycles: 1697952 (0.01%)
processing cycles: 12726856558 (74.96%)
processing cycles: 4259431602 (19.40%)
processing cycles: 512666 (0.00%)
processing cycles: 6324848608 (37.81%)
Does processing cycles mean my PMD is under stress? but i am only
hitting 200kpps rate?
This is my dpdk0 and dpdk1 port statistics
sudo ovs-vsctl get Interface dpdk0 statistics
{flow_director_filter_add_errors=153605,
flow_director_filter_remove_errors=30829, mac_local_errors=0,
mac_remote_errors=0, ovs_rx_qos_drops=0, ovs_tx_failure_drops=0,
ovs_tx_invalid_hwol_drops=0, ovs_tx_mtu_exceeded_drops=0,
ovs_tx_qos_drops=0, rx_128_to_255_packets=64338613,
rx_1_to_64_packets=367, rx_256_to_511_packets=116298,
rx_512_to_1023_packets=31264, rx_65_to_127_packets=6990079,
rx_broadcast_packets=0, rx_bytes=12124930385, rx_crc_errors=0,
rx_dropped=0, rx_errors=12, rx_fcoe_crc_errors=0, rx_fcoe_dropped=12,
rx_fcoe_mbuf_allocation_errors=0, rx_fragment_errors=367,
rx_illegal_byte_errors=0, rx_jabber_errors=0, rx_length_errors=0,
rx_mac_short_packet_dropped=128, rx_management_dropped=35741,
rx_management_packets=31264, rx_mbuf_allocation_errors=0,
rx_missed_errors=0, rx_oversize_errors=0, rx_packets=71512362,
rx_priority0_dropped=0, rx_priority0_mbuf_allocation_errors=1096,
rx_priority1_dropped=0, rx_priority1_mbuf_allocation_errors=0,
rx_priority2_dropped=0, rx_priority2_mbuf_allocation_errors=0,
rx_priority3_dropped=0, rx_priority3_mbuf_allocation_errors=0,
rx_priority4_dropped=0, rx_priority4_mbuf_allocation_errors=0,
rx_priority5_dropped=0, rx_priority5_mbuf_allocation_errors=0,
rx_priority6_dropped=0, rx_priority6_mbuf_allocation_errors=0,
rx_priority7_dropped=0, rx_priority7_mbuf_allocation_errors=0,
rx_undersize_errors=6990079, tx_128_to_255_packets=64273778,
tx_1_to_64_packets=128, tx_256_to_511_packets=43670294,
tx_512_to_1023_packets=153605, tx_65_to_127_packets=881272,
tx_broadcast_packets=10, tx_bytes=25935295292, tx_dropped=0,
tx_errors=0, tx_management_packets=0, tx_multicast_packets=153,
tx_packets=109009906}
sudo ovs-vsctl get Interface dpdk1 statistics
{flow_director_filter_add_errors=126793,
flow_director_filter_remove_errors=37969, mac_local_errors=0,
mac_remote_errors=0, ovs_rx_qos_drops=0, ovs_tx_failure_drops=0,
ovs_tx_invalid_hwol_drops=0, ovs_tx_mtu_exceeded_drops=0,
ovs_tx_qos_drops=0, rx_128_to_255_packets=64435459,
rx_1_to_64_packets=107843, rx_256_to_511_packets=230,
rx_512_to_1023_packets=13, rx_65_
[ovs-discuss] 回复: discuss Digest, Vol 148, Issue 36
ort-update --no-security-groups [port uuid]
neutron port-update --port_security_enabled=false [port uuid]
What I found:
When try to build_lswitch_arp_nd_responder_known_ips in ovn northd, it will
skip LSP, which has unknow flag.
static void
build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op,
struct hmap *lflows,
struct hmap *ports,
struct ds *actions,
struct ds *match)
{
...
if (lsp_is_external(op->nbsp) || op->has_unknown) {
return;
}
? Windows ???<https://go.microsoft.com/fwlink/?LinkId=550986>??
-- next part --
An HTML attachment was scrubbed...
URL:
<http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20211029/82dde6e4/attachment-0001.html>
--
Message: 2
Date: Fri, 29 Oct 2021 09:58:33 +0200
From: Daniel Alvarez Sanchez
To: ? ?
Cc: "b...@openvswitch.org"
Subject: Re: [ovs-discuss] OVN LSP with a unknown in address will not
build arp response lflows
Message-ID:
Content-Type: text/plain; charset="utf-8"
Hi,
On Fri, Oct 29, 2021 at 5:50 AM ? ? wrote:
> *Environment info:*
> OVN 21.06
>
> OVS 2.12.0
>
> *Reproduction:*
> 1. Create a port with neutronclient assign it to a node and close port
> security group
>
> 2. Create a ovs port and add it to br-int, and set interface iface-id same
> as neutron port uuid
>
> After it Neutron will create a LSP in OVN NB, and append unknow into LSP?s
> address field
>
> Check it in script[1]
>
>
>
> Port info:
> ()[root@ovn-tool-0 /]# ovn-nbctl find Logical_Switch_Port
> name=6a8064f9-f2cc-407d-b8da-345c6a216cb3
>
> _uuid : 88fd1a84-8695-4cef-b916-45531edaf0db
>
> addresses : ["fa:16:3e:b3:c0:e5 192.168.111.42", unknown]
>
> dhcpv4_options : 1a8ca1af-519c-4aa2-b3a3-cc74955dee1f
>
> dhcpv6_options : []
>
> dynamic_addresses : []
>
> enabled : true
>
> external_ids: {"neutron:cidrs"="192.168.111.42/24",
> "neutron:device_id"="", "neutron:device_owner"="",
> "neutron:network_name"=neutron-6ac00688-422f-4a4f-99ae-b092b2d87f7b,
> "neutron:port_name"=lc-tap-2,
> "neutron:project_id"="498e2a96e4cc4edeb0c525a081dd6830",
> "neutron:revision_number"="4", "neutron:security_group_ids"=""}
>
> ha_chassis_group: []
>
> name: "6a8064f9-f2cc-407d-b8da-345c6a216cb3"
>
> options : {mcast_flood_reports="true",
> requested-chassis=node-1.domain.tld}
>
> parent_name : []
>
> port_security : []
>
> tag : []
>
> tag_request : []
>
> type: ""
>
> up : false
>
>
>
> *Results:*
> OVN will not build arp responder lfows for this LSP
>
I believe that this is the expected behavior as you disable port security,
meaning that the traffic from that port can come from any MAC address (it's
unknown to OVN). Hence, it is up to the VM/container/whatever to reply to
ARP requests and OVN should not reply on its behalf.
Hope this helps.
Thanks!
daniel
>
> *Script:*
>
> [1]:
>
> #!/usr/bin/bash
>
>
>
> # Create port
>
> # neutron port-create --name lucheng-tap
> --binding:host_id=node-3.domain.tld share_net
>
>
>
> HOST=""
>
> MAC=""
>
>
>
> get_port_info() {
>
> source openrc
>
> port_id="$1"
>
> HOST=$(neutron port-show -F binding:host_id -f value "$port_id")
>
> MAC=$(neutron port-show -F mac_address -f value "$port_id")
>
> ip_info=$(neutron port-show -F fixed_ips -f value "$port_id")
>
> echo Port "$port_id" Mac: "$MAC" HOST: "$HOST"
>
> echo IP Info: "$ip_info"
>
> }
>
>
>
> create_ns() {
>
> port_id="$1"
>
> iface_name="lc-tap-${port_id:0:8}"
>
> netns_name="lc-vm-${port_id:0:8}"
>
> ssh "$HOST" ovs-vsctl add-port br-int "$iface_name" \
>
> -- set Interface "$iface_name" type=internal \
>
> -- set Interface "$iface_name" external_ids:iface-id="$port_id" \
>
> -- set Interface "$iface_name" external_ids:attached-mac="$MAC" \
>
> -- set Interface "$iface_name" external_ids:iface-status=active
&g
Re: [ovs-discuss] OVN LSP with a unknown in address will not build arp response lflows
Hi,
On Fri, Oct 29, 2021 at 5:50 AM 鲁 成 wrote:
> *Environment info:*
> OVN 21.06
>
> OVS 2.12.0
>
> *Reproduction:*
> 1. Create a port with neutronclient assign it to a node and close port
> security group
>
> 2. Create a ovs port and add it to br-int, and set interface iface-id same
> as neutron port uuid
>
> After it Neutron will create a LSP in OVN NB, and append unknow into LSP’s
> address field
>
> Check it in script[1]
>
>
>
> Port info:
> ()[root@ovn-tool-0 /]# ovn-nbctl find Logical_Switch_Port
> name=6a8064f9-f2cc-407d-b8da-345c6a216cb3
>
> _uuid : 88fd1a84-8695-4cef-b916-45531edaf0db
>
> addresses : ["fa:16:3e:b3:c0:e5 192.168.111.42", unknown]
>
> dhcpv4_options : 1a8ca1af-519c-4aa2-b3a3-cc74955dee1f
>
> dhcpv6_options : []
>
> dynamic_addresses : []
>
> enabled : true
>
> external_ids: {"neutron:cidrs"="192.168.111.42/24",
> "neutron:device_id"="", "neutron:device_owner"="",
> "neutron:network_name"=neutron-6ac00688-422f-4a4f-99ae-b092b2d87f7b,
> "neutron:port_name"=lc-tap-2,
> "neutron:project_id"="498e2a96e4cc4edeb0c525a081dd6830",
> "neutron:revision_number"="4", "neutron:security_group_ids"=""}
>
> ha_chassis_group: []
>
> name: "6a8064f9-f2cc-407d-b8da-345c6a216cb3"
>
> options : {mcast_flood_reports="true",
> requested-chassis=node-1.domain.tld}
>
> parent_name : []
>
> port_security : []
>
> tag : []
>
> tag_request : []
>
> type: ""
>
> up : false
>
>
>
> *Results:*
> OVN will not build arp responder lfows for this LSP
>
I believe that this is the expected behavior as you disable port security,
meaning that the traffic from that port can come from any MAC address (it's
unknown to OVN). Hence, it is up to the VM/container/whatever to reply to
ARP requests and OVN should not reply on its behalf.
Hope this helps.
Thanks!
daniel
>
> *Script:*
>
> [1]:
>
> #!/usr/bin/bash
>
>
>
> # Create port
>
> # neutron port-create --name lucheng-tap
> --binding:host_id=node-3.domain.tld share_net
>
>
>
> HOST=""
>
> MAC=""
>
>
>
> get_port_info() {
>
> source openrc
>
> port_id="$1"
>
> HOST=$(neutron port-show -F binding:host_id -f value "$port_id")
>
> MAC=$(neutron port-show -F mac_address -f value "$port_id")
>
> ip_info=$(neutron port-show -F fixed_ips -f value "$port_id")
>
> echo Port "$port_id" Mac: "$MAC" HOST: "$HOST"
>
> echo IP Info: "$ip_info"
>
> }
>
>
>
> create_ns() {
>
> port_id="$1"
>
> iface_name="lc-tap-${port_id:0:8}"
>
> netns_name="lc-vm-${port_id:0:8}"
>
> ssh "$HOST" ovs-vsctl add-port br-int "$iface_name" \
>
> -- set Interface "$iface_name" type=internal \
>
> -- set Interface "$iface_name" external_ids:iface-id="$port_id" \
>
> -- set Interface "$iface_name" external_ids:attached-mac="$MAC" \
>
> -- set Interface "$iface_name" external_ids:iface-status=active
>
>
>
> ssh "$HOST" ip netns add "$netns_name"
>
> ssh "$HOST" ip l set dev "$iface_name" address "$MAC"
>
> ssh "$HOST" ip l set "$iface_name" netns "$netns_name"
>
> ssh "$HOST" ip netns exec "$netns_name" ip l set lo up
>
> ssh "$HOST" ip netns exec "$netns_name" ip l set "$iface_name" up
>
> }
>
>
>
> main() {
>
> get_port_info "$1"
>
> create_ns "$1"
>
> }
>
>
>
> main $@
>
> neutron port-update --no-security-groups [port uuid]
>
> neutron port-update --port_security_enabled=false [port uuid]
>
>
>
> *What I found:*
>
> When try to build_lswitch_arp_nd_responder_known_ips in ovn northd, it
> will skip LSP, which has unknow flag.
>
> static void
>
> build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op,
>
> struct hmap *lflows,
>
> struct hmap *ports,
>
> struct ds *actions,
>
> struct ds *match)
>
> {
>
> ...
>
> if (lsp_is_external(op->nbsp) || op->has_unknown) {
>
> return;
>
> }
>
>
>
> 从 Windows 版邮件 发送
>
>
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
