> One of the OVS-DPDK maintainers will have to speak up about the flow
> control messages.  I don't know.

Hi Michael,

can you provide the following to help debug this:

OVS Version.
DPDK Version.
NIC Model.
pmd-cpu-mask.
lcore mask.
Port configuration for all ports (including flow control, queue creation etc).


You should note that DPDK uses a Poll Mode Driver (PMD), in essence it 
continuously polls (regardless if traffic is received or not) using the core of 
which a queue of a device is assigned to. It's expected in this case that the 
assigned CPU is 100% utilized from the point of view of tools such as htop.

Can you post the output of 'ovs-appctl dpif-netdev/pmd-rxq-show' also, this 
will help debug performance. A few more comments inline below.

> 
> Do you see log messages reporting high CPU usage?  That would ordinarily
> be the case, if threads other than the PMD threads are using excessive
> CPU.
> 
> "top" and other tools can show CPU usage by thread, and OVS gives its
> threads helpful names.  Which threads are using high CPU?
> 
> On Sun, Apr 08, 2018 at 10:15:11AM +0300, michael me wrote:
> > Hi Ben,
> >
> > Thank you so much for your reply.
> > here below are some of the log from ovs-vswitchd.
> >
> > 2018-04-08T09:52:34.897Z|00333|dpdk|WARN|Failed to enable flow control
> > on device 0 2018-04-08T09:52:34.897Z|00334|dpdk|WARN|Failed to enable
> > flow control on device 1
> > 2018-04-08T09:52:35.025Z|00335|dpdk|WARN|Failed to enable flow control
> > on device 0 2018-04-08T09:52:35.025Z|00336|dpdk|WARN|Failed to enable
> > flow control on device 1
> > 2018-04-08T09:52:36.370Z|00337|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
> > connected
> > 2018-04-08T09:52:36.370Z|00338|rconn|INFO|br-eth1<->tcp:127.0.0.1:6633:
> > connected
> > 2018-04-08T09:52:36.370Z|00339|rconn|INFO|br-eth2<->tcp:127.0.0.1:6633:
> > connected
> > 2018-04-08T09:52:37.102Z|00340|dpdk|WARN|Failed to enable flow control
> > on device 0 2018-04-08T09:52:37.102Z|00341|dpdk|WARN|Failed to enable
> > flow control on device 1
> > 2018-04-08T09:52:37.225Z|00342|dpdk|WARN|Failed to enable flow control
> > on device 0 2018-04-08T09:52:37.225Z|00343|dpdk|WARN|Failed to enable
> > flow control on device 1
> > 2018-04-08T09:52:37.298Z|00344|dpdk|WARN|Failed to enable flow control
> > on device 0 2018-04-08T09:52:37.298Z|00345|dpdk|WARN|Failed to enable
> > flow control on device 1
> > 2018-04-08T09:52:37.426Z|00346|dpdk|WARN|Failed to enable flow control
> > on device 0 2018-04-08T09:52:37.426Z|00347|dpdk|WARN|Failed to enable
> > flow control on device 1
> > 2018-04-08T09:52:47.041Z|00348|connmgr|INFO|br-int<->tcp:127.0.0.1:663
> > 3: 7 flow_mods in the 7 s starting 10 s ago (7 adds)
> > 2018-04-08T09:52:47.245Z|00349|connmgr|INFO|br-eth1<->tcp:127.0.0.1:66
> > 33: 3 flow_mods in the 7 s starting 10 s ago (3 adds)
> > 2018-04-08T09:52:47.444Z|00350|connmgr|INFO|br-eth2<->tcp:127.0.0.1:66
> > 33: 3 flow_mods in the 7 s starting 10 s ago (3 adds)
> >
> > is the  "Failed to enable flow control on device" related to my high
> > CPU load?

This looks like your trying to enable the flow control feature on a device that 
does not support it.
Are you enabling flow control for rx or tx on your devices? you could possibly 
have auto negotiate enabled when adding the port.

For completeness the options regarding flow control are documented in 

http://docs.openvswitch.org/en/latest/howto/dpdk/

> > Just to be clear, i do get traffic through though performance is not
> > great so it does make sense that there is an issue with the flow,
> > though i don't know how to verify this.

This could be related to the queue pinning. The command 'ovs-appctl 
dpif-netdev/pmd-rxq-show' could help diagnose this if you can share its output.

Thanks
Ian

> >
> > Below are the flows that i could find:
> > root@dpdkApt:/# ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4):
> >  cookie=0xb6de486b197e713c, duration=640.295s, table=0, n_packets=0,
> > n_bytes=0, idle_age=802, priority=3,in_port=3,vlan_tci=0x0000/0x1fff
> > actions=mod_vlan_vid:2,NORMAL
> >  cookie=0xb6de486b197e713c, duration=640.188s, table=0, n_packets=0,
> > n_bytes=0, idle_age=801, priority=3,in_port=4,vlan_tci=0x0000/0x1fff
> > actions=mod_vlan_vid:3,NORMAL
> >  cookie=0xb6de486b197e713c, duration=647.450s, table=0, n_packets=0,
> > n_bytes=0, idle_age=911, priority=2,in_port=3 actions=drop
> > cookie=0xb6de486b197e713c, duration=647.251s, table=0, n_packets=0,
> > n_bytes=0, idle_age=910, priority=2,in_port=4 actions=drop
> > cookie=0xb6de486b197e713c, duration=647.670s, table=0, n_packets=947,
> > n_bytes=39930, idle_age=0, priority=0 actions=NORMAL
> > cookie=0xb6de486b197e713c, duration=647.675s, table=23, n_packets=0,
> > n_bytes=0, idle_age=911, priority=0 actions=drop
> > cookie=0xb6de486b197e713c, duration=647.667s, table=24, n_packets=0,
> > n_bytes=0, idle_age=911, priority=0 actions=drop
> >
> > root@dpdkApt:/# ovs-ofctl dump-flows br-eth1 NXST_FLOW reply
> > (xid=0x4):
> >  cookie=0x990e120f393a065e, duration=645.802s, table=0, n_packets=4,
> > n_bytes=198, idle_age=776, priority=4,in_port=1,dl_vlan=2
> > actions=strip_vlan,NORMAL  cookie=0x990e120f393a065e,
> > duration=652.940s, table=0, n_packets=950, n_bytes=40026, idle_age=0,
> > priority=2,in_port=1 actions=drop  cookie=0x990e120f393a065e,
> > duration=652.967s, table=0, n_packets=0, n_bytes=0, idle_age=916,
> > priority=0 actions=NORMAL
> >
> > root@dpdkApt:/# ovs-ofctl dump-flows br-eth2 NXST_FLOW reply
> > (xid=0x4):
> >  cookie=0xb5b43b868c9fcb5f, duration=671.350s, table=0, n_packets=4,
> > n_bytes=198, idle_age=800, priority=4,in_port=2,dl_vlan=3
> > actions=strip_vlan,NORMAL  cookie=0xb5b43b868c9fcb5f,
> > duration=678.397s, table=0, n_packets=977, n_bytes=41160, idle_age=2,
> > priority=2,in_port=2 actions=drop  cookie=0xb5b43b868c9fcb5f,
> > duration=678.424s, table=0, n_packets=0, n_bytes=0, idle_age=941,
> > priority=0 actions=NORMAL
> >
> > Do you see anything that is wrong with my setup?
> > I would greatly appreciate your input.
> >
> > Thanks,
> > Michael
> >
> >
> >
> > On Wed, Apr 4, 2018 at 7:51 PM, Ben Pfaff <b...@ovn.org> wrote:
> >
> > > On Wed, Apr 04, 2018 at 06:19:32PM +0300, michael me wrote:
> > > > The setup i am working with is ovs-dpdk and i have a the PMD mask
> > > > on core 1, but i see that the service of ovs-vswitchd is almost at
> > > > 100% both on core zero and one.
> > > > in the setup i have an openstack VM running and it is pinned to core
> two.
> > >
> > > OVS normally logs some helpful messages when it uses excessive CPU.
> > > Can you check the log for those?
> > >
> _______________________________________________
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Reply via email to