Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-22 Thread Sriram Vatala via dev
Thanks Ilya and Kevin.
I will send the updated patch set.

Thanks & Regards,
Sriram.

-Original Message-
From: Ilya Maximets 
Sent: 22 October 2019 16:07
To: Kevin Traynor ; Sriram Vatala 
; 'Ilya Maximets' ; 
ovs-dev@openvswitch.org
Cc: 'Stokes, Ian' 
Subject: Re: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

On 22.10.2019 12:31, Kevin Traynor wrote:
> On 22/10/2019 09:25, Sriram Vatala wrote:
>> Hi Ilya & Kevin,
>> Thanks for your suggestions. I am summarizing our discussion, please
>> feel free to correct me, if I am wrong.
>>
>
> Hi Sriram,
>
> Will share my thought, Ilya may have different view.
>
>> 1) All custom stats calculated in OVS will have the prefix , so that
>> these stats will not intersect with the names of other stats (driver/HW 
>> etc).
>
> yes
>
>> 2) The prefix used for these custom stats is "cstat_".
>
> I think with 3) (note to explain the prefix) then 'sw_' or 'ovs_'
> would look neater in the logs but i'm ok with any of them.

OK. Lets go with 'ovs_' prefix + prefix explanation note in this case.
It looks like it will have lowest chances to overlap with existing stats.
And it also self-descriptive enough for those who doesn't read docs.

Best regards, Ilya Maximets.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev


Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-22 Thread Ilya Maximets

On 22.10.2019 12:31, Kevin Traynor wrote:

On 22/10/2019 09:25, Sriram Vatala wrote:

Hi Ilya & Kevin,
Thanks for your suggestions. I am summarizing our discussion, please feel free
to correct me, if I am wrong.



Hi Sriram,

Will share my thought, Ilya may have different view.


1) All custom stats calculated in OVS will have the prefix , so that these
stats will not intersect with the names of other stats (driver/HW etc).


yes


2) The prefix used for these custom stats is "cstat_".


I think with 3) (note to explain the prefix) then 'sw_' or 'ovs_' would
look neater in the logs but i'm ok with any of them.


OK. Lets go with 'ovs_' prefix + prefix explanation note in this case.
It looks like it will have lowest chances to overlap with existing stats.
And it also self-descriptive enough for those who doesn't read docs.

Best regards, Ilya Maximets.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev


Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-22 Thread Kevin Traynor
On 22/10/2019 11:31, Kevin Traynor wrote:
> On 22/10/2019 09:25, Sriram Vatala wrote:
>> Hi Ilya & Kevin,
>> Thanks for your suggestions. I am summarizing our discussion, please feel 
>> free 
>> to correct me, if I am wrong.
>>
> 
> Hi Sriram,
> 
> Will share my thought, Ilya may have different view.
> 
>> 1) All custom stats calculated in OVS will have the prefix , so that these 
>> stats will not intersect with the names of other stats (driver/HW etc).
> 
> yes
> 
>> 2) The prefix used for these custom stats is "cstat_".
> 
> I think with 3) (note to explain the prefix) then 'sw_' or 'ovs_' would
> look neater in the logs but i'm ok with any of them.
> 
>> 3) Instead of documenting all the stats, I will add a generic note about the 
>> prefix used for custom stats in "Documentation/topics/dpdk/vhost-user.rst 
>> where "tx_retries" is documented.
>>
> 
> yes, but these stats are not vhost specific so I think they should go
> into topics/dpdk/bridge.rst in the 'Extended & Custom Statistics' section.
> 

crossed mails, but the same comment :-)

> Kevin.
> 
>> Thanks & Regards,
>> Sriram.
>>
>> -Original Message-
>> From: Kevin Traynor 
>> Sent: 21 October 2019 20:22
>> To: Ilya Maximets ; Sriram Vatala 
>> ; ovs-dev@openvswitch.org
>> Cc: Stokes, Ian 
>> Subject: Re: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics
>>
>> On 18/10/2019 13:59, Ilya Maximets wrote:
>>> On 16.10.2019 19:48, Kevin Traynor wrote:
 On 16/10/2019 13:16, Ilya Maximets wrote:
> Hi Kevin,
>
> Thanks for review. Some comments inline.
>
> On 16.10.2019 12:15, Kevin Traynor wrote:
>> Hi Sriram,
>>
>> Thanks for working on making more fine grained stats for debugging.
>> As mentioned yesterday, this patch needs rebase so I just reviewed
>> docs and netdev-dpdk.c which applied. Comments below.
>>
>> On 21/09/2019 03:40, Sriram Vatala wrote:
>>> OVS may be unable to transmit packets for multiple reasons and
>>> today there is a single counter to track packets dropped due to
>>> any of those reasons. The most common reason is that a VM is
>>> unable to read packets fast enough causing the vhostuser port
>>> transmit queue on the OVS side to become full. This manifests as a
>>> problem with VNFs not receiving all packets. Having a separate
>>> drop counter to track packets dropped because the transmit queue
>>> is full will clearly indicate that the problem is on the VM side
>>> and not in OVS. Similarly maintaining separate
>>
>> It reads like a bit of a contradiction. On one hand the "The *most
>> common* reason is that a VM is unable to read packets fast enough".
>> While having a stat "*will clearly indicate* that the problem is on
>> the VM side".
>>
>>> counters for all possible drops helps in indicating sensible cause
>>> for packet drops.
>>>
>>> This patch adds custom software stats counters to track packets
>>> dropped at port level and these counters are displayed along with
>>> other stats in "ovs-vsctl get interface  statistics"
>>> command. The detailed stats will be available for both dpdk and
>>> vhostuser ports.
>>>
>>
>> I think the commit msg could be a bit clearer, suggest something
>> like below - feel free to modify (or reject):
>>
>> OVS may be unable to transmit packets for multiple reasons on the
>> DPDK datapath and today there is a single counter to track packets
>> dropped due to any of those reasons.
>>
>> This patch adds custom software stats for the different reasons
>> packets may be dropped during tx on the DPDK datapath in OVS.
>>
>> - MTU drops : drops that occur due to a too large packet size
>> - Qos drops: drops that occur due to egress QOS
>> - Tx failures: drops as returned by the DPDK PMD send function
>>
>> Note that the reason for tx failures is not specificied in OVS. In
>> practice for vhost ports it is most common that tx failures are
>> because there are not enough available descriptors, which is
>> usually caused by misconfiguration of the guest queues and/or
>> because the guest is not consuming packets fast enough from the queues.
>>
>> These counters are displayed along with other stats in "ovs-vsctl
>> get interface  statistics" command and are available for
>> dpdk and vhostuser/vhostuserclient ports.
>>
>> Signed-off-by: Sriram Vatala 
>>
>> ---
>>
>> v9:...
>> v8:...
>>
>>
>> Note that signed-off, --- and version info should be like this ^^^.
>> otherwise the review version comments will be in the git commit log
>> when it is applied.
>>
>>> --
>>> Changes since v8:
>>> Addressed comments given by Ilya.
>>>
>>> Signed-off-by: Sriram Vatala 
>>> ---
>>>Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
>>>lib/netdev-dpdk.c   

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-22 Thread Kevin Traynor
On 22/10/2019 09:25, Sriram Vatala wrote:
> Hi Ilya & Kevin,
> Thanks for your suggestions. I am summarizing our discussion, please feel 
> free 
> to correct me, if I am wrong.
> 

Hi Sriram,

Will share my thought, Ilya may have different view.

> 1) All custom stats calculated in OVS will have the prefix , so that these 
> stats will not intersect with the names of other stats (driver/HW etc).

yes

> 2) The prefix used for these custom stats is "cstat_".

I think with 3) (note to explain the prefix) then 'sw_' or 'ovs_' would
look neater in the logs but i'm ok with any of them.

> 3) Instead of documenting all the stats, I will add a generic note about the 
> prefix used for custom stats in "Documentation/topics/dpdk/vhost-user.rst 
> where "tx_retries" is documented.
> 

yes, but these stats are not vhost specific so I think they should go
into topics/dpdk/bridge.rst in the 'Extended & Custom Statistics' section.

Kevin.

> Thanks & Regards,
> Sriram.
> 
> -Original Message-
> From: Kevin Traynor 
> Sent: 21 October 2019 20:22
> To: Ilya Maximets ; Sriram Vatala 
> ; ovs-dev@openvswitch.org
> Cc: Stokes, Ian 
> Subject: Re: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics
> 
> On 18/10/2019 13:59, Ilya Maximets wrote:
>> On 16.10.2019 19:48, Kevin Traynor wrote:
>>> On 16/10/2019 13:16, Ilya Maximets wrote:
 Hi Kevin,

 Thanks for review. Some comments inline.

 On 16.10.2019 12:15, Kevin Traynor wrote:
> Hi Sriram,
>
> Thanks for working on making more fine grained stats for debugging.
> As mentioned yesterday, this patch needs rebase so I just reviewed
> docs and netdev-dpdk.c which applied. Comments below.
>
> On 21/09/2019 03:40, Sriram Vatala wrote:
>> OVS may be unable to transmit packets for multiple reasons and
>> today there is a single counter to track packets dropped due to
>> any of those reasons. The most common reason is that a VM is
>> unable to read packets fast enough causing the vhostuser port
>> transmit queue on the OVS side to become full. This manifests as a
>> problem with VNFs not receiving all packets. Having a separate
>> drop counter to track packets dropped because the transmit queue
>> is full will clearly indicate that the problem is on the VM side
>> and not in OVS. Similarly maintaining separate
>
> It reads like a bit of a contradiction. On one hand the "The *most
> common* reason is that a VM is unable to read packets fast enough".
> While having a stat "*will clearly indicate* that the problem is on
> the VM side".
>
>> counters for all possible drops helps in indicating sensible cause
>> for packet drops.
>>
>> This patch adds custom software stats counters to track packets
>> dropped at port level and these counters are displayed along with
>> other stats in "ovs-vsctl get interface  statistics"
>> command. The detailed stats will be available for both dpdk and
>> vhostuser ports.
>>
>
> I think the commit msg could be a bit clearer, suggest something
> like below - feel free to modify (or reject):
>
> OVS may be unable to transmit packets for multiple reasons on the
> DPDK datapath and today there is a single counter to track packets
> dropped due to any of those reasons.
>
> This patch adds custom software stats for the different reasons
> packets may be dropped during tx on the DPDK datapath in OVS.
>
> - MTU drops : drops that occur due to a too large packet size
> - Qos drops: drops that occur due to egress QOS
> - Tx failures: drops as returned by the DPDK PMD send function
>
> Note that the reason for tx failures is not specificied in OVS. In
> practice for vhost ports it is most common that tx failures are
> because there are not enough available descriptors, which is
> usually caused by misconfiguration of the guest queues and/or
> because the guest is not consuming packets fast enough from the queues.
>
> These counters are displayed along with other stats in "ovs-vsctl
> get interface  statistics" command and are available for
> dpdk and vhostuser/vhostuserclient ports.
>
> Signed-off-by: Sriram Vatala 
>
> ---
>
> v9:...
> v8:...
>
>
> Note that signed-off, --- and version info should be like this ^^^.
> otherwise the review version comments will be in the git commit log
> when it is applied.
>
>> --
>> Changes since v8:
>> Addressed comments given by Ilya.
>>
>> Signed-off-by: Sriram Vatala 
>> ---
>>Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
>>lib/netdev-dpdk.c | 81 
>> +++
>>utilities/bugtool/automake.mk |  3 +-
>>utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
>>

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-22 Thread Ilya Maximets

On 22.10.2019 10:25, Sriram Vatala wrote:

Hi Ilya & Kevin,
Thanks for your suggestions. I am summarizing our discussion, please feel free
to correct me, if I am wrong.

1) All custom stats calculated in OVS will have the prefix , so that these
stats will not intersect with the names of other stats (driver/HW etc).
2) The prefix used for these custom stats is "cstat_".
3) Instead of documenting all the stats, I will add a generic note about the
prefix used for custom stats in "Documentation/topics/dpdk/vhost-user.rst
where "tx_retries" is documented.


Looks OK.
One thing is that prefix documentation should, probably, go to the
"Extended & Custom Statistics" section of Documentation/topics/dpdk/bridge.rst .
This looks more appropriate, because we'll have those stats not only for
vhostuser ports.

Best regards, Ilya Maximets.



Thanks & Regards,
Sriram.

-Original Message-
From: Kevin Traynor 
Sent: 21 October 2019 20:22
To: Ilya Maximets ; Sriram Vatala
; ovs-dev@openvswitch.org
Cc: Stokes, Ian 
Subject: Re: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

On 18/10/2019 13:59, Ilya Maximets wrote:

On 16.10.2019 19:48, Kevin Traynor wrote:

On 16/10/2019 13:16, Ilya Maximets wrote:

Hi Kevin,

Thanks for review. Some comments inline.

On 16.10.2019 12:15, Kevin Traynor wrote:

Hi Sriram,

Thanks for working on making more fine grained stats for debugging.
As mentioned yesterday, this patch needs rebase so I just reviewed
docs and netdev-dpdk.c which applied. Comments below.

On 21/09/2019 03:40, Sriram Vatala wrote:

OVS may be unable to transmit packets for multiple reasons and
today there is a single counter to track packets dropped due to
any of those reasons. The most common reason is that a VM is
unable to read packets fast enough causing the vhostuser port
transmit queue on the OVS side to become full. This manifests as a
problem with VNFs not receiving all packets. Having a separate
drop counter to track packets dropped because the transmit queue
is full will clearly indicate that the problem is on the VM side
and not in OVS. Similarly maintaining separate


It reads like a bit of a contradiction. On one hand the "The *most
common* reason is that a VM is unable to read packets fast enough".
While having a stat "*will clearly indicate* that the problem is on
the VM side".


counters for all possible drops helps in indicating sensible cause
for packet drops.

This patch adds custom software stats counters to track packets
dropped at port level and these counters are displayed along with
other stats in "ovs-vsctl get interface  statistics"
command. The detailed stats will be available for both dpdk and
vhostuser ports.



I think the commit msg could be a bit clearer, suggest something
like below - feel free to modify (or reject):

OVS may be unable to transmit packets for multiple reasons on the
DPDK datapath and today there is a single counter to track packets
dropped due to any of those reasons.

This patch adds custom software stats for the different reasons
packets may be dropped during tx on the DPDK datapath in OVS.

- MTU drops : drops that occur due to a too large packet size
- Qos drops: drops that occur due to egress QOS
- Tx failures: drops as returned by the DPDK PMD send function

Note that the reason for tx failures is not specificied in OVS. In
practice for vhost ports it is most common that tx failures are
because there are not enough available descriptors, which is
usually caused by misconfiguration of the guest queues and/or
because the guest is not consuming packets fast enough from the queues.

These counters are displayed along with other stats in "ovs-vsctl
get interface  statistics" command and are available for
dpdk and vhostuser/vhostuserclient ports.

Signed-off-by: Sriram Vatala 

---

v9:...
v8:...


Note that signed-off, --- and version info should be like this ^^^.
otherwise the review version comments will be in the git commit log
when it is applied.


--
Changes since v8:
Addressed comments given by Ilya.

Signed-off-by: Sriram Vatala 
---
Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
lib/netdev-dpdk.c | 81
+++
utilities/bugtool/automake.mk |  3 +-
utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
.../plugins/network-status/openvswitch.xml|  1 +
5 files changed, 105 insertions(+), 18 deletions(-)
create mode 100755 utilities/bugtool/ovs-bugtool-get-port-stats

diff --git a/Documentation/topics/dpdk/vhost-user.rst
b/Documentation/topics/dpdk/vhost-user.rst
index 724aa62f6..89388a2bf 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -551,7 +551,18 @@ processing packets at the required rate.
The amount of Tx retries on a vhost-user or vhost-user-client
interface can be
shown with::

-  $ ovs-vsctl get Interface dpdkvhostclient0
statistics:tx_retries
+  $ ovs-vsctl get Interface 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-22 Thread Sriram Vatala via dev
Hi Ilya & Kevin,
Thanks for your suggestions. I am summarizing our discussion, please feel free 
to correct me, if I am wrong.

1) All custom stats calculated in OVS will have the prefix , so that these 
stats will not intersect with the names of other stats (driver/HW etc).
2) The prefix used for these custom stats is "cstat_".
3) Instead of documenting all the stats, I will add a generic note about the 
prefix used for custom stats in "Documentation/topics/dpdk/vhost-user.rst 
where "tx_retries" is documented.

Thanks & Regards,
Sriram.

-Original Message-
From: Kevin Traynor 
Sent: 21 October 2019 20:22
To: Ilya Maximets ; Sriram Vatala 
; ovs-dev@openvswitch.org
Cc: Stokes, Ian 
Subject: Re: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

On 18/10/2019 13:59, Ilya Maximets wrote:
> On 16.10.2019 19:48, Kevin Traynor wrote:
>> On 16/10/2019 13:16, Ilya Maximets wrote:
>>> Hi Kevin,
>>>
>>> Thanks for review. Some comments inline.
>>>
>>> On 16.10.2019 12:15, Kevin Traynor wrote:
 Hi Sriram,

 Thanks for working on making more fine grained stats for debugging.
 As mentioned yesterday, this patch needs rebase so I just reviewed
 docs and netdev-dpdk.c which applied. Comments below.

 On 21/09/2019 03:40, Sriram Vatala wrote:
> OVS may be unable to transmit packets for multiple reasons and
> today there is a single counter to track packets dropped due to
> any of those reasons. The most common reason is that a VM is
> unable to read packets fast enough causing the vhostuser port
> transmit queue on the OVS side to become full. This manifests as a
> problem with VNFs not receiving all packets. Having a separate
> drop counter to track packets dropped because the transmit queue
> is full will clearly indicate that the problem is on the VM side
> and not in OVS. Similarly maintaining separate

 It reads like a bit of a contradiction. On one hand the "The *most
 common* reason is that a VM is unable to read packets fast enough".
 While having a stat "*will clearly indicate* that the problem is on
 the VM side".

> counters for all possible drops helps in indicating sensible cause
> for packet drops.
>
> This patch adds custom software stats counters to track packets
> dropped at port level and these counters are displayed along with
> other stats in "ovs-vsctl get interface  statistics"
> command. The detailed stats will be available for both dpdk and
> vhostuser ports.
>

 I think the commit msg could be a bit clearer, suggest something
 like below - feel free to modify (or reject):

 OVS may be unable to transmit packets for multiple reasons on the
 DPDK datapath and today there is a single counter to track packets
 dropped due to any of those reasons.

 This patch adds custom software stats for the different reasons
 packets may be dropped during tx on the DPDK datapath in OVS.

 - MTU drops : drops that occur due to a too large packet size
 - Qos drops: drops that occur due to egress QOS
 - Tx failures: drops as returned by the DPDK PMD send function

 Note that the reason for tx failures is not specificied in OVS. In
 practice for vhost ports it is most common that tx failures are
 because there are not enough available descriptors, which is
 usually caused by misconfiguration of the guest queues and/or
 because the guest is not consuming packets fast enough from the queues.

 These counters are displayed along with other stats in "ovs-vsctl
 get interface  statistics" command and are available for
 dpdk and vhostuser/vhostuserclient ports.

 Signed-off-by: Sriram Vatala 

 ---

 v9:...
 v8:...


 Note that signed-off, --- and version info should be like this ^^^.
 otherwise the review version comments will be in the git commit log
 when it is applied.

> --
> Changes since v8:
> Addressed comments given by Ilya.
>
> Signed-off-by: Sriram Vatala 
> ---
>Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
>lib/netdev-dpdk.c | 81 
> +++
>utilities/bugtool/automake.mk |  3 +-
>utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
>.../plugins/network-status/openvswitch.xml|  1 +
>5 files changed, 105 insertions(+), 18 deletions(-)
>create mode 100755 utilities/bugtool/ovs-bugtool-get-port-stats
>
> diff --git a/Documentation/topics/dpdk/vhost-user.rst
> b/Documentation/topics/dpdk/vhost-user.rst
> index 724aa62f6..89388a2bf 100644
> --- a/Documentation/topics/dpdk/vhost-user.rst
> +++ b/Documentation/topics/dpdk/vhost-user.rst
> @@ -551,7 +551,18 @@ processing packets at the required rate.
>The amount of Tx retries on a vhost-user or 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-21 Thread Kevin Traynor
On 18/10/2019 13:59, Ilya Maximets wrote:
> On 16.10.2019 19:48, Kevin Traynor wrote:
>> On 16/10/2019 13:16, Ilya Maximets wrote:
>>> Hi Kevin,
>>>
>>> Thanks for review. Some comments inline.
>>>
>>> On 16.10.2019 12:15, Kevin Traynor wrote:
 Hi Sriram,

 Thanks for working on making more fine grained stats for debugging. As
 mentioned yesterday, this patch needs rebase so I just reviewed docs and
 netdev-dpdk.c which applied. Comments below.

 On 21/09/2019 03:40, Sriram Vatala wrote:
> OVS may be unable to transmit packets for multiple reasons and
> today there is a single counter to track packets dropped due to
> any of those reasons. The most common reason is that a VM is
> unable to read packets fast enough causing the vhostuser port
> transmit queue on the OVS side to become full. This manifests
> as a problem with VNFs not receiving all packets. Having a
> separate drop counter to track packets dropped because the
> transmit queue is full will clearly indicate that the problem
> is on the VM side and not in OVS. Similarly maintaining separate

 It reads like a bit of a contradiction. On one hand the "The *most
 common* reason is that a VM is unable to read packets fast enough".
 While having a stat "*will clearly indicate* that the problem is on the
 VM side".

> counters for all possible drops helps in indicating sensible
> cause for packet drops.
>
> This patch adds custom software stats counters to track packets
> dropped at port level and these counters are displayed along with
> other stats in "ovs-vsctl get interface  statistics"
> command. The detailed stats will be available for both dpdk and
> vhostuser ports.
>

 I think the commit msg could be a bit clearer, suggest something like
 below - feel free to modify (or reject):

 OVS may be unable to transmit packets for multiple reasons on the DPDK
 datapath and today there is a single counter to track packets dropped
 due to any of those reasons.

 This patch adds custom software stats for the different reasons packets
 may be dropped during tx on the DPDK datapath in OVS.

 - MTU drops : drops that occur due to a too large packet size
 - Qos drops: drops that occur due to egress QOS
 - Tx failures: drops as returned by the DPDK PMD send function

 Note that the reason for tx failures is not specificied in OVS. In
 practice for vhost ports it is most common that tx failures are because
 there are not enough available descriptors, which is usually caused by
 misconfiguration of the guest queues and/or because the guest is not
 consuming packets fast enough from the queues.

 These counters are displayed along with other stats in "ovs-vsctl get
 interface  statistics" command and are available for dpdk and
 vhostuser/vhostuserclient ports.

 Signed-off-by: Sriram Vatala 

 ---

 v9:...
 v8:...


 Note that signed-off, --- and version info should be like this ^^^.
 otherwise the review version comments will be in the git commit log when
 it is applied.

> --
> Changes since v8:
> Addressed comments given by Ilya.
>
> Signed-off-by: Sriram Vatala 
> ---
>Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
>lib/netdev-dpdk.c | 81 +++
>utilities/bugtool/automake.mk |  3 +-
>utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
>.../plugins/network-status/openvswitch.xml|  1 +
>5 files changed, 105 insertions(+), 18 deletions(-)
>create mode 100755 utilities/bugtool/ovs-bugtool-get-port-stats
>
> diff --git a/Documentation/topics/dpdk/vhost-user.rst 
> b/Documentation/topics/dpdk/vhost-user.rst
> index 724aa62f6..89388a2bf 100644
> --- a/Documentation/topics/dpdk/vhost-user.rst
> +++ b/Documentation/topics/dpdk/vhost-user.rst
> @@ -551,7 +551,18 @@ processing packets at the required rate.
>The amount of Tx retries on a vhost-user or vhost-user-client 
> interface can be
>shown with::
>
> -  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries
> +  $ ovs-vsctl get Interface dpdkvhostclient0 
> statistics:netdev_dpdk_tx_retries

 I think the "netdev_dpdk_" prefixes should be removed for a few reasons,

 - These are custom stats that will only be displayed for the dpdk ports
 so they don't need any extra prefix to say they are for dpdk ports

 - The 'tx_retries' stat is part of OVS 2.12, I don't like to change its
 name to a different one in OVS 2.13 unless there is a very good reason

 - The existing stats don't have this type of prefixes (granted most of
 them are general stats):
>>>
>>> The main reason for the 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-18 Thread Ilya Maximets

On 16.10.2019 19:48, Kevin Traynor wrote:

On 16/10/2019 13:16, Ilya Maximets wrote:

Hi Kevin,

Thanks for review. Some comments inline.

On 16.10.2019 12:15, Kevin Traynor wrote:

Hi Sriram,

Thanks for working on making more fine grained stats for debugging. As
mentioned yesterday, this patch needs rebase so I just reviewed docs and
netdev-dpdk.c which applied. Comments below.

On 21/09/2019 03:40, Sriram Vatala wrote:

OVS may be unable to transmit packets for multiple reasons and
today there is a single counter to track packets dropped due to
any of those reasons. The most common reason is that a VM is
unable to read packets fast enough causing the vhostuser port
transmit queue on the OVS side to become full. This manifests
as a problem with VNFs not receiving all packets. Having a
separate drop counter to track packets dropped because the
transmit queue is full will clearly indicate that the problem
is on the VM side and not in OVS. Similarly maintaining separate


It reads like a bit of a contradiction. On one hand the "The *most
common* reason is that a VM is unable to read packets fast enough".
While having a stat "*will clearly indicate* that the problem is on the
VM side".


counters for all possible drops helps in indicating sensible
cause for packet drops.

This patch adds custom software stats counters to track packets
dropped at port level and these counters are displayed along with
other stats in "ovs-vsctl get interface  statistics"
command. The detailed stats will be available for both dpdk and
vhostuser ports.



I think the commit msg could be a bit clearer, suggest something like
below - feel free to modify (or reject):

OVS may be unable to transmit packets for multiple reasons on the DPDK
datapath and today there is a single counter to track packets dropped
due to any of those reasons.

This patch adds custom software stats for the different reasons packets
may be dropped during tx on the DPDK datapath in OVS.

- MTU drops : drops that occur due to a too large packet size
- Qos drops: drops that occur due to egress QOS
- Tx failures: drops as returned by the DPDK PMD send function

Note that the reason for tx failures is not specificied in OVS. In
practice for vhost ports it is most common that tx failures are because
there are not enough available descriptors, which is usually caused by
misconfiguration of the guest queues and/or because the guest is not
consuming packets fast enough from the queues.

These counters are displayed along with other stats in "ovs-vsctl get
interface  statistics" command and are available for dpdk and
vhostuser/vhostuserclient ports.

Signed-off-by: Sriram Vatala 

---

v9:...
v8:...


Note that signed-off, --- and version info should be like this ^^^.
otherwise the review version comments will be in the git commit log when
it is applied.


--
Changes since v8:
Addressed comments given by Ilya.

Signed-off-by: Sriram Vatala 
---
   Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
   lib/netdev-dpdk.c | 81 +++
   utilities/bugtool/automake.mk |  3 +-
   utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
   .../plugins/network-status/openvswitch.xml|  1 +
   5 files changed, 105 insertions(+), 18 deletions(-)
   create mode 100755 utilities/bugtool/ovs-bugtool-get-port-stats

diff --git a/Documentation/topics/dpdk/vhost-user.rst 
b/Documentation/topics/dpdk/vhost-user.rst
index 724aa62f6..89388a2bf 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -551,7 +551,18 @@ processing packets at the required rate.
   The amount of Tx retries on a vhost-user or vhost-user-client interface can 
be
   shown with::
   
-  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries

+  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:netdev_dpdk_tx_retries


I think the "netdev_dpdk_" prefixes should be removed for a few reasons,

- These are custom stats that will only be displayed for the dpdk ports
so they don't need any extra prefix to say they are for dpdk ports

- The 'tx_retries' stat is part of OVS 2.12, I don't like to change its
name to a different one in OVS 2.13 unless there is a very good reason

- The existing stats don't have this type of prefixes (granted most of
them are general stats):


The main reason for the prefix for me is to distinguish those stats
from the stats that comest from the driver/HW.  Our new stats has
very generic names that could be named a lot like or even equal to
HW stats.  This might be not an issue for vhostuser, but for HW
NICs this may be really confusing for users.

I'm not insisting on 'netdev_dpdk_' prefix, but, IMHO, we need a
way to clearly distinguish those stats.  We may use different prefixes
like 'sw_' or just 'ovs_'.



ok, I can see that we might want to distinguish custom stats to indicate
that they are specific for that device but that also has the side effect
of making 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-16 Thread Ben Pfaff
On Wed, Oct 16, 2019 at 02:16:14PM +0200, Ilya Maximets wrote:
> BTW, you will not find any description for statistics provided by the linux
> or DPDK drivers.  You could only look at the driver code or device spec.

I'm not sure the meanings are even consistent.  I've heard that
different Linux drivers count the number of bytes in a packet
differently, for example.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev


Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-16 Thread Kevin Traynor
On 16/10/2019 13:16, Ilya Maximets wrote:
> Hi Kevin,
> 
> Thanks for review. Some comments inline.
> 
> On 16.10.2019 12:15, Kevin Traynor wrote:
>> Hi Sriram,
>>
>> Thanks for working on making more fine grained stats for debugging. As
>> mentioned yesterday, this patch needs rebase so I just reviewed docs and
>> netdev-dpdk.c which applied. Comments below.
>>
>> On 21/09/2019 03:40, Sriram Vatala wrote:
>>> OVS may be unable to transmit packets for multiple reasons and
>>> today there is a single counter to track packets dropped due to
>>> any of those reasons. The most common reason is that a VM is
>>> unable to read packets fast enough causing the vhostuser port
>>> transmit queue on the OVS side to become full. This manifests
>>> as a problem with VNFs not receiving all packets. Having a
>>> separate drop counter to track packets dropped because the
>>> transmit queue is full will clearly indicate that the problem
>>> is on the VM side and not in OVS. Similarly maintaining separate
>>
>> It reads like a bit of a contradiction. On one hand the "The *most
>> common* reason is that a VM is unable to read packets fast enough".
>> While having a stat "*will clearly indicate* that the problem is on the
>> VM side".
>>
>>> counters for all possible drops helps in indicating sensible
>>> cause for packet drops.
>>>
>>> This patch adds custom software stats counters to track packets
>>> dropped at port level and these counters are displayed along with
>>> other stats in "ovs-vsctl get interface  statistics"
>>> command. The detailed stats will be available for both dpdk and
>>> vhostuser ports.
>>>
>>
>> I think the commit msg could be a bit clearer, suggest something like
>> below - feel free to modify (or reject):
>>
>> OVS may be unable to transmit packets for multiple reasons on the DPDK
>> datapath and today there is a single counter to track packets dropped
>> due to any of those reasons.
>>
>> This patch adds custom software stats for the different reasons packets
>> may be dropped during tx on the DPDK datapath in OVS.
>>
>> - MTU drops : drops that occur due to a too large packet size
>> - Qos drops: drops that occur due to egress QOS
>> - Tx failures: drops as returned by the DPDK PMD send function
>>
>> Note that the reason for tx failures is not specificied in OVS. In
>> practice for vhost ports it is most common that tx failures are because
>> there are not enough available descriptors, which is usually caused by
>> misconfiguration of the guest queues and/or because the guest is not
>> consuming packets fast enough from the queues.
>>
>> These counters are displayed along with other stats in "ovs-vsctl get
>> interface  statistics" command and are available for dpdk and
>> vhostuser/vhostuserclient ports.
>>
>> Signed-off-by: Sriram Vatala 
>>
>> ---
>>
>> v9:...
>> v8:...
>>
>>
>> Note that signed-off, --- and version info should be like this ^^^.
>> otherwise the review version comments will be in the git commit log when
>> it is applied.
>>
>>> --
>>> Changes since v8:
>>> Addressed comments given by Ilya.
>>>
>>> Signed-off-by: Sriram Vatala 
>>> ---
>>>   Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
>>>   lib/netdev-dpdk.c | 81 +++
>>>   utilities/bugtool/automake.mk |  3 +-
>>>   utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
>>>   .../plugins/network-status/openvswitch.xml|  1 +
>>>   5 files changed, 105 insertions(+), 18 deletions(-)
>>>   create mode 100755 utilities/bugtool/ovs-bugtool-get-port-stats
>>>
>>> diff --git a/Documentation/topics/dpdk/vhost-user.rst 
>>> b/Documentation/topics/dpdk/vhost-user.rst
>>> index 724aa62f6..89388a2bf 100644
>>> --- a/Documentation/topics/dpdk/vhost-user.rst
>>> +++ b/Documentation/topics/dpdk/vhost-user.rst
>>> @@ -551,7 +551,18 @@ processing packets at the required rate.
>>>   The amount of Tx retries on a vhost-user or vhost-user-client interface 
>>> can be
>>>   shown with::
>>>   
>>> -  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries
>>> +  $ ovs-vsctl get Interface dpdkvhostclient0 
>>> statistics:netdev_dpdk_tx_retries
>>
>> I think the "netdev_dpdk_" prefixes should be removed for a few reasons,
>>
>> - These are custom stats that will only be displayed for the dpdk ports
>> so they don't need any extra prefix to say they are for dpdk ports
>>
>> - The 'tx_retries' stat is part of OVS 2.12, I don't like to change its
>> name to a different one in OVS 2.13 unless there is a very good reason
>>
>> - The existing stats don't have this type of prefixes (granted most of
>> them are general stats):
> 
> The main reason for the prefix for me is to distinguish those stats
> from the stats that comest from the driver/HW.  Our new stats has
> very generic names that could be named a lot like or even equal to
> HW stats.  This might be not an issue for vhostuser, but for HW
> NICs this may be really confusing for users.
> 
> I'm not insisting on 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-16 Thread Sriram Vatala via dev
Hi Kevin & Ilya,
Thanks Kevin for reviewing the patch. Response inline.
Thanks Ilya for your response.

-Original Message-
From: Ilya Maximets 
Sent: 16 October 2019 17:46
To: Kevin Traynor ; Sriram Vatala 
; ovs-dev@openvswitch.org; Ilya Maximets 

Cc: Stokes, Ian 
Subject: Re: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi Kevin,

Thanks for review. Some comments inline.

On 16.10.2019 12:15, Kevin Traynor wrote:
> Hi Sriram,
>
> Thanks for working on making more fine grained stats for debugging. As
> mentioned yesterday, this patch needs rebase so I just reviewed docs
> and netdev-dpdk.c which applied. Comments below.
>
> On 21/09/2019 03:40, Sriram Vatala wrote:
>> OVS may be unable to transmit packets for multiple reasons and today
>> there is a single counter to track packets dropped due to any of
>> those reasons. The most common reason is that a VM is unable to read
>> packets fast enough causing the vhostuser port transmit queue on the
>> OVS side to become full. This manifests as a problem with VNFs not
>> receiving all packets. Having a separate drop counter to track
>> packets dropped because the transmit queue is full will clearly
>> indicate that the problem is on the VM side and not in OVS. Similarly
>> maintaining separate
>

> It reads like a bit of a contradiction. On one hand the "The *most
> common* reason is that a VM is unable to read packets fast enough".
> While having a stat "*will clearly indicate* that the problem is on
> the VM side".
>

I mentioned one use-case to emphasis the need for separate drop stats 
besides combined drop counter. I will rephrase the commit message.

>> counters for all possible drops helps in indicating sensible cause
>> for packet drops.
>>
>> This patch adds custom software stats counters to track packets
>> dropped at port level and these counters are displayed along with
>> other stats in "ovs-vsctl get interface  statistics"
>> command. The detailed stats will be available for both dpdk and
>> vhostuser ports.
>>
>
> I think the commit msg could be a bit clearer, suggest something like
> below - feel free to modify (or reject):
>
> OVS may be unable to transmit packets for multiple reasons on the DPDK
> datapath and today there is a single counter to track packets dropped
> due to any of those reasons.
>
> This patch adds custom software stats for the different reasons
> packets may be dropped during tx on the DPDK datapath in OVS.
>
> - MTU drops : drops that occur due to a too large packet size
> - Qos drops: drops that occur due to egress QOS
> - Tx failures: drops as returned by the DPDK PMD send function
>
> Note that the reason for tx failures is not specificied in OVS. In
> practice for vhost ports it is most common that tx failures are
> because there are not enough available descriptors, which is usually
> caused by misconfiguration of the guest queues and/or because the
> guest is not consuming packets fast enough from the queues.
>
> These counters are displayed along with other stats in "ovs-vsctl get
> interface  statistics" command and are available for dpdk and
> vhostuser/vhostuserclient ports.
>
Looks good to me. Thanks for the suggestion.
> Signed-off-by: Sriram Vatala 
>
> ---
>
> v9:...
> v8:...
>
>
> Note that signed-off, --- and version info should be like this ^^^.
> otherwise the review version comments will be in the git commit log
> when it is applied.
>
   I will take care of this in my next patch.
>> --
>> Changes since v8:
>> Addressed comments given by Ilya.
>>
>> Signed-off-by: Sriram Vatala 
>> ---
>>   Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
>>   lib/netdev-dpdk.c | 81 +++
>>   utilities/bugtool/automake.mk |  3 +-
>>   utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
>>   .../plugins/network-status/openvswitch.xml|  1 +
>>   5 files changed, 105 insertions(+), 18 deletions(-)
>>   create mode 100755 utilities/bugtool/ovs-bugtool-get-port-stats
>>
>> diff --git a/Documentation/topics/dpdk/vhost-user.rst
>> b/Documentation/topics/dpdk/vhost-user.rst
>> index 724aa62f6..89388a2bf 100644
>> --- a/Documentation/topics/dpdk/vhost-user.rst
>> +++ b/Documentation/topics/dpdk/vhost-user.rst
>> @@ -551,7 +551,18 @@ processing packets at the required rate.
>>   The amount of Tx retries on a vhost-user or vhost-user-client interface 
>> can be
>>   shown with::
>>
>> -  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries
>> +  $ ovs-vsctl get Interface dpdkvhostclient0
>> + statistics:netdev_dpdk_tx_retries
>
> I think the "netdev_dpdk_" prefixes should be removed for a few
> reasons,
>
> - These are custom stats that will only be displayed for the dpdk
> ports so they don't need any extra prefix to say they are for dpdk
> ports
>
> - The 'tx_retries' stat is part of OVS 2.12, I don't like to change
> its name to a different one in OVS 2.13 unless there is a very good
> reason
>
> - The existing 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-16 Thread Ilya Maximets

Hi Kevin,

Thanks for review. Some comments inline.

On 16.10.2019 12:15, Kevin Traynor wrote:

Hi Sriram,

Thanks for working on making more fine grained stats for debugging. As
mentioned yesterday, this patch needs rebase so I just reviewed docs and
netdev-dpdk.c which applied. Comments below.

On 21/09/2019 03:40, Sriram Vatala wrote:

OVS may be unable to transmit packets for multiple reasons and
today there is a single counter to track packets dropped due to
any of those reasons. The most common reason is that a VM is
unable to read packets fast enough causing the vhostuser port
transmit queue on the OVS side to become full. This manifests
as a problem with VNFs not receiving all packets. Having a
separate drop counter to track packets dropped because the
transmit queue is full will clearly indicate that the problem
is on the VM side and not in OVS. Similarly maintaining separate


It reads like a bit of a contradiction. On one hand the "The *most
common* reason is that a VM is unable to read packets fast enough".
While having a stat "*will clearly indicate* that the problem is on the
VM side".


counters for all possible drops helps in indicating sensible
cause for packet drops.

This patch adds custom software stats counters to track packets
dropped at port level and these counters are displayed along with
other stats in "ovs-vsctl get interface  statistics"
command. The detailed stats will be available for both dpdk and
vhostuser ports.



I think the commit msg could be a bit clearer, suggest something like
below - feel free to modify (or reject):

OVS may be unable to transmit packets for multiple reasons on the DPDK
datapath and today there is a single counter to track packets dropped
due to any of those reasons.

This patch adds custom software stats for the different reasons packets
may be dropped during tx on the DPDK datapath in OVS.

- MTU drops : drops that occur due to a too large packet size
- Qos drops: drops that occur due to egress QOS
- Tx failures: drops as returned by the DPDK PMD send function

Note that the reason for tx failures is not specificied in OVS. In
practice for vhost ports it is most common that tx failures are because
there are not enough available descriptors, which is usually caused by
misconfiguration of the guest queues and/or because the guest is not
consuming packets fast enough from the queues.

These counters are displayed along with other stats in "ovs-vsctl get
interface  statistics" command and are available for dpdk and
vhostuser/vhostuserclient ports.

Signed-off-by: Sriram Vatala 

---

v9:...
v8:...


Note that signed-off, --- and version info should be like this ^^^.
otherwise the review version comments will be in the git commit log when
it is applied.


--
Changes since v8:
Addressed comments given by Ilya.

Signed-off-by: Sriram Vatala 
---
  Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
  lib/netdev-dpdk.c | 81 +++
  utilities/bugtool/automake.mk |  3 +-
  utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
  .../plugins/network-status/openvswitch.xml|  1 +
  5 files changed, 105 insertions(+), 18 deletions(-)
  create mode 100755 utilities/bugtool/ovs-bugtool-get-port-stats

diff --git a/Documentation/topics/dpdk/vhost-user.rst 
b/Documentation/topics/dpdk/vhost-user.rst
index 724aa62f6..89388a2bf 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -551,7 +551,18 @@ processing packets at the required rate.
  The amount of Tx retries on a vhost-user or vhost-user-client interface can be
  shown with::
  
-  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries

+  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:netdev_dpdk_tx_retries


I think the "netdev_dpdk_" prefixes should be removed for a few reasons,

- These are custom stats that will only be displayed for the dpdk ports
so they don't need any extra prefix to say they are for dpdk ports

- The 'tx_retries' stat is part of OVS 2.12, I don't like to change its
name to a different one in OVS 2.13 unless there is a very good reason

- The existing stats don't have this type of prefixes (granted most of
them are general stats):


The main reason for the prefix for me is to distinguish those stats
from the stats that comest from the driver/HW.  Our new stats has
very generic names that could be named a lot like or even equal to
HW stats.  This might be not an issue for vhostuser, but for HW
NICs this may be really confusing for users.

I'm not insisting on 'netdev_dpdk_' prefix, but, IMHO, we need a
way to clearly distinguish those stats.  We may use different prefixes
like 'sw_' or just 'ovs_'.


# ovs-vsctl get Interface dpdkvhost0 statistics
{"rx_1024_to_1522_packets"=0, "rx_128_to_255_packets"=0,
"rx_1523_to_max_packets"=0, "rx_1_to_64_packets"=25622176,
"rx_256_to_511_packets"=0, "rx_512_to_1023_packets"=0,
"rx_65_to_127_packets"=0, 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-16 Thread Kevin Traynor
Hi Sriram,

Thanks for working on making more fine grained stats for debugging. As
mentioned yesterday, this patch needs rebase so I just reviewed docs and
netdev-dpdk.c which applied. Comments below.

On 21/09/2019 03:40, Sriram Vatala wrote:
> OVS may be unable to transmit packets for multiple reasons and
> today there is a single counter to track packets dropped due to
> any of those reasons. The most common reason is that a VM is
> unable to read packets fast enough causing the vhostuser port
> transmit queue on the OVS side to become full. This manifests
> as a problem with VNFs not receiving all packets. Having a
> separate drop counter to track packets dropped because the
> transmit queue is full will clearly indicate that the problem
> is on the VM side and not in OVS. Similarly maintaining separate

It reads like a bit of a contradiction. On one hand the "The *most
common* reason is that a VM is unable to read packets fast enough".
While having a stat "*will clearly indicate* that the problem is on the
VM side".

> counters for all possible drops helps in indicating sensible
> cause for packet drops.
> 
> This patch adds custom software stats counters to track packets
> dropped at port level and these counters are displayed along with
> other stats in "ovs-vsctl get interface  statistics"
> command. The detailed stats will be available for both dpdk and
> vhostuser ports.
> 

I think the commit msg could be a bit clearer, suggest something like
below - feel free to modify (or reject):

OVS may be unable to transmit packets for multiple reasons on the DPDK
datapath and today there is a single counter to track packets dropped
due to any of those reasons.

This patch adds custom software stats for the different reasons packets
may be dropped during tx on the DPDK datapath in OVS.

- MTU drops : drops that occur due to a too large packet size
- Qos drops: drops that occur due to egress QOS
- Tx failures: drops as returned by the DPDK PMD send function

Note that the reason for tx failures is not specificied in OVS. In
practice for vhost ports it is most common that tx failures are because
there are not enough available descriptors, which is usually caused by
misconfiguration of the guest queues and/or because the guest is not
consuming packets fast enough from the queues.

These counters are displayed along with other stats in "ovs-vsctl get
interface  statistics" command and are available for dpdk and
vhostuser/vhostuserclient ports.

Signed-off-by: Sriram Vatala 

---

v9:...
v8:...


Note that signed-off, --- and version info should be like this ^^^.
otherwise the review version comments will be in the git commit log when
it is applied.

> --
> Changes since v8:
> Addressed comments given by Ilya.
> 
> Signed-off-by: Sriram Vatala 
> ---
>  Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
>  lib/netdev-dpdk.c | 81 +++
>  utilities/bugtool/automake.mk |  3 +-
>  utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
>  .../plugins/network-status/openvswitch.xml|  1 +
>  5 files changed, 105 insertions(+), 18 deletions(-)
>  create mode 100755 utilities/bugtool/ovs-bugtool-get-port-stats
> 
> diff --git a/Documentation/topics/dpdk/vhost-user.rst 
> b/Documentation/topics/dpdk/vhost-user.rst
> index 724aa62f6..89388a2bf 100644
> --- a/Documentation/topics/dpdk/vhost-user.rst
> +++ b/Documentation/topics/dpdk/vhost-user.rst
> @@ -551,7 +551,18 @@ processing packets at the required rate.
>  The amount of Tx retries on a vhost-user or vhost-user-client interface can 
> be
>  shown with::
>  
> -  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries
> +  $ ovs-vsctl get Interface dpdkvhostclient0 
> statistics:netdev_dpdk_tx_retries

I think the "netdev_dpdk_" prefixes should be removed for a few reasons,

- These are custom stats that will only be displayed for the dpdk ports
so they don't need any extra prefix to say they are for dpdk ports

- The 'tx_retries' stat is part of OVS 2.12, I don't like to change its
name to a different one in OVS 2.13 unless there is a very good reason

- The existing stats don't have this type of prefixes (granted most of
them are general stats):
# ovs-vsctl get Interface dpdkvhost0 statistics
{"rx_1024_to_1522_packets"=0, "rx_128_to_255_packets"=0,
"rx_1523_to_max_packets"=0, "rx_1_to_64_packets"=25622176,
"rx_256_to_511_packets"=0, "rx_512_to_1023_packets"=0,
"rx_65_to_127_packets"=0, rx_bytes=1537330560, rx_dropped=0,
rx_errors=0, rx_packets=25622176, tx_bytes=3829825920, tx_dropped=0,
tx_packets=63830432, tx_retries=0}

Also, just to note that if there are changes to existing
interfaces/behaviour it should really mention that in the commit message
so it is highlighted.

> +
> +When the guest is not able to consume the packets fast enough, the transmit
> +queue of the port gets filled up i.e queue runs out of free descriptors.
> +This is the most likely reason why dpdk transmit API 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-14 Thread Sriram Vatala via dev
Hi,
Please consider this as gentle remainder and kindly review the patch set.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 11 October 2019 16:58
To: 'ovs-dev@openvswitch.org' ;
'i.maxim...@ovn.org' 
Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi,
Please consider this as a gentle remainder.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 09 October 2019 11:53
To: 'ovs-dev@openvswitch.org' 
Cc: 'ktray...@redhat.com' ; 'Ilya Maximets'
; 'i.maxim...@ovn.org' 
Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi All,
Please consider this as a gentle remainder.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 04 October 2019 13:37
To: 'ovs-dev@openvswitch.org' 
Cc: 'ktray...@redhat.com' ; 'Ilya Maximets'

Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi,
Please consider this as a gentle remainder.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 25 September 2019 09:58
To: 'ovs-dev@openvswitch.org' 
Cc: 'ktray...@redhat.com' ; 'Ilya Maximets'

Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi,
Addressed comments given in patch v8. Kindly review patch v9.
 
Changes since patch v8 :
Rebased my changes on top of Ilya's patch " Reuse vhost function for dpdk
ETH custom stats". Patch set 1/2.
Addressed comments given in patch v8.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 21 September 2019 08:10
To: ovs-dev@openvswitch.org; i.maxim...@samsung.com
Cc: ktray...@redhat.com; Sriram Vatala 
Subject: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

OVS may be unable to transmit packets for multiple reasons and today there
is a single counter to track packets dropped due to any of those reasons.
The most common reason is that a VM is unable to read packets fast enough
causing the vhostuser port transmit queue on the OVS side to become full.
This manifests as a problem with VNFs not receiving all packets. Having a
separate drop counter to track packets dropped because the transmit queue is
full will clearly indicate that the problem is on the VM side and not in
OVS. Similarly maintaining separate counters for all possible drops helps in
indicating sensible cause for packet drops.

This patch adds custom software stats counters to track packets dropped at
port level and these counters are displayed along with other stats in
"ovs-vsctl get interface  statistics"
command. The detailed stats will be available for both dpdk and vhostuser
ports.

--
Changes since v8:
Addressed comments given by Ilya.

Signed-off-by: Sriram Vatala 
---
 Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
 lib/netdev-dpdk.c | 81 +++
 utilities/bugtool/automake.mk |  3 +-
 utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
 .../plugins/network-status/openvswitch.xml|  1 +
 5 files changed, 105 insertions(+), 18 deletions(-)  create mode 100755
utilities/bugtool/ovs-bugtool-get-port-stats

diff --git a/Documentation/topics/dpdk/vhost-user.rst
b/Documentation/topics/dpdk/vhost-user.rst
index 724aa62f6..89388a2bf 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -551,7 +551,18 @@ processing packets at the required rate.
 The amount of Tx retries on a vhost-user or vhost-user-client interface can
be  shown with::
 
-  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries
+  $ ovs-vsctl get Interface dpdkvhostclient0 
+ statistics:netdev_dpdk_tx_retries
+
+When the guest is not able to consume the packets fast enough, the 
+transmit queue of the port gets filled up i.e queue runs out of free
descriptors.
+This is the most likely reason why dpdk transmit API will fail to send 
+packets besides other reasons.
+
+The amount of tx failure drops on a dpdk vhost/physical interface can 
+be shown with::
+
+  $ ovs-vsctl get Interface dpdkvhostclient0 \
+statistics:netdev_dpdk_tx_failure_drops
 
 vhost-user Dequeue Zero Copy (experimental)
 ---
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index
652b57e3b..fd8f9102e 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -174,6 +174,20 @@ static const struct vhost_device_ops
virtio_net_device_ops =
 .destroy_connection = destroy_connection,  };
 
+/* Custom software stats for dpdk ports */ struct netdev_dpdk_sw_stats 
+{
+/* No. of retries when unable to transmit. */
+uint64_t tx_retries;
+/* Packet drops when unable to transmit; Probably Tx queue is full. */
+uint64_t tx_failure_drops;
+/* Packet length greater than device MTU. */
+uint64_t tx_mtu_exceeded_drops;
+/* Packet drops in egress policer processing. */
+uint64_t tx_qos_drops;
+/* Packet drops in ingress policer 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-11 Thread Sriram Vatala via dev
Hi,
Please consider this as a gentle remainder.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 09 October 2019 11:53
To: 'ovs-dev@openvswitch.org' 
Cc: 'ktray...@redhat.com' ; 'Ilya Maximets'
; 'i.maxim...@ovn.org' 
Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi All,
Please consider this as a gentle remainder.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 04 October 2019 13:37
To: 'ovs-dev@openvswitch.org' 
Cc: 'ktray...@redhat.com' ; 'Ilya Maximets'

Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi,
Please consider this as a gentle remainder.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 25 September 2019 09:58
To: 'ovs-dev@openvswitch.org' 
Cc: 'ktray...@redhat.com' ; 'Ilya Maximets'

Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi,
Addressed comments given in patch v8. Kindly review patch v9.
 
Changes since patch v8 :
Rebased my changes on top of Ilya's patch " Reuse vhost function for dpdk
ETH custom stats". Patch set 1/2.
Addressed comments given in patch v8.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 21 September 2019 08:10
To: ovs-dev@openvswitch.org; i.maxim...@samsung.com
Cc: ktray...@redhat.com; Sriram Vatala 
Subject: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

OVS may be unable to transmit packets for multiple reasons and today there
is a single counter to track packets dropped due to any of those reasons.
The most common reason is that a VM is unable to read packets fast enough
causing the vhostuser port transmit queue on the OVS side to become full.
This manifests as a problem with VNFs not receiving all packets. Having a
separate drop counter to track packets dropped because the transmit queue is
full will clearly indicate that the problem is on the VM side and not in
OVS. Similarly maintaining separate counters for all possible drops helps in
indicating sensible cause for packet drops.

This patch adds custom software stats counters to track packets dropped at
port level and these counters are displayed along with other stats in
"ovs-vsctl get interface  statistics"
command. The detailed stats will be available for both dpdk and vhostuser
ports.

--
Changes since v8:
Addressed comments given by Ilya.

Signed-off-by: Sriram Vatala 
---
 Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
 lib/netdev-dpdk.c | 81 +++
 utilities/bugtool/automake.mk |  3 +-
 utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
 .../plugins/network-status/openvswitch.xml|  1 +
 5 files changed, 105 insertions(+), 18 deletions(-)  create mode 100755
utilities/bugtool/ovs-bugtool-get-port-stats

diff --git a/Documentation/topics/dpdk/vhost-user.rst
b/Documentation/topics/dpdk/vhost-user.rst
index 724aa62f6..89388a2bf 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -551,7 +551,18 @@ processing packets at the required rate.
 The amount of Tx retries on a vhost-user or vhost-user-client interface can
be  shown with::
 
-  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries
+  $ ovs-vsctl get Interface dpdkvhostclient0 
+ statistics:netdev_dpdk_tx_retries
+
+When the guest is not able to consume the packets fast enough, the 
+transmit queue of the port gets filled up i.e queue runs out of free
descriptors.
+This is the most likely reason why dpdk transmit API will fail to send 
+packets besides other reasons.
+
+The amount of tx failure drops on a dpdk vhost/physical interface can 
+be shown with::
+
+  $ ovs-vsctl get Interface dpdkvhostclient0 \
+statistics:netdev_dpdk_tx_failure_drops
 
 vhost-user Dequeue Zero Copy (experimental)
 ---
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index
652b57e3b..fd8f9102e 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -174,6 +174,20 @@ static const struct vhost_device_ops
virtio_net_device_ops =
 .destroy_connection = destroy_connection,  };
 
+/* Custom software stats for dpdk ports */ struct netdev_dpdk_sw_stats 
+{
+/* No. of retries when unable to transmit. */
+uint64_t tx_retries;
+/* Packet drops when unable to transmit; Probably Tx queue is full. */
+uint64_t tx_failure_drops;
+/* Packet length greater than device MTU. */
+uint64_t tx_mtu_exceeded_drops;
+/* Packet drops in egress policer processing. */
+uint64_t tx_qos_drops;
+/* Packet drops in ingress policer processing. */
+uint64_t rx_qos_drops;
+};
+
 enum { DPDK_RING_SIZE = 256 };
 BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE));
 enum { DRAIN_TSC = 20ULL };
@@ -413,11 +427,10 @@ struct netdev_dpdk {
 
 PADDED_MEMBERS(CACHE_LINE_SIZE,
 struct netdev_stats stats;
-/* Custom stat for 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-09 Thread Sriram Vatala via dev
Hi All,
Please consider this as a gentle remainder.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 04 October 2019 13:37
To: 'ovs-dev@openvswitch.org' 
Cc: 'ktray...@redhat.com' ; 'Ilya Maximets'

Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi,
Please consider this as a gentle remainder.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 25 September 2019 09:58
To: 'ovs-dev@openvswitch.org' 
Cc: 'ktray...@redhat.com' ; 'Ilya Maximets'

Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi,
Addressed comments given in patch v8. Kindly review patch v9.
 
Changes since patch v8 :
Rebased my changes on top of Ilya's patch " Reuse vhost function for dpdk
ETH custom stats". Patch set 1/2.
Addressed comments given in patch v8.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 21 September 2019 08:10
To: ovs-dev@openvswitch.org; i.maxim...@samsung.com
Cc: ktray...@redhat.com; Sriram Vatala 
Subject: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

OVS may be unable to transmit packets for multiple reasons and today there
is a single counter to track packets dropped due to any of those reasons.
The most common reason is that a VM is unable to read packets fast enough
causing the vhostuser port transmit queue on the OVS side to become full.
This manifests as a problem with VNFs not receiving all packets. Having a
separate drop counter to track packets dropped because the transmit queue is
full will clearly indicate that the problem is on the VM side and not in
OVS. Similarly maintaining separate counters for all possible drops helps in
indicating sensible cause for packet drops.

This patch adds custom software stats counters to track packets dropped at
port level and these counters are displayed along with other stats in
"ovs-vsctl get interface  statistics"
command. The detailed stats will be available for both dpdk and vhostuser
ports.

--
Changes since v8:
Addressed comments given by Ilya.

Signed-off-by: Sriram Vatala 
---
 Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
 lib/netdev-dpdk.c | 81 +++
 utilities/bugtool/automake.mk |  3 +-
 utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
 .../plugins/network-status/openvswitch.xml|  1 +
 5 files changed, 105 insertions(+), 18 deletions(-)  create mode 100755
utilities/bugtool/ovs-bugtool-get-port-stats

diff --git a/Documentation/topics/dpdk/vhost-user.rst
b/Documentation/topics/dpdk/vhost-user.rst
index 724aa62f6..89388a2bf 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -551,7 +551,18 @@ processing packets at the required rate.
 The amount of Tx retries on a vhost-user or vhost-user-client interface can
be  shown with::
 
-  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries
+  $ ovs-vsctl get Interface dpdkvhostclient0 
+ statistics:netdev_dpdk_tx_retries
+
+When the guest is not able to consume the packets fast enough, the 
+transmit queue of the port gets filled up i.e queue runs out of free
descriptors.
+This is the most likely reason why dpdk transmit API will fail to send 
+packets besides other reasons.
+
+The amount of tx failure drops on a dpdk vhost/physical interface can 
+be shown with::
+
+  $ ovs-vsctl get Interface dpdkvhostclient0 \
+statistics:netdev_dpdk_tx_failure_drops
 
 vhost-user Dequeue Zero Copy (experimental)
 ---
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index
652b57e3b..fd8f9102e 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -174,6 +174,20 @@ static const struct vhost_device_ops
virtio_net_device_ops =
 .destroy_connection = destroy_connection,  };
 
+/* Custom software stats for dpdk ports */ struct netdev_dpdk_sw_stats 
+{
+/* No. of retries when unable to transmit. */
+uint64_t tx_retries;
+/* Packet drops when unable to transmit; Probably Tx queue is full. */
+uint64_t tx_failure_drops;
+/* Packet length greater than device MTU. */
+uint64_t tx_mtu_exceeded_drops;
+/* Packet drops in egress policer processing. */
+uint64_t tx_qos_drops;
+/* Packet drops in ingress policer processing. */
+uint64_t rx_qos_drops;
+};
+
 enum { DPDK_RING_SIZE = 256 };
 BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE));
 enum { DRAIN_TSC = 20ULL };
@@ -413,11 +427,10 @@ struct netdev_dpdk {
 
 PADDED_MEMBERS(CACHE_LINE_SIZE,
 struct netdev_stats stats;
-/* Custom stat for retries when unable to transmit. */
-uint64_t tx_retries;
+struct netdev_dpdk_sw_stats *sw_stats;
 /* Protects stats */
 rte_spinlock_t stats_lock;
-/* 4 pad bytes here. */
+/* 36 pad bytes here. */
 );
 
 PADDED_MEMBERS(CACHE_LINE_SIZE,
@@ -1173,7 +1186,9 @@ 

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-10-04 Thread Sriram Vatala via dev
Hi,
Please consider this as a gentle remainder.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 25 September 2019 09:58
To: 'ovs-dev@openvswitch.org' 
Cc: 'ktray...@redhat.com' ; 'Ilya Maximets'

Subject: RE: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

Hi,
Addressed comments given in patch v8. Kindly review patch v9.
 
Changes since patch v8 :
Rebased my changes on top of Ilya's patch " Reuse vhost function for dpdk
ETH custom stats". Patch set 1/2.
Addressed comments given in patch v8.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 21 September 2019 08:10
To: ovs-dev@openvswitch.org; i.maxim...@samsung.com
Cc: ktray...@redhat.com; Sriram Vatala 
Subject: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

OVS may be unable to transmit packets for multiple reasons and today there
is a single counter to track packets dropped due to any of those reasons.
The most common reason is that a VM is unable to read packets fast enough
causing the vhostuser port transmit queue on the OVS side to become full.
This manifests as a problem with VNFs not receiving all packets. Having a
separate drop counter to track packets dropped because the transmit queue is
full will clearly indicate that the problem is on the VM side and not in
OVS. Similarly maintaining separate counters for all possible drops helps in
indicating sensible cause for packet drops.

This patch adds custom software stats counters to track packets dropped at
port level and these counters are displayed along with other stats in
"ovs-vsctl get interface  statistics"
command. The detailed stats will be available for both dpdk and vhostuser
ports.

--
Changes since v8:
Addressed comments given by Ilya.

Signed-off-by: Sriram Vatala 
---
 Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
 lib/netdev-dpdk.c | 81 +++
 utilities/bugtool/automake.mk |  3 +-
 utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
 .../plugins/network-status/openvswitch.xml|  1 +
 5 files changed, 105 insertions(+), 18 deletions(-)  create mode 100755
utilities/bugtool/ovs-bugtool-get-port-stats

diff --git a/Documentation/topics/dpdk/vhost-user.rst
b/Documentation/topics/dpdk/vhost-user.rst
index 724aa62f6..89388a2bf 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -551,7 +551,18 @@ processing packets at the required rate.
 The amount of Tx retries on a vhost-user or vhost-user-client interface can
be  shown with::
 
-  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries
+  $ ovs-vsctl get Interface dpdkvhostclient0 
+ statistics:netdev_dpdk_tx_retries
+
+When the guest is not able to consume the packets fast enough, the 
+transmit queue of the port gets filled up i.e queue runs out of free
descriptors.
+This is the most likely reason why dpdk transmit API will fail to send 
+packets besides other reasons.
+
+The amount of tx failure drops on a dpdk vhost/physical interface can 
+be shown with::
+
+  $ ovs-vsctl get Interface dpdkvhostclient0 \
+statistics:netdev_dpdk_tx_failure_drops
 
 vhost-user Dequeue Zero Copy (experimental)
 ---
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index
652b57e3b..fd8f9102e 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -174,6 +174,20 @@ static const struct vhost_device_ops
virtio_net_device_ops =
 .destroy_connection = destroy_connection,  };
 
+/* Custom software stats for dpdk ports */ struct netdev_dpdk_sw_stats 
+{
+/* No. of retries when unable to transmit. */
+uint64_t tx_retries;
+/* Packet drops when unable to transmit; Probably Tx queue is full. */
+uint64_t tx_failure_drops;
+/* Packet length greater than device MTU. */
+uint64_t tx_mtu_exceeded_drops;
+/* Packet drops in egress policer processing. */
+uint64_t tx_qos_drops;
+/* Packet drops in ingress policer processing. */
+uint64_t rx_qos_drops;
+};
+
 enum { DPDK_RING_SIZE = 256 };
 BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE));
 enum { DRAIN_TSC = 20ULL };
@@ -413,11 +427,10 @@ struct netdev_dpdk {
 
 PADDED_MEMBERS(CACHE_LINE_SIZE,
 struct netdev_stats stats;
-/* Custom stat for retries when unable to transmit. */
-uint64_t tx_retries;
+struct netdev_dpdk_sw_stats *sw_stats;
 /* Protects stats */
 rte_spinlock_t stats_lock;
-/* 4 pad bytes here. */
+/* 36 pad bytes here. */
 );
 
 PADDED_MEMBERS(CACHE_LINE_SIZE,
@@ -1173,7 +1186,9 @@ common_construct(struct netdev *netdev, dpdk_port_t
port_no,
 dev->rte_xstats_ids = NULL;
 dev->rte_xstats_ids_size = 0;
 
-dev->tx_retries = (dev->type == DPDK_DEV_VHOST) ? 0 : UINT64_MAX;
+dev->sw_stats = (struct netdev_dpdk_sw_stats *)
+xzalloc(sizeof *dev->sw_stats);
+

Re: [ovs-dev] [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

2019-09-24 Thread Sriram Vatala via dev
Hi,
Addressed comments given in patch v8. Kindly review patch v9.
 
Changes since patch v8 :
Rebased my changes on top of Ilya's patch " Reuse vhost function for dpdk
ETH custom stats". Patch set 1/2.
Addressed comments given in patch v8.

Thanks & Regards,
Sriram.

-Original Message-
From: Sriram Vatala  
Sent: 21 September 2019 08:10
To: ovs-dev@openvswitch.org; i.maxim...@samsung.com
Cc: ktray...@redhat.com; Sriram Vatala 
Subject: [PATCH v9 2/2] netdev-dpdk:Detailed packet drop statistics

OVS may be unable to transmit packets for multiple reasons and today there
is a single counter to track packets dropped due to any of those reasons.
The most common reason is that a VM is unable to read packets fast enough
causing the vhostuser port transmit queue on the OVS side to become full.
This manifests as a problem with VNFs not receiving all packets. Having a
separate drop counter to track packets dropped because the transmit queue is
full will clearly indicate that the problem is on the VM side and not in
OVS. Similarly maintaining separate counters for all possible drops helps in
indicating sensible cause for packet drops.

This patch adds custom software stats counters to track packets dropped at
port level and these counters are displayed along with other stats in
"ovs-vsctl get interface  statistics"
command. The detailed stats will be available for both dpdk and vhostuser
ports.

--
Changes since v8:
Addressed comments given by Ilya.

Signed-off-by: Sriram Vatala 
---
 Documentation/topics/dpdk/vhost-user.rst  | 13 ++-
 lib/netdev-dpdk.c | 81 +++
 utilities/bugtool/automake.mk |  3 +-
 utilities/bugtool/ovs-bugtool-get-port-stats  | 25 ++
 .../plugins/network-status/openvswitch.xml|  1 +
 5 files changed, 105 insertions(+), 18 deletions(-)  create mode 100755
utilities/bugtool/ovs-bugtool-get-port-stats

diff --git a/Documentation/topics/dpdk/vhost-user.rst
b/Documentation/topics/dpdk/vhost-user.rst
index 724aa62f6..89388a2bf 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -551,7 +551,18 @@ processing packets at the required rate.
 The amount of Tx retries on a vhost-user or vhost-user-client interface can
be  shown with::
 
-  $ ovs-vsctl get Interface dpdkvhostclient0 statistics:tx_retries
+  $ ovs-vsctl get Interface dpdkvhostclient0 
+ statistics:netdev_dpdk_tx_retries
+
+When the guest is not able to consume the packets fast enough, the 
+transmit queue of the port gets filled up i.e queue runs out of free
descriptors.
+This is the most likely reason why dpdk transmit API will fail to send 
+packets besides other reasons.
+
+The amount of tx failure drops on a dpdk vhost/physical interface can 
+be shown with::
+
+  $ ovs-vsctl get Interface dpdkvhostclient0 \
+statistics:netdev_dpdk_tx_failure_drops
 
 vhost-user Dequeue Zero Copy (experimental)
 ---
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index
652b57e3b..fd8f9102e 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -174,6 +174,20 @@ static const struct vhost_device_ops
virtio_net_device_ops =
 .destroy_connection = destroy_connection,  };
 
+/* Custom software stats for dpdk ports */ struct netdev_dpdk_sw_stats 
+{
+/* No. of retries when unable to transmit. */
+uint64_t tx_retries;
+/* Packet drops when unable to transmit; Probably Tx queue is full. */
+uint64_t tx_failure_drops;
+/* Packet length greater than device MTU. */
+uint64_t tx_mtu_exceeded_drops;
+/* Packet drops in egress policer processing. */
+uint64_t tx_qos_drops;
+/* Packet drops in ingress policer processing. */
+uint64_t rx_qos_drops;
+};
+
 enum { DPDK_RING_SIZE = 256 };
 BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE));
 enum { DRAIN_TSC = 20ULL };
@@ -413,11 +427,10 @@ struct netdev_dpdk {
 
 PADDED_MEMBERS(CACHE_LINE_SIZE,
 struct netdev_stats stats;
-/* Custom stat for retries when unable to transmit. */
-uint64_t tx_retries;
+struct netdev_dpdk_sw_stats *sw_stats;
 /* Protects stats */
 rte_spinlock_t stats_lock;
-/* 4 pad bytes here. */
+/* 36 pad bytes here. */
 );
 
 PADDED_MEMBERS(CACHE_LINE_SIZE,
@@ -1173,7 +1186,9 @@ common_construct(struct netdev *netdev, dpdk_port_t
port_no,
 dev->rte_xstats_ids = NULL;
 dev->rte_xstats_ids_size = 0;
 
-dev->tx_retries = (dev->type == DPDK_DEV_VHOST) ? 0 : UINT64_MAX;
+dev->sw_stats = (struct netdev_dpdk_sw_stats *)
+xzalloc(sizeof *dev->sw_stats);
+dev->sw_stats->tx_retries = (dev->type == DPDK_DEV_VHOST) ? 0 : 
+ UINT64_MAX;
 
 return 0;
 }
@@ -1359,6 +1374,7 @@ common_destruct(struct netdev_dpdk *dev)
 ovs_list_remove(>list_node);
 free(ovsrcu_get_protected(struct ingress_policer *,
   >ingress_policer));
+