date:20160517

[dpdk-dev] [PATCH] librte_ether: use RTE_ETH_VALID_PORTID_OR_ERR_RET to check port_id

2016-05-17 Thread Mauricio Vásquez

Hello Thomas,



On Fri, May 13, 2016 at 6:20 PM, Thomas Monjalon 
wrote:

> 2016-04-29 17:23, Mauricio Vasquez B:
> > The RTE_ETH_VALID_PORTID_OR_ERR_RET macro is used in some places
> > to check if a port id is valid or not. This commit makes use of it in
> > some new parts of the code.
>
> There are other occurences:
> rte_eth_dev_socket_id
>
I missed it.

> rte_eth_add_rx_callback
> rte_eth_add_tx_callback
> rte_eth_remove_rx_callback
> rte_eth_remove_tx_callback
>
The macro can not be used on those ones because they set the rte_errno
variable before returning.

> I think it could be done also in examples/ethtool/lib.
>
I'll send v2 including that one.

Mauricio V,

[dpdk-dev] [PATCH v2] librte_ether: use RTE_ETH_VALID_PORTID_OR_ERR_RET to check port_id

2016-05-17 Thread Mauricio Vasquez B

The RTE_ETH_VALID_PORTID_OR_ERR_RET macro is used in some places
to check if a port id is valid or not. This commit makes use of it in
some new parts of the code.

Signed-off-by: Mauricio Vasquez B 
---
v2:
 - add missed case
 - change also cases in examples/ethtool/lib/rte_ethtool.c
 examples/ethtool/lib/rte_ethtool.c | 15 +++--
 lib/librte_ether/rte_ethdev.c  | 43 ++
 2 files changed, 18 insertions(+), 40 deletions(-)

diff --git a/examples/ethtool/lib/rte_ethtool.c 
b/examples/ethtool/lib/rte_ethtool.c
index 42e05f1..9b18e46 100644
--- a/examples/ethtool/lib/rte_ethtool.c
+++ b/examples/ethtool/lib/rte_ethtool.c
@@ -51,8 +51,7 @@ rte_ethtool_get_drvinfo(uint8_t port_id, struct 
ethtool_drvinfo *drvinfo)
if (drvinfo == NULL)
return -EINVAL;

-   if (!rte_eth_dev_is_valid_port(port_id))
-   return -ENODEV;
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);

memset(_info, 0, sizeof(dev_info));
rte_eth_dev_info_get(port_id, _info);
@@ -120,8 +119,8 @@ rte_ethtool_get_link(uint8_t port_id)
 {
struct rte_eth_link link;

-   if (!rte_eth_dev_is_valid_port(port_id))
-   return -ENODEV;
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+
rte_eth_link_get(port_id, );
return link.link_status;
 }
@@ -267,8 +266,8 @@ rte_ethtool_net_open(uint8_t port_id)
 int
 rte_ethtool_net_stop(uint8_t port_id)
 {
-   if (!rte_eth_dev_is_valid_port(port_id))
-   return -ENODEV;
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+
rte_eth_dev_stop(port_id);

return 0;
@@ -277,8 +276,8 @@ rte_ethtool_net_stop(uint8_t port_id)
 int
 rte_ethtool_net_get_mac_addr(uint8_t port_id, struct ether_addr *addr)
 {
-   if (!rte_eth_dev_is_valid_port(port_id))
-   return -ENODEV;
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+
if (addr == NULL)
return -EINVAL;
rte_eth_macaddr_get(port_id, addr);
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index a31018e..4c06fb4 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -369,8 +369,8 @@ rte_eth_dev_is_valid_port(uint8_t port_id)
 int
 rte_eth_dev_socket_id(uint8_t port_id)
 {
-   if (!rte_eth_dev_is_valid_port(port_id))
-   return -1;
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -1);
+
return rte_eth_devices[port_id].data->numa_node;
 }

@@ -383,8 +383,8 @@ rte_eth_dev_count(void)
 static enum rte_eth_dev_type
 rte_eth_dev_get_device_type(uint8_t port_id)
 {
-   if (!rte_eth_dev_is_valid_port(port_id))
-   return RTE_ETH_DEV_UNKNOWN;
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, RTE_ETH_DEV_UNKNOWN);
+
return rte_eth_devices[port_id].dev_type;
 }

@@ -479,10 +479,7 @@ rte_eth_dev_is_detachable(uint8_t port_id)
 {
uint32_t dev_flags;

-   if (!rte_eth_dev_is_valid_port(port_id)) {
-   RTE_PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
-   return -EINVAL;
-   }
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);

switch (rte_eth_devices[port_id].data->kdrv) {
case RTE_KDRV_IGB_UIO:
@@ -1994,10 +1991,7 @@ rte_eth_dev_rss_reta_query(uint8_t port_id,
struct rte_eth_dev *dev;
int ret;

-   if (port_id >= nb_ports) {
-   RTE_PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
-   return -ENODEV;
-   }
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);

/* Check mask bits */
ret = rte_eth_check_reta_mask(reta_conf, reta_size);
@@ -2641,10 +2635,7 @@ rte_eth_dev_rx_intr_ctl(uint8_t port_id, int epfd, int 
op, void *data)
uint16_t qid;
int rc;

-   if (!rte_eth_dev_is_valid_port(port_id)) {
-   RTE_PMD_DEBUG_TRACE("Invalid port_id=%u\n", port_id);
-   return -ENODEV;
-   }
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);

dev = _eth_devices[port_id];
intr_handle = >pci_dev->intr_handle;
@@ -2699,10 +2690,7 @@ rte_eth_dev_rx_intr_ctl_q(uint8_t port_id, uint16_t 
queue_id,
struct rte_intr_handle *intr_handle;
int rc;

-   if (!rte_eth_dev_is_valid_port(port_id)) {
-   RTE_PMD_DEBUG_TRACE("Invalid port_id=%u\n", port_id);
-   return -ENODEV;
-   }
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);

dev = _eth_devices[port_id];
if (queue_id >= dev->data->nb_rx_queues) {
@@ -2734,10 +2722,7 @@ rte_eth_dev_rx_intr_enable(uint8_t port_id,
 {
struct rte_eth_dev *dev;

-   if (!rte_eth_dev_is_valid_port(port_id)) {
-   RTE_PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
-   return -ENODEV;
-   }
+   RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);

dev = _eth_devices[port_id];

@@ -2751,10 +2736,7 @@

[dpdk-dev] [PATCH v3 11/11] app/test: do not dump PCI devices in blacklist test

2016-05-17 Thread Jan Viktorin

Dumping of devices in a unittest is useless. Instead, test whether the test
has been set up well - i.e. there are no devices.

Signed-off-by: Jan Viktorin 
---
 app/test/test_pci.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 69066cb..5506a11 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -150,8 +150,8 @@ test_pci_blacklist(void)
 {
struct rte_devargs_list save_devargs_list;

-   printf("Dump all devices\n");
-   rte_eal_pci_dump(stdout);
+   TEST_ASSERT(TAILQ_EMPTY(_driver_list),
+   "pci_driver_list not empty");

rte_eal_pci_register(_driver);
rte_eal_pci_register(_driver2);
-- 
2.8.0

[dpdk-dev] [PATCH v3 10/11] app/test: scan PCI bus using a fake sysfs

2016-05-17 Thread Jan Viktorin

Scan the PCI bus by providing a fake sysfs with a PCI device. The fake sysfs
is a packed file hierarchy linked into the test.

Signed-off-by: Jan Viktorin 
---
 app/test/Makefile  |   1 +
 app/test/test_pci.c|  58 +++--
 .../bus/pci/devices/:01:00.0/class |   1 +
 .../bus/pci/devices/:01:00.0/config| Bin 0 -> 64 bytes
 .../devices/:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/:01:00.0/device|   1 +
 .../bus/pci/devices/:01:00.0/dma_mask_bits |   1 +
 .../bus/pci/devices/:01:00.0/enable|   1 +
 .../bus/pci/devices/:01:00.0/irq   |   1 +
 .../bus/pci/devices/:01:00.0/modalias  |   1 +
 .../bus/pci/devices/:01:00.0/msi_bus   |   1 +
 .../bus/pci/devices/:01:00.0/numa_node |   1 +
 .../bus/pci/devices/:01:00.0/resource  |  13 +
 .../bus/pci/devices/:01:00.0/sriov_numvfs  |   1 +
 .../bus/pci/devices/:01:00.0/sriov_totalvfs|   1 +
 .../bus/pci/devices/:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/:01:00.0/uevent|   6 +++
 .../bus/pci/devices/:01:00.0/vendor|   1 +
 19 files changed, 89 insertions(+), 3 deletions(-)
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/config
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/device
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/irq
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/msi_bus
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/numa_node
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/resource
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/sriov_numvfs
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/sriov_totalvfs
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/subsystem_device
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/vendor

diff --git a/app/test/Makefile b/app/test/Makefile
index de5fa50..1c6c29e 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -73,6 +73,7 @@ SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
 $(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
+$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
 SRCS-y += test_per_lcore.c
diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index b8b729c..69066cb 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -43,6 +43,7 @@
 #include 

 #include "test.h"
+#include "resource.h"

 /* Generic maximum number of drivers to have room to allocate all drivers */
 #define NUM_MAX_DRIVERS 256
@@ -215,37 +216,88 @@ static int test_pci_sysfs(void)
return 0;
 }

-/* backup real drivers (not used for testing) */
+/* backup real devices & drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+struct pci_device_list real_pci_device_list =
+   TAILQ_HEAD_INITIALIZER(real_pci_device_list);
+
+REGISTER_LINKED_RESOURCE(test_pci_sysfs);

 static int
 test_pci_setup(void)
 {
+   struct rte_pci_device *dev;
struct rte_pci_driver *dr;
+   const struct resource *r;
+   int ret;
+
+   r = resource_find("test_pci_sysfs");
+   TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+   ret = resource_untar(r);
+   TEST_ASSERT_SUCCESS(ret, "failed to untar %s", r->name);
+
+   ret = setenv("SYSFS_PCI_DEVICES", "test_pci_sysfs/bus/pci/devices", 1);
+   TEST_ASSERT_SUCCESS(ret, "failed to setenv");

-   /* Unregister original driver list */
+   /* Unregister original devices & drivers lists */
while (!TAILQ_EMPTY(_driver_list)) {
dr = TAILQ_FIRST(_driver_list);
rte_eal_pci_unregister(dr);
TAILQ_INSERT_TAIL(_pci_driver_list, dr, next);
}

+   while (!TAILQ_EMPTY(_device_list)) {
+   dev = TAILQ_FIRST(_device_list);
+

[dpdk-dev] [PATCH v3 09/11] eal/pci: allow to override sysfs

2016-05-17 Thread Jan Viktorin

The SYSFS_PCI_DEVICES is a constant that makes the PCI testing difficult as
it points to an absolute path. We remove using this constant and introducing
a function pci_get_sysfs_path that gives the same value. However, the user can
pass a SYSFS_PCI_DEVICES env variable to override the path. It is now possible
to create a fake sysfs hierarchy for testing.

Signed-off-by: Jan Viktorin 
---
v3:
* changed subject
* test_pci_sysfs has been slightly modified to be more understandable
* fixed whitespace in *version.map files
---
 app/test/test_pci.c | 28 +
 drivers/net/szedata2/rte_eth_szedata2.c |  2 +-
 drivers/net/virtio/virtio_pci.c |  2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  7 +++
 lib/librte_eal/common/eal_common_pci.c  | 13 
 lib/librte_eal/common/include/rte_pci.h |  2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c   |  6 +++---
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c   |  7 ---
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c  |  2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  7 +++
 10 files changed, 66 insertions(+), 10 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 6445ab0..b8b729c 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -190,6 +190,31 @@ test_pci_blacklist(void)
return 0;
 }

+static int test_pci_sysfs(void)
+{
+   const char *orig;
+   const char *newpath;
+   int ret;
+
+   orig = pci_get_sysfs_path();
+   ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);
+   TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
+
+   newpath = pci_get_sysfs_path();
+   TEST_ASSERT(!strcmp(newpath, "My Documents"),
+   "pci_get_sysfs_path() should return 'My Documents' "
+   "but gives %s", newpath);
+
+   ret = setenv("SYSFS_PCI_DEVICES", orig, 1);
+   TEST_ASSERT_SUCCESS(ret, "Failed setenv back to '%s'", orig);
+
+   newpath = pci_get_sysfs_path();
+   TEST_ASSERT(!strcmp(orig, newpath),
+   "pci_get_sysfs_path returned unexpected path: "
+   "%s (expected: %s)", newpath, orig);
+   return 0;
+}
+
 /* backup real drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
@@ -227,6 +252,9 @@ test_pci_cleanup(void)
 int
 test_pci(void)
 {
+   if (test_pci_sysfs())
+   return -1;
+
if (test_pci_setup())
return -1;

diff --git a/drivers/net/szedata2/rte_eth_szedata2.c 
b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..985a8d6 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1481,7 +1481,7 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev)
return -EINVAL;
}
snprintf(rsc_filename, PATH_MAX,
-   SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%u",
+   "%s/" PCI_PRI_FMT "/resource%u", pci_get_sysfs_path(),
pci_addr->domain, pci_addr->bus,
pci_addr->devid, pci_addr->function, PCI_RESOURCE_NUMBER);
fd = open(rsc_filename, O_RDWR);
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 9cdca06..845141b 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -179,7 +179,7 @@ legacy_virtio_has_msix(const struct rte_pci_addr *loc)
char dirname[PATH_MAX];

snprintf(dirname, sizeof(dirname),
-SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/msi_irqs",
+"%s/" PCI_PRI_FMT "/msi_irqs", pci_get_sysfs_path(),
 loc->domain, loc->bus, loc->devid, loc->function);

d = opendir(dirname);
diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map 
b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 58c2951..f8c3dea 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -151,3 +151,10 @@ DPDK_16.04 {
rte_eal_primary_proc_alive;

 } DPDK_2.2;
+
+DPDK_16.07 {
+   global:
+
+   pci_get_sysfs_path;
+
+} DPDK_16.04;
diff --git a/lib/librte_eal/common/eal_common_pci.c 
b/lib/librte_eal/common/eal_common_pci.c
index 3cae4cb..0ec3b61 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -85,6 +85,19 @@
 struct pci_driver_list pci_driver_list;
 struct pci_device_list pci_device_list;

+#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+
+const char *pci_get_sysfs_path(void)
+{
+   const char *path = NULL;
+
+   path = getenv("SYSFS_PCI_DEVICES");
+   if (path == NULL)
+   return SYSFS_PCI_DEVICES;
+
+   return path;
+}
+
 static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 {
struct rte_devargs *devargs;
diff --git

[dpdk-dev] [PATCH v3 08/11] app/test: convert current pci_test into a single test case

2016-05-17 Thread Jan Viktorin

The current test_pci is just a single test case that tests the blacklisting
of devices. Rename it to test_pci_blacklist and call it from the test_pci.
The setup and cleanup are moved out of the test_pci_blacklist entirely
to cover all other tests.

Signed-off-by: Jan Viktorin 
---
v3:
* improved commit description
---
 app/test/test_pci.c | 85 +
 1 file changed, 47 insertions(+), 38 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 50078a0..6445ab0 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,51 +144,14 @@ static void free_devargs_list(void)
}
 }

-/* backup real drivers (not used for testing) */
-struct pci_driver_list real_pci_driver_list =
-   TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
-
 static int
-test_pci_setup(void)
-{
-   struct rte_pci_driver *dr;
-
-   /* Unregister original driver list */
-   while (!TAILQ_EMPTY(_driver_list)) {
-   dr = TAILQ_FIRST(_driver_list);
-   rte_eal_pci_unregister(dr);
-   TAILQ_INSERT_TAIL(_pci_driver_list, dr, next);
-   }
-
-   return 0;
-}
-
-static int
-test_pci_cleanup(void)
-{
-   struct rte_pci_driver *dr;
-
-   /* Restore original driver list */
-   while (!TAILQ_EMPTY(_pci_driver_list)) {
-   dr = TAILQ_FIRST(_pci_driver_list);
-   TAILQ_REMOVE(_pci_driver_list, dr, next);
-   rte_eal_pci_register(dr);
-   }
-
-   return 0;
-}
-
-int
-test_pci(void)
+test_pci_blacklist(void)
 {
struct rte_devargs_list save_devargs_list;

printf("Dump all devices\n");
rte_eal_pci_dump(stdout);

-   if (test_pci_setup())
-   return -1;
-
rte_eal_pci_register(_driver);
rte_eal_pci_register(_driver2);

@@ -224,6 +187,52 @@ test_pci(void)
rte_eal_pci_unregister(_driver);
rte_eal_pci_unregister(_driver2);

+   return 0;
+}
+
+/* backup real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+   TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
+static int
+test_pci_setup(void)
+{
+   struct rte_pci_driver *dr;
+
+   /* Unregister original driver list */
+   while (!TAILQ_EMPTY(_driver_list)) {
+   dr = TAILQ_FIRST(_driver_list);
+   rte_eal_pci_unregister(dr);
+   TAILQ_INSERT_TAIL(_pci_driver_list, dr, next);
+   }
+
+   return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+   struct rte_pci_driver *dr;
+
+   /* Restore original driver list */
+   while (!TAILQ_EMPTY(_pci_driver_list)) {
+   dr = TAILQ_FIRST(_pci_driver_list);
+   TAILQ_REMOVE(_pci_driver_list, dr, next);
+   rte_eal_pci_register(dr);
+   }
+
+   return 0;
+}
+
+int
+test_pci(void)
+{
+   if (test_pci_setup())
+   return -1;
+
+   if (test_pci_blacklist())
+   return -1;
+
if (test_pci_cleanup())
return -1;

-- 
2.8.0

[dpdk-dev] [PATCH v3 07/11] app/test: extract test_pci_setup and test_pci_cleanup

2016-05-17 Thread Jan Viktorin

Signed-off-by: Jan Viktorin 
---
 app/test/test_pci.c | 47 ++-
 1 file changed, 34 insertions(+), 13 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 8b7c8bb..50078a0 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -148,21 +148,46 @@ static void free_devargs_list(void)
 struct pci_driver_list real_pci_driver_list =
TAILQ_HEAD_INITIALIZER(real_pci_driver_list);

+static int
+test_pci_setup(void)
+{
+   struct rte_pci_driver *dr;
+
+   /* Unregister original driver list */
+   while (!TAILQ_EMPTY(_driver_list)) {
+   dr = TAILQ_FIRST(_driver_list);
+   rte_eal_pci_unregister(dr);
+   TAILQ_INSERT_TAIL(_pci_driver_list, dr, next);
+   }
+
+   return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+   struct rte_pci_driver *dr;
+
+   /* Restore original driver list */
+   while (!TAILQ_EMPTY(_pci_driver_list)) {
+   dr = TAILQ_FIRST(_pci_driver_list);
+   TAILQ_REMOVE(_pci_driver_list, dr, next);
+   rte_eal_pci_register(dr);
+   }
+
+   return 0;
+}
+
 int
 test_pci(void)
 {
struct rte_devargs_list save_devargs_list;
-   struct rte_pci_driver *dr = NULL;

printf("Dump all devices\n");
rte_eal_pci_dump(stdout);

-   /* Unregister all previous drivers */
-   while (!TAILQ_EMPTY(_driver_list)) {
-   dr = TAILQ_FIRST(_driver_list);
-   rte_eal_pci_unregister(dr);
-   TAILQ_INSERT_TAIL(_pci_driver_list, dr, next);
-   }
+   if (test_pci_setup())
+   return -1;

rte_eal_pci_register(_driver);
rte_eal_pci_register(_driver2);
@@ -199,12 +224,8 @@ test_pci(void)
rte_eal_pci_unregister(_driver);
rte_eal_pci_unregister(_driver2);

-   /* Restore original driver list */
-   while (!TAILQ_EMPTY(_pci_driver_list)) {
-   dr = TAILQ_FIRST(_pci_driver_list);
-   TAILQ_REMOVE(_pci_driver_list, dr, next);
-   rte_eal_pci_register(dr);
-   }
+   if (test_pci_cleanup())
+   return -1;

return 0;
 }
-- 
2.8.0

[dpdk-dev] [PATCH v3 06/11] app/test: use linked list to store PCI drivers

2016-05-17 Thread Jan Viktorin

The test unregisters all drivers before start. The drivers were stored
into a fixed-sized array. This is inflexible. This patch change this to
utilize a linked list for the same purpose.

Signed-off-by: Jan Viktorin 
---
v3:
* fixed commit message
* used "backup" to describe the real_pci_driver_list
---
 app/test/test_pci.c | 18 --
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 0ed357e..8b7c8bb 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,21 +144,24 @@ static void free_devargs_list(void)
}
 }

+/* backup real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+   TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
 int
 test_pci(void)
 {
struct rte_devargs_list save_devargs_list;
struct rte_pci_driver *dr = NULL;
-   struct rte_pci_driver *save_pci_driver_list[NUM_MAX_DRIVERS];
-   unsigned i, num_drivers = 0;

printf("Dump all devices\n");
rte_eal_pci_dump(stdout);

/* Unregister all previous drivers */
-   TAILQ_FOREACH(dr, _driver_list, next) {
+   while (!TAILQ_EMPTY(_driver_list)) {
+   dr = TAILQ_FIRST(_driver_list);
rte_eal_pci_unregister(dr);
-   save_pci_driver_list[num_drivers++] = dr;
+   TAILQ_INSERT_TAIL(_pci_driver_list, dr, next);
}

rte_eal_pci_register(_driver);
@@ -197,8 +200,11 @@ test_pci(void)
rte_eal_pci_unregister(_driver2);

/* Restore original driver list */
-   for (i = 0; i < num_drivers; i++)
-   rte_eal_pci_register(save_pci_driver_list[i]);
+   while (!TAILQ_EMPTY(_pci_driver_list)) {
+   dr = TAILQ_FIRST(_pci_driver_list);
+   TAILQ_REMOVE(_pci_driver_list, dr, next);
+   rte_eal_pci_register(dr);
+   }

return 0;
 }
-- 
2.8.0

[dpdk-dev] [PATCH v3 05/11] app/test: support resources archived by tar

2016-05-17 Thread Jan Viktorin

When a more complex resource (a file hierarchy) is needed, packing every single
file as a single resource would be very ineffective. For that purpose, it is
possible to pack the files into a tar archive, extract it before test from the
resource and finally clean up all the created files.

This patch introduces functions resource_untar and resource_rm_by_tar to
perform those tasks. An example of using those functions is included as a test.

A new dependency is required to build the app/test: libarchive.

Signed-off-by: Jan Viktorin 
---
 app/test/Makefile|   9 +++
 app/test/resource.c  | 195 +++
 app/test/resource.h  |  13 
 app/test/test_resource.c |  29 +++
 4 files changed, 246 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index cfe68a6..de5fa50 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -51,6 +51,13 @@ $(1).res.o: $(2)
/dev/stdin $$@ < $$<
 endef

+define linked_tar_resource
+$(1).tar: $(2)
+   tar -C $$(dir $$<) -cf $$@ $$(notdir $$<)
+
+$(call linked_resource,$(1),$(1).tar)
+endef
+
 #
 # library name
 #
@@ -64,6 +71,7 @@ SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
+$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
@@ -183,6 +191,7 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE

 LDLIBS += -lm
+LDLIBS += -larchive

 # Disable VTA for memcpy test
 ifeq ($(CC), gcc)
diff --git a/app/test/resource.c b/app/test/resource.c
index acb63c1..668fc52 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -33,6 +33,8 @@

 #include 
 #include 
+#include 
+#include 
 #include 
 #include 

@@ -95,6 +97,199 @@ int resource_fwrite_file(const struct resource *r, const 
char *fname)
return ret;
 }

+static int do_copy(struct archive *r, struct archive *w)
+{
+   const void *buf;
+   size_t len;
+   off_t off;
+   int ret;
+
+   while (1) {
+   ret = archive_read_data_block(r, , , );
+   if (ret == ARCHIVE_RETRY)
+   continue;
+
+   if (ret == ARCHIVE_EOF)
+   return 0;
+
+   if (ret != ARCHIVE_OK)
+   return ret;
+
+   do {
+   ret = archive_write_data_block(w, buf, len, off);
+   if (ret != ARCHIVE_OK && ret != ARCHIVE_RETRY)
+   return ret;
+   } while (ret != ARCHIVE_OK);
+   }
+}
+
+int resource_untar(const struct resource *res)
+{
+   struct archive *r;
+   struct archive *w;
+   struct archive_entry *e;
+   void *p;
+   int flags = 0;
+   int ret;
+
+   p = malloc(resource_size(res));
+   if (p == NULL)
+   rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+   memcpy(p, res->begin, resource_size(res));
+
+   r = archive_read_new();
+   if (r == NULL) {
+   free(p);
+   return -1;
+   }
+
+   archive_read_support_format_all(r);
+   archive_read_support_filter_all(r);
+
+   w = archive_write_disk_new();
+   if (w == NULL) {
+   archive_read_free(r);
+   free(p);
+   return -1;
+   }
+
+   flags |= ARCHIVE_EXTRACT_PERM;
+   flags |= ARCHIVE_EXTRACT_FFLAGS;
+   archive_write_disk_set_options(w, flags);
+   archive_write_disk_set_standard_lookup(w);
+
+   ret = archive_read_open_memory(r, p, resource_size(res));
+   if (ret != ARCHIVE_OK)
+   goto fail;
+
+   while (1) {
+   ret = archive_read_next_header(r, );
+   if (ret == ARCHIVE_EOF)
+   break;
+   if (ret != ARCHIVE_OK)
+   goto fail;
+
+   ret = archive_write_header(w, e);
+   if (ret == ARCHIVE_EOF)
+   break;
+   if (ret != ARCHIVE_OK)
+   goto fail;
+
+   if (archive_entry_size(e) == 0)
+   continue;
+
+   ret = do_copy(r, w);
+   if (ret != ARCHIVE_OK)
+   goto fail;
+
+   ret = archive_write_finish_entry(w);
+   if (ret != ARCHIVE_OK)
+   goto fail;
+   }
+
+   archive_write_free(w);
+   archive_read_free(r);
+   free(p);
+   return 0;
+
+fail:
+   archive_write_free(w);
+   archive_read_free(r);
+   free(p);
+   rte_panic("Failed: %s\n", archive_error_string(r));
+   return -1;
+}
+
+int resource_rm_by_tar(const struct resource *res)
+{
+   struct archive *r;
+   struct archive_entry *e;
+   void *p;
+   int try_again = 1;
+   int attempts = 0;
+   int ret;
+
+   p = malloc(resource_size(res));
+

[dpdk-dev] [PATCH v3 04/11] app/test: add functions to create files from resources

2016-05-17 Thread Jan Viktorin

A resource can be written into the target filesystem by calling resource_fwrite
or resource_fwrite_file. Such file can be created before a test is started and
removed after the test finishes.

Signed-off-by: Jan Viktorin 
---
 app/test/resource.c  | 35 +++
 app/test/resource.h  | 14 ++
 app/test/test_resource.c | 10 ++
 3 files changed, 59 insertions(+)

diff --git a/app/test/resource.c b/app/test/resource.c
index 30513db..acb63c1 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -31,6 +31,7 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */

+#include 
 #include 
 #include 
 #include 
@@ -60,6 +61,40 @@ const struct resource *resource_find(const char *name)
return NULL;
 }

+int resource_fwrite(const struct resource *r, FILE *f)
+{
+   const size_t goal = resource_size(r);
+   size_t total = 0;
+
+   while (total < goal) {
+   size_t wlen = fwrite(r->begin + total, 1, goal - total, f);
+   if (wlen == 0) {
+   perror(__func__);
+   return -1;
+   }
+
+   total += wlen;
+   }
+
+   return 0;
+}
+
+int resource_fwrite_file(const struct resource *r, const char *fname)
+{
+   FILE *f;
+   int ret;
+
+   f = fopen(fname, "w");
+   if (f == NULL) {
+   perror(__func__);
+   return -1;
+   }
+
+   ret = resource_fwrite(r, f);
+   fclose(f);
+   return ret;
+}
+
 void resource_register(struct resource *r)
 {
TAILQ_INSERT_TAIL(_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 2ef817f..849df05 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -45,6 +45,7 @@
  */

 #include 
+#include 
 #include 

 #include 
@@ -75,6 +76,19 @@ size_t resource_size(const struct resource *r);
 const struct resource *resource_find(const char *name);

 /**
+ * Write the raw data of the resource to the given file.
+ * @return 0 on success
+ */
+int resource_fwrite(const struct resource *r, FILE *f);
+
+/**
+ * Write the raw data of the resource to the given file given by name.
+ * The name is relative to the current working directory.
+ * @return 0 on success
+ */
+int resource_fwrite_file(const struct resource *r, const char *fname);
+
+/**
  * Register a resource in the global list of resources.
  * Not intended for direct use, please check the REGISTER_RESOURCE
  * macro.
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index b397fa8..3d1bf00 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -65,6 +65,7 @@ REGISTER_LINKED_RESOURCE(test_resource_c);
 static int test_resource_c(void)
 {
const struct resource *r;
+   FILE *f;

r = resource_find("test_resource_c");
TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
@@ -72,6 +73,15 @@ static int test_resource_c(void)
"Found resource %s, expected test_resource_c",
r->name);

+   TEST_ASSERT_SUCCESS(resource_fwrite_file(r, "test_resource.c"),
+   "Failed to to write file %s", r->name);
+
+   f = fopen("test_resource.c", "r");
+   TEST_ASSERT_NOT_NULL(f,
+   "Missing extracted file resource.c");
+   fclose(f);
+   remove("test_resource.c");
+
return 0;
 }

-- 
2.8.0

[dpdk-dev] [PATCH v3 03/11] app/test: support resources externally linked

2016-05-17 Thread Jan Viktorin

To include resources from other source that the C source code we can take
advantage of the objcopy behaviour, i.e. packing of an arbitrary file as an
object file that is linked to the target program.

A linked object file is always accessible as a pair

extern const char beg_;
extern const char end_;
(extern const char siz_;)

A unit test that packs the resource.c source file is included.

Signed-off-by: Jan Viktorin 
---
 app/test/Makefile| 19 +++
 app/test/resource.h  | 10 ++
 app/test/test_resource.c | 18 ++
 3 files changed, 47 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index 7fbdd18..cfe68a6 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -33,6 +33,24 @@ include $(RTE_SDK)/mk/rte.vars.mk

 ifeq ($(CONFIG_RTE_APP_TEST),y)

+# Define an externally linked resource. A linked resource is an arbitrary
+# file that is linked into the test binary. The application refers to this
+# resource by name. The linked generates identifiers beg_ and end_
+# for referencing by the C code.
+#
+# Parameters: , 
+define linked_resource
+SRCS-y += $(1).res.o
+$(1).res.o: $(2)
+   $(OBJCOPY) -I binary -B $(RTE_OBJCOPY_ARCH) -O $(RTE_OBJCOPY_TARGET) \
+   --rename-section \
+   .data=.rodata,alloc,load,data,contents,readonly  \
+   --redefine-sym _binary__dev_stdin_start=beg_$(1) \
+   --redefine-sym _binary__dev_stdin_end=end_$(1)   \
+   --redefine-sym _binary__dev_stdin_size=siz_$(1)  \
+   /dev/stdin $$@ < $$<
+endef
+
 #
 # library name
 #
@@ -45,6 +63,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
+$(eval $(call linked_resource,test_resource_c,resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.h b/app/test/resource.h
index 37c589c..2ef817f 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -82,6 +82,16 @@ const struct resource *resource_find(const char *name);
 void resource_register(struct resource *r);

 /**
+ * Definition of a resource linked externally (by means of the used toolchain).
+ * Only the base name of the resource is expected. The name refers to the
+ * linked pointers beg_ and end_ provided externally.
+ */
+#define REGISTER_LINKED_RESOURCE(n) \
+extern const char beg_ ##n; \
+extern const char end_ ##n; \
+REGISTER_RESOURCE(n, _ ##n, _ ##n); \
+
+/**
  * Definition of a resource described by its name, and pointers begin, end.
  */
 #define REGISTER_RESOURCE(n, b, e) \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 69391ad..b397fa8 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -60,11 +60,29 @@ static int test_resource_dpdk(void)
return 0;
 }

+REGISTER_LINKED_RESOURCE(test_resource_c);
+
+static int test_resource_c(void)
+{
+   const struct resource *r;
+
+   r = resource_find("test_resource_c");
+   TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
+   TEST_ASSERT(!strcmp(r->name, "test_resource_c"),
+   "Found resource %s, expected test_resource_c",
+   r->name);
+
+   return 0;
+}
+
 static int test_resource(void)
 {
if (test_resource_dpdk())
return -1;

+   if (test_resource_c())
+   return -1;
+
return 0;
 }

-- 
2.8.0

[dpdk-dev] [PATCH v3 02/11] mk: define objcopy-specific target and arch

2016-05-17 Thread Jan Viktorin

The program objcopy uses non-standard conventions to name the target and arch.
Define the values for supported architectures.

FIXME: tile and ppc_64 are not present.

Signed-off-by: Jan Viktorin 
---
 mk/arch/arm/rte.vars.mk | 5 +
 mk/arch/arm64/rte.vars.mk   | 5 +
 mk/arch/i686/rte.vars.mk| 5 +
 mk/arch/x86_64/rte.vars.mk  | 5 +
 mk/arch/x86_x32/rte.vars.mk | 5 +
 5 files changed, 25 insertions(+)

diff --git a/mk/arch/arm/rte.vars.mk b/mk/arch/arm/rte.vars.mk
index bd85140..2f8cf7c 100644
--- a/mk/arch/arm/rte.vars.mk
+++ b/mk/arch/arm/rte.vars.mk
@@ -37,3 +37,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf

 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-littlearm
+RTE_OBJCOPY_ARCH = arm
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/arm64/rte.vars.mk b/mk/arch/arm64/rte.vars.mk
index 32e3a5f..c168426 100644
--- a/mk/arch/arm64/rte.vars.mk
+++ b/mk/arch/arm64/rte.vars.mk
@@ -56,3 +56,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf

 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-littleaarch64
+RTE_OBJCOPY_ARCH = aarch64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/i686/rte.vars.mk b/mk/arch/i686/rte.vars.mk
index 8ba9a23..6a25312 100644
--- a/mk/arch/i686/rte.vars.mk
+++ b/mk/arch/i686/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?= -melf_i386
 CPU_ASFLAGS ?= -felf

 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-i386
+RTE_OBJCOPY_ARCH = i386
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_64/rte.vars.mk b/mk/arch/x86_64/rte.vars.mk
index b986f04..83723c8 100644
--- a/mk/arch/x86_64/rte.vars.mk
+++ b/mk/arch/x86_64/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf64

 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_x32/rte.vars.mk b/mk/arch/x86_x32/rte.vars.mk
index 3103dfc..676f316 100644
--- a/mk/arch/x86_x32/rte.vars.mk
+++ b/mk/arch/x86_x32/rte.vars.mk
@@ -61,3 +61,8 @@ ifneq ($(shell echo | $(CC) $(CPU_CFLAGS) -E - 2>/dev/null 
1>/dev/null && echo 0
 endif

 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
-- 
2.8.0

[dpdk-dev] [PATCH v3 01/11] app/test: introduce resources for tests

2016-05-17 Thread Jan Viktorin

Certain internal mechanisms of DPDK access different file system structures
(e.g. /sys/bus/pci/devices). It is difficult to test those cases automatically
by a unit test when such path is not hard-coded and there is no simple way how
to distribute fake ones with the current testing environment.

This patch adds a possibility to declare a resource embedded in the test binary
itself. The structure resource cover the generic situation - it provides a name
for lookup and pointers to the embedded data blob. A resource is registered
in a constructor by the macro REGISTER_RESOURCE.

Some initial tests of simple resources is included and added into the group_1.

Signed-off-by: Jan Viktorin 
---
v3:
* fixed doc comments
---
 app/test/Makefile |  2 +
 app/test/autotest_data.py |  6 +++
 app/test/resource.c   | 66 +++
 app/test/resource.h   | 98 +++
 app/test/test_resource.c  | 75 
 5 files changed, 247 insertions(+)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_resource.c

diff --git a/app/test/Makefile b/app/test/Makefile
index a4907d5..7fbdd18 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -43,6 +43,8 @@ APP = test
 #
 SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
+SRCS-y += resource.c
+SRCS-y += test_resource.c
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py
index dde4511..a80a105 100644
--- a/app/test/autotest_data.py
+++ b/app/test/autotest_data.py
@@ -88,6 +88,12 @@ parallel_test_group_list = [
 "Report" : None,
},
{
+"Name" :   "Resource autotest",
+"Command" :"resource_autotest",
+"Func" :   default_autotest,
+"Report" : None,
+   },
+   {
 "Name" :   "Dump log history",
 "Command" :"dump_log_history",
 "Func" :   dump_autotest,
diff --git a/app/test/resource.c b/app/test/resource.c
new file mode 100644
index 000..30513db
--- /dev/null
+++ b/app/test/resource.c
@@ -0,0 +1,66 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ *   notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ *   notice, this list of conditions and the following disclaimer in
+ *   the documentation and/or other materials provided with the
+ *   distribution.
+ * * Neither the name of RehiveTech nor the names of its
+ *   contributors may be used to endorse or promote products derived
+ *   from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+
+#include "resource.h"
+
+struct resource_list resource_list = TAILQ_HEAD_INITIALIZER(resource_list);
+
+size_t resource_size(const struct resource *r)
+{
+   return r->end - r->begin;
+}
+
+const struct resource *resource_find(const char *name)
+{
+   struct resource *r;
+
+   TAILQ_FOREACH(r, _list, next) {
+   RTE_VERIFY(r->name);
+
+   if (!strcmp(r->name, name))
+   return r;
+   }
+
+   return NULL;
+}
+
+void resource_register(struct resource *r)
+{
+   TAILQ_INSERT_TAIL(_list, r, next);
+}
diff --git a/app/test/resource.h b/app/test/resource.h
new file mode 100644
index 000..37c589c
--- /dev/null
+++ b/app/test/resource.h
@@ -0,0 +1,98 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary

[dpdk-dev] [PATCH v3 00/11] Include resources in tests

2016-05-17 Thread Jan Viktorin

Hello,

the third round with fixes based on suggestions by T. Monjalon.

This patch set introduces a mechanism to include a resource (in general a blob)
into the test binary. This allows to make tests less dependent on the target
testing environment. The first use case is testing of PCI bus scan by changing
the hard-coded path (/sys/bus/pci/devices) to something different and provide
a fake tree of devices with the test.

Regards
J. Viktorin

---
v1:
* included 5 patches improving the PCI tests
* fixed using of non-existing RTE_INIT macro

v2:
* Makefile macro resource renamed to linked_resource
* introduced macro linked_tar_resource
* added more comments
* clarified relation between REGISTER_LINKED_RESOURCE and the Makefile
* untar is checked to not loop infinitely
* improved commit messages
* objcopy params extracted to a separated patcha (missing tile and ppc)
* few random bits (usually suggested by T. Monjalon)
* included a note about the new dependency libarchive in the particular commit

v3:
* resource_autotest added to autotest_data.py
* improved test of affecting pci_get_sysfs_path() by setenv
* few other bits...
---
Jan Viktorin (11):
  app/test: introduce resources for tests
  mk: define objcopy-specific target and arch
  app/test: support resources externally linked
  app/test: add functions to create files from resources
  app/test: support resources archived by tar
  app/test: use linked list to store PCI drivers
  app/test: extract test_pci_setup and test_pci_cleanup
  app/test: convert current pci_test into a single test case
  eal/pci: allow to override sysfs
  app/test: scan PCI bus using a fake sysfs
  app/test: do not dump PCI devices in blacklist test

 app/test/Makefile  |  31 +++
 app/test/autotest_data.py  |   6 +
 app/test/resource.c| 296 +
 app/test/resource.h| 135 ++
 app/test/test_pci.c| 148 +--
 .../bus/pci/devices/:01:00.0/class |   1 +
 .../bus/pci/devices/:01:00.0/config| Bin 0 -> 64 bytes
 .../devices/:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/:01:00.0/device|   1 +
 .../bus/pci/devices/:01:00.0/dma_mask_bits |   1 +
 .../bus/pci/devices/:01:00.0/enable|   1 +
 .../bus/pci/devices/:01:00.0/irq   |   1 +
 .../bus/pci/devices/:01:00.0/modalias  |   1 +
 .../bus/pci/devices/:01:00.0/msi_bus   |   1 +
 .../bus/pci/devices/:01:00.0/numa_node |   1 +
 .../bus/pci/devices/:01:00.0/resource  |  13 +
 .../bus/pci/devices/:01:00.0/sriov_numvfs  |   1 +
 .../bus/pci/devices/:01:00.0/sriov_totalvfs|   1 +
 .../bus/pci/devices/:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/:01:00.0/uevent|   6 +
 .../bus/pci/devices/:01:00.0/vendor|   1 +
 app/test/test_resource.c   | 132 +
 drivers/net/szedata2/rte_eth_szedata2.c|   2 +-
 drivers/net/virtio/virtio_pci.c|   2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map  |   7 +
 lib/librte_eal/common/eal_common_pci.c |  13 +
 lib/librte_eal/common/include/rte_pci.h|   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c  |   6 +-
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c  |   7 +-
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c |   2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map|   7 +
 mk/arch/arm/rte.vars.mk|   5 +
 mk/arch/arm64/rte.vars.mk  |   5 +
 mk/arch/i686/rte.vars.mk   |   5 +
 mk/arch/x86_64/rte.vars.mk |   5 +
 mk/arch/x86_x32/rte.vars.mk|   5 +
 37 files changed, 828 insertions(+), 26 deletions(-)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/config
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/device
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/irq
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/msi_bus
 create mode 100644 
app/test/test_pci_sysfs/bus/pci/devices/:01:00.0/numa_node
 create mode 100644

[dpdk-dev] [PATCH v1 03/28] eal/linux: extract function rte_eal_unbind_kernel_driver

2016-05-17 Thread Jan Viktorin

On Fri, 13 May 2016 09:22:23 +0800
Jianbo Liu  wrote:

> On 6 May 2016 at 21:47, Jan Viktorin  wrote:
> > Generalize the PCI-specific pci_unbind_kernel_driver. It is now divided into
> > two parts. First, determination of the path and string identification of the
> > device to be unbound. Second, the actual unbind operation which is generic.
> >
> > Signed-off-by: Jan Viktorin 
> > ---
> >  lib/librte_eal/common/eal_private.h   | 13 +
> >  lib/librte_eal/linuxapp/eal/eal.c | 26 ++
> >  lib/librte_eal/linuxapp/eal/eal_pci.c | 33 
> > +
> >  3 files changed, 48 insertions(+), 24 deletions(-)
> >
> > diff --git a/lib/librte_eal/common/eal_private.h 
> > b/lib/librte_eal/common/eal_private.h
> > index 81816a6..3fb8353 100644
> > --- a/lib/librte_eal/common/eal_private.h
> > +++ b/lib/librte_eal/common/eal_private.h
> > @@ -289,6 +289,19 @@ int rte_eal_alarm_init(void);
> >  int rte_eal_check_module(const char *module_name);
> >
> >  /**
> > + * Unbind kernel driver bound to the device specified by the given devpath,
> > + * and its string identification.
> > + *
> > + * @param devpath  path to the device directory ("/sys/.../devices/")
> > + * @param devididentification of the device ()
> > + *
> > + * @return
> > + *  -1  unbind has failed
> > + *   0  module has been unbound
> > + */
> > +int rte_eal_unbind_kernel_driver(const char *devpath, const char *devid);
> > +
> > +/**
> >   * Get cpu core_id.
> >   *
> >   * This function is private to the EAL.
> > diff --git a/lib/librte_eal/linuxapp/eal/eal.c 
> > b/lib/librte_eal/linuxapp/eal/eal.c
> > index e8fce6b..844f958 100644
> > --- a/lib/librte_eal/linuxapp/eal/eal.c
> > +++ b/lib/librte_eal/linuxapp/eal/eal.c
> > @@ -949,3 +949,29 @@ rte_eal_check_module(const char *module_name)
> > /* Module has been found */
> > return 1;
> >  }
> > +
> > +int
> > +rte_eal_unbind_kernel_driver(const char *devpath, const char *devid)
> > +{
> > +   char filename[PATH_MAX];
> > +   FILE *f;
> > +
> > +   snprintf(filename, sizeof(filename),
> > +"%s/driver/unbind", devpath);
> > +
> > +   f = fopen(filename, "w");
> > +   if (f == NULL) /* device was not bound */
> > +   return 0;
> > +
> > +   if (fwrite(devid, strlen(devid), 1, f) == 0) {
> > +   RTE_LOG(ERR, EAL, "%s(): could not write to %s\n", __func__,
> > +   filename);
> > +   goto error;
> > +   }
> > +
> > +   fclose(f);
> > +   return 0;
> > +error:
> > +   fclose(f);
> > +   return -1;
> > +}
> > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c 
> > b/lib/librte_eal/linuxapp/eal/eal_pci.c
> > index fd7e34f..312cb14 100644
> > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
> > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
> > @@ -59,38 +59,23 @@ int
> >  pci_unbind_kernel_driver(struct rte_pci_device *dev)
> >  {
> > int n;
> > -   FILE *f;
> > -   char filename[PATH_MAX];
> > -   char buf[BUFSIZ];
> > +   char devpath[PATH_MAX];
> > +   char devid[BUFSIZ];
> > struct rte_pci_addr *loc = >addr;
> >
> > -   /* open /sys/bus/pci/devices/:BB:CC.D/driver */
> > -   snprintf(filename, sizeof(filename),
> > -SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/driver/unbind",
> > +   /* devpath /sys/bus/pci/devices/:BB:CC.D */
> > +   snprintf(devpath, sizeof(devpath),
> > +SYSFS_PCI_DEVICES "/" PCI_PRI_FMT,
> >  loc->domain, loc->bus, loc->devid, loc->function);
> >
> > -   f = fopen(filename, "w");
> > -   if (f == NULL) /* device was not bound */
> > -   return 0;
> > -
> > -   n = snprintf(buf, sizeof(buf), PCI_PRI_FMT "\n",
> > +   n = snprintf(devid, sizeof(devid), PCI_PRI_FMT "\n",
> >  loc->domain, loc->bus, loc->devid, loc->function);
> > -   if ((n < 0) || (n >= (int)sizeof(buf))) {
> > +   if ((n < 0) || (n >= (int)sizeof(devid))) {  
> 
> Is it better to move "(n >= (int)sizeof(devid))" before snprintf and
> it has different reason from "n < 0"?

I don't understant this comment. I cannot move the check for _n_ before
the snprintf as it is its return value... Can you provide an example of
your idea?

Do you mean to split the condition to if (n < 0) and else if (n >= ...)?

> 
> > RTE_LOG(ERR, EAL, "%s(): snprintf failed\n", __func__);
> > -   goto error;
> > -   }
> > -   if (fwrite(buf, n, 1, f) == 0) {
> > -   RTE_LOG(ERR, EAL, "%s(): could not write to %s\n", __func__,
> > -   filename);
> > -   goto error;
> > +   return -1;
> > }
> >
> > -   fclose(f);
> > -   return 0;
> > -
> > -error:
> > -   fclose(f);
> > -   return -1;
> > +   return rte_eal_unbind_kernel_driver(devpath, devid);
> >  }
> >
> >  static int
>

[dpdk-dev] [PATCH v1 09/28] eal: introduce --no-soc option

2016-05-17 Thread Jan Viktorin

On Fri, 13 May 2016 11:28:18 +0800
Jianbo Liu  wrote:

> On 6 May 2016 at 21:47, Jan Viktorin  wrote:
> > This option has the same meaning for the SoC infra as the --no-pci
> > for the PCI infra.
> >
> > Signed-off-by: Jan Viktorin 
> > ---
> >  lib/librte_eal/common/eal_common_options.c | 5 +
> >  lib/librte_eal/common/eal_internal_cfg.h   | 1 +
> >  lib/librte_eal/common/eal_options.h| 2 ++
> >  3 files changed, 8 insertions(+)
> >
> > diff --git a/lib/librte_eal/common/eal_common_options.c 
> > b/lib/librte_eal/common/eal_common_options.c
> > index 3efc90f..09d64f7 100644
> > --- a/lib/librte_eal/common/eal_common_options.c
> > +++ b/lib/librte_eal/common/eal_common_options.c
> > @@ -85,6 +85,7 @@ eal_long_options[] = {
> > {OPT_NO_HPET,   0, NULL, OPT_NO_HPET_NUM  },
> > {OPT_NO_HUGE,   0, NULL, OPT_NO_HUGE_NUM  },
> > {OPT_NO_PCI,0, NULL, OPT_NO_PCI_NUM   },
> > +   {OPT_NO_SOC,0, NULL, OPT_NO_SOC_NUM   },
> > {OPT_NO_SHCONF, 0, NULL, OPT_NO_SHCONF_NUM},
> > {OPT_PCI_BLACKLIST, 1, NULL, OPT_PCI_BLACKLIST_NUM},
> > {OPT_PCI_WHITELIST, 1, NULL, OPT_PCI_WHITELIST_NUM},
> > @@ -841,6 +842,10 @@ eal_parse_common_option(int opt, const char *optarg,
> > conf->no_pci = 1;
> > break;
> >
> > +   case OPT_NO_SOC_NUM:
> > +   conf->no_soc = 1;  
> 
> Could it be better to rename to enable_soc, and disable soc by default?

Sure, I tried to be consistent with PCI...

[...]


-- 
   Jan Viktorin  E-mail: Viktorin at RehiveTech.com
   System Architect  Web:www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

[dpdk-dev] What is the status of mempool manager?

2016-05-17 Thread Jan Viktorin

Hello Olivier...

On Tue, 17 May 2016 15:56:32 +0200
Olivier MATZ  wrote:

> Hi Jan,
> 
> On 05/17/2016 02:34 PM, Jan Viktorin wrote:
> > Hello,
> >
> > I was trying to find out the status of the mempool rework and found
> > this 36-pieces long patch set:
> >
> > http://dpdk.org/ml/archives/dev/2016-April/037464.html
> >
> > (I failed to apply it to 16.04, 2.1.0, current HEAD.)  
> 
> It should be integrated soon, as there was no comment during
> some time. I'll send a new version that applies on head
> + minor cosmetic changes.

Cool ;).

> 
> > and some bits around:
> >
> > http://dpdk.org/ml/archives/dev/2016-May/038390.html  
> 
> I need to review this one, it's on my todo list.
> 
> > http://dpdk.org/ml/archives/dev/2016-April/037509.html  
> 
> I think the code is ready to be integrated, but I was expecting
> some reactions on the v4, especially some precisions about the
> use cases and the needs. See in the link "Things that should
> still be discussed".
> 
> > http://dpdk.org/ml/archives/dev/2016-April/037457.html  
> 
> It is integrated since today ;)
> 
> > http://dpdk.org/ml/archives/dev/2016-April/036979.html  
> 
> Some comments have been done by Konstantin and myself, I
> think a new version will be submitted. It will be integrated
> in 16.07.

Thanks for the summary.

> 
> > ...but I am confused.
> >
> > I am trying to find out how to write a custom memory pool based on
> > the uio_dmem_genirq driver. It provides DMA memory via the UIO API
> > (maps).
> >
> > If I have a PMD running of top of this UIO, I need to implement a
> > custom allocator that gets memory from the dev->mem_resource (with
> > the mappings preloaded from the UIO by EAL).
> >
> > This is related to the SoC infra as given here:
> >
> > http://dpdk.org/ml/archives/dev/2016-May/038486.html
> >
> > Any idea how to start with this?  
> 
> I think your feedback would be very valuable.

OK, CC me.

> 
> The "external mempool manager" series allows to register new
> handler that replaces the internal ring storing the objects. It
> does not change the way the memory is allocated.
> 
> The big patchset introduce new way of allocating memory used by
> memory pool (no need to be virtually contiguous anymore). You
> could throw an eye on rte_mempool_populate*() functions that
> populate a mempool with memory for storing the objects.

Thanks for this hint. I've already noticed this function...

> 
> I'm not sure I'm getting your exact needs.
> Do you need to populate a ring-based mempool with specific memory,
> retrieved from UIO API?

Probably yes.

> Do you need to replace the default internal ring by something else?

I don't think so.

> Do you need to access the mempool from the hardware?

No.

Thanks
Jan

> 
> Regards,
> Olivier



-- 
   Jan Viktorin  E-mail: Viktorin at RehiveTech.com
   System Architect  Web:www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

[dpdk-dev] [PATCH v4] eal: make hugetlb initialization more robust

2016-05-17 Thread Thomas Monjalon

2016-05-12 00:44, Jianfeng Tan:
> This patch adds an option, --huge-trybest, to use a recover mechanism to
> the case that there are not so many hugepages (declared in sysfs), which
> can be used. It relys on a mem access to fault-in hugepages, and if fails

relys -> relies

> with SIGBUS, recover to previously saved stack environment with
> siglongjmp().
> 
> Besides, this solution fixes an issue when hugetlbfs is specified with an
> option of size. Currently DPDK does not respect the quota of a hugetblfs
> mount. It fails to init the EAL because it tries to map the number of free
> hugepages in the system rather than using the number specified in the quota
> for that mount.

It looks to be a bug. Why adding an option?
What is the benefit of the old behaviour, not using --try-best?

> +static sigjmp_buf jmpenv;
> +
> +static void sigbus_handler(int signo __rte_unused)
> +{
> + siglongjmp(jmpenv, 1);
> +}
> +
> +/* Put setjmp into a wrap method to avoid compiling error. Any non-volatile,
> + * non-static local variable in the stack frame calling sigsetjmp might be
> + * clobbered by a call to longjmp.
> + */
> +static int wrap_sigsetjmp(void)
> +{
> + return sigsetjmp(jmpenv, 1);
> +}

Please add the word "huge" to these variables and functions.

> +static struct sigaction action_old;
> +static int need_recover;
> +
> +static void
> +register_sigbus(void)
> +{
> + sigset_t mask;
> + struct sigaction action;
> +
> + sigemptyset();
> + sigaddset(, SIGBUS);
> + action.sa_flags = 0;
> + action.sa_mask = mask;
> + action.sa_handler = sigbus_handler;
> +
> + need_recover = !sigaction(SIGBUS, , _old);
> +}
> +
> +static void
> +recover_sigbus(void)
> +{
> + if (need_recover) {
> + sigaction(SIGBUS, _old, NULL);
> + need_recover = 0;
> + }
> +}

Idem, Please add the word "huge".

[dpdk-dev] [PATCH v4] eal: make hugetlb initialization more robust

2016-05-17 Thread David Marchand

Hello Jianfeng,

On Thu, May 12, 2016 at 2:44 AM, Jianfeng Tan  wrote:
> This patch adds an option, --huge-trybest, to use a recover mechanism to
> the case that there are not so many hugepages (declared in sysfs), which
> can be used. It relys on a mem access to fault-in hugepages, and if fails
> with SIGBUS, recover to previously saved stack environment with
> siglongjmp().
>
> Besides, this solution fixes an issue when hugetlbfs is specified with an
> option of size. Currently DPDK does not respect the quota of a hugetblfs
> mount. It fails to init the EAL because it tries to map the number of free
> hugepages in the system rather than using the number specified in the quota
> for that mount.
>
> It's still an open issue with CONFIG_RTE_EAL_SINGLE_FILE_SEGMENTS. Under
> this case (such as IVSHMEM target), having hugetlbfs mounts with quota will
> fail to remap hugepages as it relies on having mapped all free hugepages
> in the system.

For such a case case, maybe having some warning log message when it
fails would help the user.
+ a known issue in the release notes ?


> diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c 
> b/lib/librte_eal/linuxapp/eal/eal_memory.c
> index 5b9132c..8c77010 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_memory.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
> @@ -417,12 +434,33 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl,
> hugepg_tbl[i].final_va = virtaddr;
> }
>
> +   if (orig && internal_config.huge_trybest) {
> +   /* In linux, hugetlb limitations, like cgroup, are
> +* enforced at fault time instead of mmap(), even
> +* with the option of MAP_POPULATE. Kernel will send
> +* a SIGBUS signal. To avoid to be killed, save stack
> +* environment here, if SIGBUS happens, we can jump
> +* back here.
> +*/
> +   if (wrap_sigsetjmp()) {
> +   RTE_LOG(DEBUG, EAL, "SIGBUS: Cannot mmap more 
> "
> +   "hugepages of size %u MB\n",
> +   (unsigned)(hugepage_sz / 0x10));
> +   munmap(virtaddr, hugepage_sz);
> +   close(fd);
> +   unlink(hugepg_tbl[i].filepath);
> +   return i;
> +   }
> +   *(int *)virtaddr = 0;
> +   }
> +
> +
> /* set shared flock on the file. */
> if (flock(fd, LOCK_SH | LOCK_NB) == -1) {
> -   RTE_LOG(ERR, EAL, "%s(): Locking file failed:%s \n",
> +   RTE_LOG(DEBUG, EAL, "%s(): Locking file failed:%s \n",
> __func__, strerror(errno));
> close(fd);
> -   return -1;
> +   return i;
> }
>
> close(fd);

Maybe I missed something, but we are writing into some hugepage before
the flock has been called.
Are we sure there is nobody else using this hugepage ?

Especially, can't this cause trouble to a primary process running if
we start the exact same primary process ?


-- 
David Marchand

[dpdk-dev] [PATCH] eal: fix log level/type retrieving on a standard pthread

2016-05-17 Thread David Marchand

On Mon, May 9, 2016 at 6:13 PM, Olivier Matz  wrote:
> From: Maxime Leroy 
>
> The functions rte_log_cur_msg_loglevel() and rte_log_cur_msg_logtype()
> return the current log level/type for the message being processed. They
> are used when implementing a user-defined logging stream.
>
> The current log levels and types were stored in a table indexed by the
> lcore_id, only returning a valid value for dataplane threads. Setting
> and getting these values in a non dataplane thread was ignored, using
> the global value instead.
>
> To fix this issue, a per-thread variable could be used (with
> RTE_DEFINE_PER_LCORE), allowing any pthread to set and retrieve its
> current log level or type.
>
> Signed-off-by: Maxime Leroy 
> Signed-off-by: Olivier Matz 

Acked-by: David Marchand 

-- 
David Marchand

[dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

2016-05-17 Thread David Marchand

Hello Olivier,

On Tue, May 17, 2016 at 11:59 AM, Olivier Matz  
wrote:
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c 
> b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> index ac449c5..077ad96 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> @@ -411,81 +412,153 @@ pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
> RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%lx\n", start);
>
> p->base = start;
> +   p->len = 0;
> return 0;
> +}
>  #else
> -   RTE_SET_USED(dev);
> -   RTE_SET_USED(bar);
> -   RTE_SET_USED(p);
> +int
> +pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
> +  struct rte_pci_ioport *p)
> +{
> +   FILE *f;
> +   char buf[BUFSIZ];
> +   char filename[PATH_MAX];
> +   uint64_t phys_addr, end_addr, flags;
> +   int fd, i;
> +   void *addr;
> +
> +   /* open and read addresses of the corresponding resource in sysfs */
> +   snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource",
> +   SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
> +dev->addr.devid, dev->addr.function);
> +   f = fopen(filename, "r");
> +   if (f == NULL) {
> +   RTE_LOG(ERR, EAL, "Cannot open sysfs resource: %s\n",
> +   strerror(errno));
> +   return -1;
> +   }
> +   for (i = 0; i < bar + 1; i++) {
> +   if (fgets(buf, sizeof(buf), f) == NULL) {
> +   RTE_LOG(ERR, EAL, "Cannot read sysfs resource\n");
> +   goto error;
> +   }
> +   }
> +   if (pci_parse_one_sysfs_resource(buf, sizeof(buf), _addr,
> +   _addr, ) < 0)
> +   goto error;
> +   if ((flags & IORESOURCE_IO) == 0) {
> +   RTE_LOG(ERR, EAL, "BAR %d is not an IO resource\n", bar);
> +   goto error;
> +   }
> +   snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource%d",
> +   SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
> +dev->addr.devid, dev->addr.function, bar);
> +
> +   /* mmap the pci resource */
> +   fd = open(filename, O_RDWR);
> +   if (fd < 0) {
> +   RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", filename,
> +   strerror(errno));
> +   goto error;
> +   }
> +   addr = mmap(NULL, end_addr + 1, PROT_READ | PROT_WRITE,
> +   MAP_SHARED, fd, 0);

Sorry, did not catch it in v1, but a close(fd) is missing here.
With this, I think the patchset looks good.

Just missing some opinion from the virtio maintainers ?


> +   if (addr == MAP_FAILED) {
> +   RTE_LOG(ERR, EAL, "Cannot mmap IO port resource: %s\n",
> +   strerror(errno));
> +   goto error;
> +   }
> +
> +   /* strangely, the base address is mmap addr + phys_addr */
> +   p->base = (uintptr_t)addr + phys_addr;
> +   p->len = end_addr + 1;
> +   RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%"PRIx64"\n", p->base);
> +   fclose(f);
> +
> +   return 0;
> +
> +error:
> +   fclose(f);
> return -1;
> -#endif
>  }
> +#endif


-- 
David Marchand

[dpdk-dev] [PATCH v3 8/8] doc: update doc for packet capture framework

2016-05-17 Thread Reshma Pattan

Added programmers guide for librte_pdump.
Added sample application guide for app/pdump application.
Updated release note for packet capture framework changes.

Signed-off-by: Reshma Pattan 
---
 MAINTAINERS |   3 +
 doc/guides/prog_guide/index.rst |   1 +
 doc/guides/prog_guide/pdump_library.rst | 121 
 doc/guides/rel_notes/release_16_07.rst  |   6 ++
 doc/guides/sample_app_ug/index.rst  |   1 +
 doc/guides/sample_app_ug/pdump.rst  | 109 
 6 files changed, 241 insertions(+)
 create mode 100644 doc/guides/prog_guide/pdump_library.rst
 create mode 100644 doc/guides/sample_app_ug/pdump.rst

diff --git a/MAINTAINERS b/MAINTAINERS
index 58f5ba4..d4d0630 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -439,6 +439,9 @@ Pdump
 M: Reshma Pattan 
 F: lib/librte_pdump/
 F: app/pdump/
+F: doc/guides/prog_guide/pdump_library.rst
+F: doc/guides/sample_app_ug/pdump.rst
+

 Hierarchical scheduler
 M: Cristian Dumitrescu 
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index b862d0c..4caf969 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -71,6 +71,7 @@ Programmer's Guide
 writing_efficient_code
 profile_app
 glossary
+pdump_library


 **Figures**
diff --git a/doc/guides/prog_guide/pdump_library.rst 
b/doc/guides/prog_guide/pdump_library.rst
new file mode 100644
index 000..6af77b9
--- /dev/null
+++ b/doc/guides/prog_guide/pdump_library.rst
@@ -0,0 +1,121 @@
+..  BSD LICENSE
+Copyright(c) 2016 Intel Corporation. All rights reserved.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+* Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above copyright
+notice, this list of conditions and the following disclaimer in
+the documentation and/or other materials provided with the
+distribution.
+* Neither the name of Intel Corporation nor the names of its
+contributors may be used to endorse or promote products derived
+from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+.. _Pdump_Library:
+
+pdump Library
+=
+
+Pdump library provides framework for packet capturing on DPDK.
+
+Operation
+-
+
+Pdump library provides APIs to support packet capturing on dpdk ethernet 
devices.
+Library provides APIs to initialize the packet capture framework, 
enable/disable
+the packet capture and un initialize the packet capture framework.
+
+Pdump library works on server and client based model.
+
+Sever is responsible for enabling/disabling the packet captures.
+Clients are responsible for requesting enable/disable of the
+packet captures.
+
+As part of packet capture framework initialization, pthread and
+the server socket is created. Only one server socket is allowed on the system.
+As part of enabling/disabling the packet capture, client sockets are created
+and multiple client sockets are allowed.
+Who ever calls initialization first they will succeed with the initialization,
+next subsequent calls of initialization are not allowed. So next users can only
+request enabling/disabling the packet capture.
+
+Library provides below APIs
+
+``rte_pdump_init()``
+This API initializes the packet capture framework.
+
+``rte_pdump_enable()``
+This API enables the packet capturing on a given port and queue.
+Note: filter option in the API is place holder for future use.
+
+``rte_pdump_enable_by_deviceid()``
+This API enables the packet capturing on a given device id
+(device name or pci address) and queue.
+Note: filter option in the API is place holder for future use.
+
+``rte_pdump_disable()``
+This API disables the packet capturing on a given port and queue.
+
+``rte_pdump_disable_by_deviceid()``
+This API disables the packet capturing on a given device_id and queue.
+
+``rte_pdump_uninit()``
+This API un initializes the packet

[dpdk-dev] [PATCH v3 7/8] app/test-pmd: add pdump initialization uninitialization

2016-05-17 Thread Reshma Pattan

Call rte_pdump_init and rte_pdump_uninit for packet
capturing initialization and uninitialization.

Signed-off-by: Reshma Pattan 
---
 app/test-pmd/testpmd.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 26a174c..e131363 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -75,6 +75,7 @@
 #ifdef RTE_LIBRTE_PMD_XENVIRT
 #include 
 #endif
+#include 

 #include "testpmd.h"
 #include "mempool_osdep.h"
@@ -2018,6 +2019,8 @@ signal_handler(int signum)
if (signum == SIGINT || signum == SIGTERM) {
printf("\nSignal %d received, preparing to exit...\n",
signum);
+   /* uninitialize packet capture framework */
+   rte_pdump_uninit();
force_quit();
/* exit with the expected status */
signal(signum, SIG_DFL);
@@ -2038,6 +2041,9 @@ main(int argc, char** argv)
if (diag < 0)
rte_panic("Cannot init EAL\n");

+   /* initialize packet capture framework */
+   rte_pdump_init();
+
nb_ports = (portid_t) rte_eth_dev_count();
if (nb_ports == 0)
RTE_LOG(WARNING, EAL, "No probed ethernet devices\n");
-- 
2.5.0

[dpdk-dev] [PATCH v3 6/8] app/pdump: add pdump tool for packet capturing

2016-05-17 Thread Reshma Pattan

New tool added for packet capturing on dpdk.
This tool supports command line options.
This tool runs as secondary process by default.

Command line supports various parameters to capture
the packets.

User should pass on a)port and queue (or) b)pci address
and queue (or) c)device name and queue to capture
the packets.

Users also need to pass on either pcap file name or
any linux iface, on to which packets captured from dpdk
ports will be sent on for the users to view using tcpdump.

Users have option to capture packets either a) in RX
direction, b)(or) in TX direction c)(or) from both the
directions.

User can pass on ring_size and mempool parameters using
command line, but these are optional parameters.
These are used to create ring and mempool objects for packet
mirroring from primary application to tool. If user doesn't
provide any values, default values will be used internally
for the creation of the ring and mempool.

Signed-off-by: Reshma Pattan 
---
 MAINTAINERS|   1 +
 app/Makefile   |   1 +
 app/pdump/Makefile |  45 +++
 app/pdump/main.c   | 935 +
 4 files changed, 982 insertions(+)
 create mode 100644 app/pdump/Makefile
 create mode 100644 app/pdump/main.c

diff --git a/MAINTAINERS b/MAINTAINERS
index a716d85..58f5ba4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -438,6 +438,7 @@ F: doc/guides/sample_app_ug/packet_ordering.rst
 Pdump
 M: Reshma Pattan 
 F: lib/librte_pdump/
+F: app/pdump/

 Hierarchical scheduler
 M: Cristian Dumitrescu 
diff --git a/app/Makefile b/app/Makefile
index 1151e09..c593efa 100644
--- a/app/Makefile
+++ b/app/Makefile
@@ -37,5 +37,6 @@ DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += test-pipeline
 DIRS-$(CONFIG_RTE_TEST_PMD) += test-pmd
 DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += cmdline_test
 DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += proc_info
+DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += pdump

 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/app/pdump/Makefile b/app/pdump/Makefile
new file mode 100644
index 000..96bb4af
--- /dev/null
+++ b/app/pdump/Makefile
@@ -0,0 +1,45 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2016 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+# * Redistributions of source code must retain the above copyright
+#   notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#   notice, this list of conditions and the following disclaimer in
+#   the documentation and/or other materials provided with the
+#   distribution.
+# * Neither the name of Intel Corporation nor the names of its
+#   contributors may be used to endorse or promote products derived
+#   from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+APP = dpdk_pdump
+
+CFLAGS += $(WERROR_FLAGS)
+
+# all source are stored in SRCS-y
+
+SRCS-y := main.c
+
+# this application needs libraries first
+DEPDIRS-y += lib
+
+include $(RTE_SDK)/mk/rte.app.mk
diff --git a/app/pdump/main.c b/app/pdump/main.c
new file mode 100644
index 000..cc996c6
--- /dev/null
+++ b/app/pdump/main.c
@@ -0,0 +1,935 @@
+/*
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 Intel Corporation. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ *   notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ *   notice, this list of conditions and the following disclaimer in
+ *   the documentation and/or other materials provided with the
+ *   distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ *   contributors may be used to endorse or promote products derived
+ *   from

[dpdk-dev] [PATCH v3 5/8] lib/librte_pdump: add new library for packet capturing support

2016-05-17 Thread Reshma Pattan

Added new library for packet capturing support.

Added public api rte_pdump_init, applications should call
this as part of their application setup to have packet
capturing framework ready.

Added public api rte_pdump_uninit to un initialize the packet
capturing framework.

Added public apis rte_pdump_enable and rte_pdump_disable to
enable and disable packet capturing on specific port and queue.

Added public apis rte_pdump_enable_by_deviceid and
rte_pdump_disable_by_deviceid to enable and disable packet
capturing on a specific device (pci address or name) and queue.

Signed-off-by: Reshma Pattan 
---
 MAINTAINERS|   4 +
 config/common_base |   5 +
 lib/Makefile   |   1 +
 lib/librte_pdump/Makefile  |  55 +++
 lib/librte_pdump/rte_pdump.c   | 816 +
 lib/librte_pdump/rte_pdump.h   | 186 
 lib/librte_pdump/rte_pdump_version.map |  12 +
 mk/rte.app.mk  |   1 +
 8 files changed, 1080 insertions(+)
 create mode 100644 lib/librte_pdump/Makefile
 create mode 100644 lib/librte_pdump/rte_pdump.c
 create mode 100644 lib/librte_pdump/rte_pdump.h
 create mode 100644 lib/librte_pdump/rte_pdump_version.map

diff --git a/MAINTAINERS b/MAINTAINERS
index ba4053a..a716d85 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -435,6 +435,10 @@ F: app/test/test_reorder*
 F: examples/packet_ordering/
 F: doc/guides/sample_app_ug/packet_ordering.rst

+Pdump
+M: Reshma Pattan 
+F: lib/librte_pdump/
+
 Hierarchical scheduler
 M: Cristian Dumitrescu 
 F: lib/librte_sched/
diff --git a/config/common_base b/config/common_base
index 3535c6e..259bf0a 100644
--- a/config/common_base
+++ b/config/common_base
@@ -484,6 +484,11 @@ CONFIG_RTE_LIBRTE_DISTRIBUTOR=y
 CONFIG_RTE_LIBRTE_REORDER=y

 #
+# Compile the pdump library
+#
+CONFIG_RTE_LIBRTE_PDUMP=y
+
+#
 # Compile librte_port
 #
 CONFIG_RTE_LIBRTE_PORT=y
diff --git a/lib/Makefile b/lib/Makefile
index f254dba..ca7c02f 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -57,6 +57,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_PORT) += librte_port
 DIRS-$(CONFIG_RTE_LIBRTE_TABLE) += librte_table
 DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += librte_pipeline
 DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += librte_reorder
+DIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += librte_pdump

 ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
 DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
diff --git a/lib/librte_pdump/Makefile b/lib/librte_pdump/Makefile
new file mode 100644
index 000..af81a28
--- /dev/null
+++ b/lib/librte_pdump/Makefile
@@ -0,0 +1,55 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2016 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+# * Redistributions of source code must retain the above copyright
+#   notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#   notice, this list of conditions and the following disclaimer in
+#   the documentation and/or other materials provided with the
+#   distribution.
+# * Neither the name of Intel Corporation nor the names of its
+#   contributors may be used to endorse or promote products derived
+#   from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pdump.a
+
+CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+CFLAGS += -D_GNU_SOURCE
+
+EXPORT_MAP := rte_pdump_version.map
+
+LIBABIVER := 1
+
+# all source are stored in SRCS-y
+SRCS-$(CONFIG_RTE_LIBRTE_PDUMP) := rte_pdump.c
+
+# install this header file
+SYMLINK-$(CONFIG_RTE_LIBRTE_PDUMP)-include := rte_pdump.h
+
+# this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += lib/librte_mbuf
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += lib/librte_eal
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += lib/librte_ether
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_pdump/rte_pdump.c b/lib/librte_pdump/rte_pdump.c
new file mode 100644
index

[dpdk-dev] [PATCH v3 4/8] librte_ether: make rte_eth_dev_get_port_by_name api public

2016-05-17 Thread Reshma Pattan

Converted rte_eth_dev_get_port_by_name to public API.

Signed-off-by: Reshma Pattan 
---
 lib/librte_ether/rte_ethdev.c  |  2 +-
 lib/librte_ether/rte_ethdev.h  | 15 +++
 lib/librte_ether/rte_ether_version.map |  1 +
 3 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 3ee5b9f..a50bb1e 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -427,7 +427,7 @@ rte_eth_dev_get_name_by_port(uint8_t port_id, char *name)
return 0;
 }

-static int
+int
 rte_eth_dev_get_port_by_name(const char *name, uint8_t *port_id)
 {
int i;
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 106318f..b20f5cd 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -4283,6 +4283,21 @@ rte_eth_dev_l2_tunnel_offload_set(uint8_t port_id,
  uint32_t mask,
  uint8_t en);

+/**
+* Get the port id from pci adrress or device name
+* Ex: :2:00.0 or vdev name eth_pcap0
+*
+* @param name
+*  pci address or name of the device
+* @param port_id
+*   pointer to port identifier of the device
+* @return
+*   - (0) if successful.
+*   - (-ENODEV) on failure.
+*/
+int
+rte_eth_dev_get_port_by_name(const char *name, uint8_t *port_id);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/librte_ether/rte_ether_version.map 
b/lib/librte_ether/rte_ether_version.map
index d06d648..512f38f 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -137,5 +137,6 @@ DPDK_16.07 {
global:

rte_eth_add_first_rx_callback;
+   rte_eth_dev_get_port_by_name;
rte_eth_dev_info_get;
 } DPDK_16.04;
-- 
2.5.0

[dpdk-dev] [PATCH v3 3/8] librte_ether: add new fields to rte_eth_dev_info struct

2016-05-17 Thread Reshma Pattan

Add new fields to rte_eth_dev_info struct
New fields nb_rx_queues and nb_tx_queues are added to
rte_eth_dev_info structure.
Changes to API rte_eth_dev_info_get() are done to update
these new fields to rte_eth_dev_info object.

Signed-off-by: Reshma Pattan 
---
 lib/librte_ether/rte_ethdev.c  | 2 ++
 lib/librte_ether/rte_ethdev.h  | 3 +++
 lib/librte_ether/rte_ether_version.map | 1 +
 3 files changed, 6 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 6331759..3ee5b9f 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -1666,6 +1666,8 @@ rte_eth_dev_info_get(uint8_t port_id, struct 
rte_eth_dev_info *dev_info)
(*dev->dev_ops->dev_infos_get)(dev, dev_info);
dev_info->pci_dev = dev->pci_dev;
dev_info->driver_name = dev->data->drv_name;
+   dev_info->nb_rx_queues = dev->data->nb_rx_queues;
+   dev_info->nb_tx_queues = dev->data->nb_tx_queues;
 }

 int
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 92b07a9..106318f 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -882,6 +882,9 @@ struct rte_eth_dev_info {
struct rte_eth_desc_lim rx_desc_lim;  /**< RX descriptors limits */
struct rte_eth_desc_lim tx_desc_lim;  /**< TX descriptors limits */
uint32_t speed_capa;  /**< Supported speeds bitmap (ETH_LINK_SPEED_). */
+   /** Configured number of rx/tx queues */
+   uint16_t nb_rx_queues; /**< Number of RX queues. */
+   uint16_t nb_tx_queues; /**< Number of TX queues. */
 };

 /**
diff --git a/lib/librte_ether/rte_ether_version.map 
b/lib/librte_ether/rte_ether_version.map
index c990b04..d06d648 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -137,4 +137,5 @@ DPDK_16.07 {
global:

rte_eth_add_first_rx_callback;
+   rte_eth_dev_info_get;
 } DPDK_16.04;
-- 
2.5.0

[dpdk-dev] [PATCH v3 2/8] librte_ether: add new api rte_eth_add_first_rx_callback

2016-05-17 Thread Reshma Pattan

Added new public api rte_eth_add_first_rx_callback to add given
callback as head of list.

Signed-off-by: Reshma Pattan 
---
 lib/librte_ether/rte_ethdev.c  | 35 ++
 lib/librte_ether/rte_ethdev.h  | 27 ++
 lib/librte_ether/rte_ether_version.map |  6 ++
 3 files changed, 68 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index cf25d8a..6331759 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -2959,6 +2959,41 @@ rte_eth_add_rx_callback(uint8_t port_id, uint16_t 
queue_id,
 }

 void *
+rte_eth_add_first_rx_callback(uint8_t port_id, uint16_t queue_id,
+   rte_rx_callback_fn fn, void *user_param)
+{
+#ifndef RTE_ETHDEV_RXTX_CALLBACKS
+   rte_errno = ENOTSUP;
+   return NULL;
+#endif
+   /* check input parameters */
+   if (!rte_eth_dev_is_valid_port(port_id) || fn == NULL ||
+   queue_id >= 
rte_eth_devices[port_id].data->nb_rx_queues) {
+   rte_errno = EINVAL;
+   return NULL;
+   }
+
+   struct rte_eth_rxtx_callback *cb = rte_zmalloc(NULL, sizeof(*cb), 0);
+
+   if (cb == NULL) {
+   rte_errno = ENOMEM;
+   return NULL;
+   }
+
+   cb->fn.rx = fn;
+   cb->param = user_param;
+
+   rte_spinlock_lock(_eth_rx_cb_lock);
+   /* Add the callbacks at fisrt position*/
+   cb->next = rte_eth_devices[port_id].post_rx_burst_cbs[queue_id];
+   rte_smp_wmb();
+   rte_eth_devices[port_id].post_rx_burst_cbs[queue_id] = cb;
+   rte_spinlock_unlock(_eth_rx_cb_lock);
+
+   return cb;
+}
+
+void *
 rte_eth_add_tx_callback(uint8_t port_id, uint16_t queue_id,
rte_tx_callback_fn fn, void *user_param)
 {
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 2757510..92b07a9 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -3825,6 +3825,33 @@ int rte_eth_dev_get_dcb_info(uint8_t port_id,
 void *rte_eth_add_rx_callback(uint8_t port_id, uint16_t queue_id,
rte_rx_callback_fn fn, void *user_param);

+/*
+* Add a callback that must be called first on packet RX on a given port and 
queue.
+*
+* This API configures a first function to be called for each burst of
+* packets received on a given NIC port queue. The return value is a pointer
+* that can be used to later remove the callback using
+* rte_eth_remove_rx_callback().
+*
+* Multiple functions are called in the order that they are added.
+*
+* @param port_id
+*   The port identifier of the Ethernet device.
+* @param queue_id
+*   The queue on the Ethernet device on which the callback is to be added.
+* @param fn
+*   The callback function
+* @param user_param
+*   A generic pointer parameter which will be passed to each invocation of the
+*   callback function on this port and queue.
+*
+* @return
+*   NULL on error.
+*   On success, a pointer value which can later be used to remove the callback.
+*/
+void *rte_eth_add_first_rx_callback(uint8_t port_id, uint16_t queue_id,
+   rte_rx_callback_fn fn, void *user_param);
+
 /**
  * Add a callback to be called on packet TX on a given port and queue.
  *
diff --git a/lib/librte_ether/rte_ether_version.map 
b/lib/librte_ether/rte_ether_version.map
index 214ecc7..c990b04 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -132,3 +132,9 @@ DPDK_16.04 {
rte_eth_tx_buffer_set_err_callback;

 } DPDK_2.2;
+
+DPDK_16.07 {
+   global:
+
+   rte_eth_add_first_rx_callback;
+} DPDK_16.04;
-- 
2.5.0

[dpdk-dev] [PATCH v3 1/8] librte_ether: protect add/remove of rxtx callbacks with spinlocks

2016-05-17 Thread Reshma Pattan

Added spinlocks around add/remove logic of rxtx callbacks to
avoid corruption of callback lists in multithreaded context.

Signed-off-by: Reshma Pattan 
---
 lib/librte_ether/rte_ethdev.c | 82 +--
 1 file changed, 40 insertions(+), 42 deletions(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index a31018e..cf25d8a 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -77,6 +77,12 @@ static uint8_t nb_ports;
 /* spinlock for eth device callbacks */
 static rte_spinlock_t rte_eth_dev_cb_lock = RTE_SPINLOCK_INITIALIZER;

+/* spinlock for add/remove rx callbacks */
+static rte_spinlock_t rte_eth_rx_cb_lock = RTE_SPINLOCK_INITIALIZER;
+
+/* spinlock for add/remove tx callbacks */
+static rte_spinlock_t rte_eth_tx_cb_lock = RTE_SPINLOCK_INITIALIZER;
+
 /* store statistics names and its offset in stats structure  */
 struct rte_eth_xstats_name_off {
char name[RTE_ETH_XSTATS_NAME_SIZE];
@@ -1639,7 +1645,6 @@ rte_eth_dev_set_rx_queue_stats_mapping(uint8_t port_id, 
uint16_t rx_queue_id,
STAT_QMAP_RX);
 }

-
 void
 rte_eth_dev_info_get(uint8_t port_id, struct rte_eth_dev_info *dev_info)
 {
@@ -2925,7 +2930,6 @@ rte_eth_add_rx_callback(uint8_t port_id, uint16_t 
queue_id,
rte_errno = EINVAL;
return NULL;
}
-
struct rte_eth_rxtx_callback *cb = rte_zmalloc(NULL, sizeof(*cb), 0);

if (cb == NULL) {
@@ -2936,6 +2940,7 @@ rte_eth_add_rx_callback(uint8_t port_id, uint16_t 
queue_id,
cb->fn.rx = fn;
cb->param = user_param;

+   rte_spinlock_lock(_eth_rx_cb_lock);
/* Add the callbacks in fifo order. */
struct rte_eth_rxtx_callback *tail =
rte_eth_devices[port_id].post_rx_burst_cbs[queue_id];
@@ -2948,6 +2953,7 @@ rte_eth_add_rx_callback(uint8_t port_id, uint16_t 
queue_id,
tail = tail->next;
tail->next = cb;
}
+   rte_spinlock_unlock(_eth_rx_cb_lock);

return cb;
 }
@@ -2977,6 +2983,7 @@ rte_eth_add_tx_callback(uint8_t port_id, uint16_t 
queue_id,
cb->fn.tx = fn;
cb->param = user_param;

+   rte_spinlock_lock(_eth_tx_cb_lock);
/* Add the callbacks in fifo order. */
struct rte_eth_rxtx_callback *tail =
rte_eth_devices[port_id].pre_tx_burst_cbs[queue_id];
@@ -2989,6 +2996,7 @@ rte_eth_add_tx_callback(uint8_t port_id, uint16_t 
queue_id,
tail = tail->next;
tail->next = cb;
}
+   rte_spinlock_unlock(_eth_tx_cb_lock);

return cb;
 }
@@ -3007,29 +3015,24 @@ rte_eth_remove_rx_callback(uint8_t port_id, uint16_t 
queue_id,
}

struct rte_eth_dev *dev = _eth_devices[port_id];
-   struct rte_eth_rxtx_callback *cb = dev->post_rx_burst_cbs[queue_id];
-   struct rte_eth_rxtx_callback *prev_cb;
-
-   /* Reset head pointer and remove user cb if first in the list. */
-   if (cb == user_cb) {
-   dev->post_rx_burst_cbs[queue_id] = user_cb->next;
-   return 0;
-   }
-
-   /* Remove the user cb from the callback list. */
-   do {
-   prev_cb = cb;
-   cb = cb->next;
-
+   struct rte_eth_rxtx_callback *cb;
+   struct rte_eth_rxtx_callback **prev_cb;
+   int ret = -EINVAL;
+
+   rte_spinlock_lock(_eth_rx_cb_lock);
+   prev_cb = >post_rx_burst_cbs[queue_id];
+   for (; *prev_cb != NULL; prev_cb = >next) {
+   cb = *prev_cb;
if (cb == user_cb) {
-   prev_cb->next = user_cb->next;
-   return 0;
+   /* Remove the user cb from the callback list. */
+   *prev_cb = cb->next;
+   ret = 0;
+   break;
}
+   }
+   rte_spinlock_unlock(_eth_rx_cb_lock);

-   } while (cb != NULL);
-
-   /* Callback wasn't found. */
-   return -EINVAL;
+   return ret;
 }

 int
@@ -3046,29 +3049,24 @@ rte_eth_remove_tx_callback(uint8_t port_id, uint16_t 
queue_id,
}

struct rte_eth_dev *dev = _eth_devices[port_id];
-   struct rte_eth_rxtx_callback *cb = dev->pre_tx_burst_cbs[queue_id];
-   struct rte_eth_rxtx_callback *prev_cb;
-
-   /* Reset head pointer and remove user cb if first in the list. */
-   if (cb == user_cb) {
-   dev->pre_tx_burst_cbs[queue_id] = user_cb->next;
-   return 0;
-   }
-
-   /* Remove the user cb from the callback list. */
-   do {
-   prev_cb = cb;
-   cb = cb->next;
-
+   int ret = -EINVAL;
+   struct rte_eth_rxtx_callback *cb;
+   struct rte_eth_rxtx_callback **prev_cb;
+
+   rte_spinlock_lock(_eth_tx_cb_lock);
+   prev_cb = >pre_tx_burst_cbs[queue_id];
+   for (; *prev_cb != NULL; prev_cb = >next) {
+   cb = *prev_cb;

[dpdk-dev] [PATCH v3 0/8] add packet capture framework

2016-05-17 Thread Reshma Pattan

This patchset include below changes
1)Changes to librte_ether.
2)New library librte_pdump added for packet capture framework.
3)New app/pdump tool added for packet capturing.
4)Test pmd changes done to initialize packet capture framework.
5)Documentation update.

1)librte_pdump
==
To support packet capturing on dpdk ethernet devices, a new library librte_pdump
is added.Users can develop their own packet capturing application using new 
library APIs.

Operation:
--
Pdump library provides APIs to support packet capturing on dpdk ethernet 
devices.
Library provides APIs to initialize the packet capture framework, enable/disable
the packet capture and un initialize the packet capture framework.

Pdump library works on server and client based model.

Sever is responsible for enabling/disabling the packet captures.
Clients are responsible for requesting enable/disable of the
packet captures.

As part of packet capture framework initialization, pthread and
the server socket is created. Only one server socket is allowed on the system.
As part of enabling/disabling the packet capture, client sockets are created
and multiple client sockets are allowed.
Who ever calls initialization first they will succeed with the initialization,
next subsequent calls of initialization are not allowed. So next users can only
request enabling/disabling the packet capture.

Applications using below APIs need to pass port/device_id, queue, mempool and
ring parameters. Library uses user provided ring and mempool to mirror the rx/tx
packets of the port for users. Users need to deque the rings and write the 
packets
to vdev(pcap/tuntap) to view the packets using any standard tools.

Note:
Mempool and Ring should be mc/mp supportable.
Mempool mbuf size should be big enough to handle the rx/tx packets of a port.

APIs:
-
rte_pdump_init()
rte_pdump_enable()
rte_pdump_enable_by_deviceid()
rte_pdump_disable()
rte_pdump_disable_by_deviceid()
rte_pdump_uninit()

2)app/pdump tool

Tool app/pdump is based on librte_pdump for packet capturing.

This tool by default runs as secondary process, and provides the support for
the command line options for packet capture.

 ./$(RTE_TARGET)/app/pdump -- --pdump='(port= |
   device_id=),
   (queue=2), (rx-dev= |
   tx-dev= |
   rxtx-dev=),
   [ring-size=1024], [mbuf-size=2048], [total-num-mbufs=8191]'

Parameters inside the parenthesis represents the mandatory parameters.
Parameters inside the square brackets represents optional parameters.
User has to pass on packet capture parameters under --pdump parameters, 
multiples of
--pdump can be passed to capture packets on different port and queue 
combinations

Operation:
--
*Tool parse the user command line arguments,
creates the mempool, ring and the PCAP PMD vdev with 'tx_stream' as either
of the device passed in rx-dev|tx-dev|rxtx-dev parameters.

*Then calls the APIs of librte_pdump i.e. 
rte_pdump_enable()/rte_pdump_enable_by_deviceid()
to enable packet capturing on a specific port/device_id and queue by passing on
port|device_id, queue, mempool and ring info.

*Tool runs in while loop to dequeue the packets from the ring and write them to 
pcap device.

*Tool can be stopped using SIGINT, upon which tool calls
rte_pdump_disable()/rte_pdump_disable_by_deviceid() and free the allocated 
resources.

Note:
CONFIG_RTE_LIBRTE_PMD_PCAP flag should be set to yes to compile and run the 
pdump tool.

3)Test-pmd changes
==
Changes are done to test-pmd application to initialize/uninitialize the packet 
capture framework.
So app/pdump tool can be run to see packets of dpdk ports that are used by 
test-pmd.

Similarly any application which needs packet capture should call 
initialize/uninitialize apis of
librate_pdump and use pdump tool to start the capture.

4)Packet capture flow between pdump tool and librte_pdump
=
* Pdump tool (Secondary process) requests packet capture
for specific port|device_id and queue combinations.

*Library in secondary process context creates client socket and communicates
the port|device_id, queue, ring and mempool to server.

*Library initializes server in primary process 'test-pmd' context and serves 
client
request to enable ethernet rxtx call-backs for given port|device_id and queue.?

*Copy the rx/tx packets to passed mempool and enqueue the packets to ring for 
secondary process.

*Pdump tool will dequeue the packets from ring and writes them to PCAPMD vdev,
so ultimately packets will be seen on device passed in rx-dev|tx-dev|rxtx-dev.

*Once the pdump tool is terminated with SIGINT it will disable packet capturing.

*Library receives the disable packet capture request, communicate the info to 
server,
server will remove the ethernet rxtx call-backs.

*Packet capture can be seen using tcpdump command
"tcpdump -ni " (or) "tcpdump ?nr "

5)Example command line
==
sudo

[dpdk-dev] [ovs-dev] Traffic scheduling by qos_sched library in DPDK

2016-05-17 Thread Dumitrescu, Cristian

Hi Gayathri,

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of
> gayathri.manepalli at wipro.com
> Sent: Friday, May 13, 2016 7:44 AM
> To: dev at dpdk.org
> Cc: Stokes, Ian 
> Subject: [dpdk-dev] [ovs-dev] Traffic scheduling by qos_sched library in
> DPDK
> 
> Hi Team,
> 
> I started working on implementing the QoS Shaping in OVS+DPDK by making
> use of rte_sched library provided in DPDK. 

Great news, thank you!

Meanwhile to compare the
> performance, started performance test with DPDK sample scheduling
> application. Below are the configuration details of system which I am using,
> 
> Server Type : Intel ATOM
> 
> Huge page configuration: (Each page of 2M size)
> 
> [root at ATOM qos_sched]# grep -i huge /proc/meminfo
> AnonHugePages: 90112 kB
> HugePages_Total:7168
> HugePages_Free: 7168
> HugePages_Rsvd:0
> HugePages_Surp:0
> Hugepagesize:   2048 kB
> 
> Port Capacity : 1G (All four ports)
> I am able to successfully run the qos_sched with three pfc's as below,
> 
> ./build/qos_sched -c 0x3e -n 1 --socket-mem 14336 -- --pfc "0,1,2,3" --pfc
> "1,0,3,2" --pfc "2,3,4,5" --cfg ./profile.cfg
> 
> Issue :
> 
> When I am trying to add one more profile configuration flow(4th one) , I am
> getting below error
> 
> Command : ./build/qos_sched -c 0x3e -n 1 --socket-mem 14336 -- --pfc
> "0,1,2,3" --pfc "1,0,3,2" --pfc "2,3,4,5" --pfc "3,2,5,4"  --cfg ./profile.cfg
> 
> Error:
> 
> done:  Link Up - speed 1000 Mbps - full-duplex
> SCHED: Low level config for pipe profile 0:
> Token bucket: period = 1, credits per period = 1, size = 10
> Traffic classes: period = 500, credits per period = [500, 
> 500,
> 500, 500]
>Traffic class 3 oversubscription: weight = 0
> WRR cost: [1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]
> SCHED: Low level config for subport 0:
> Token bucket: period = 1, credits per period = 1, size = 10
> Traffic classes: period = 125, credits per period = [125, 
> 125,
> 125, 125]
> Traffic class 3 oversubscription: wm min = 0, wm max = 0
> EAL: Error - exiting with code: 1
>   Cause: Cannot init mbuf pool for socket 3
> 
> Analysis:
> 
> I have analyzed DPDK source code to find the root cause. I could see that in
> qos_sched, memory allocation while creating each mbug pool
> (rte_mempool_create) for corresponding RX port is as below,
> 
> 
> MBUF_SIZE = (1528 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
> 
> mp_size  =  2*1024*1024
> 
> 
> 
> From above I understood that, for each pfc/ rx port around 4G of huge pages
> are consumed. Whereas ATOM is capable of maximum 7168 huge pages of
> 2M which is 14336M in total. So I am not able to configure beyond three pfc's.
> But I would like to measure the performance with 4 port & 6 port scenario
> which requires 4-6 pfc's configured.
> 
> 
> 
> Is there any other alternative through which I can configure more number of
> pfc's with my system configuration provided above.
> 
> 

Yes, you are probably running out of memory.

QoS hierarchical scheduler can be seen like a big reservoir of packets. Each 
rte_sched_port object basically has thousands of packet queues internally 
(number of queues is configurable). Ideally, to protect against the worst case 
scenario, , the number of buffers provisioned per each output port that has the 
hierarchical scheduler enabled needs to be at least equal to: number of 
scheduler queues x queue size. For example, for 64K queues (4K pipes with 16 
queues each) of 64 packets each, this means 4M buffers per output port, which 
(assuming 2KB buffers in the mempool) leads to 8 GB of memory per output port. 
In our examples/qos_sched sample application we decided to go mid-way instead 
of worst case scenario, therefore we use 2M buffers per output port.

So possible workarounds for you to maximize the amount of ports with 
hierarchical scheduler support given a fixed amount memory:
1. Lower the amount of buffers you use per output port, e.g. from 2M to 1M and 
restest;
2. Use less pipes per output port (configurable), so number of scheduling 
queues is less, which reduces the pressure on mempool size.

Note that you can use distinct mempools per output port (like in 
examples/qos_sched application) or a single big global mempool shared by all 
ports; this is transparent for the librte_sched library.

Regards,
Cristian

> 
> Thanks & Regards,
> 
> Gayathri
> 
> The information contained in this electronic message and any attachments to
> this message are intended for the exclusive use of the addressee(s) and may
> contain proprietary, confidential or privileged information. If you are not 
> the
> intended recipient, you should not disseminate, distribute or copy this e-
> mail. Please notify the sender immediately and destroy all copies of this
> message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The

[dpdk-dev] [PATCH v2] examples/ip_pipeline: fix NULL packet processing in source port

2016-05-17 Thread Fan Zhang

This patch fixes the NULL packet processing problem. Originally,
pipeline's write attempt to NULL packets generated by source
port may crash the application.

This patch fixes the problem by enforcing each source port
defined in cfg file containing a user specified or default pcap
file path.

Fixes: 0e1e7d53 ("add pcap file source")

Signed-off-by: Fan Zhang 
Acked-by: Cristian Dumitrescu 
---
 examples/ip_pipeline/config_parse.c | 235 ++--
 examples/ip_pipeline/init.c |  48 +++-
 2 files changed, 24 insertions(+), 259 deletions(-)

diff --git a/examples/ip_pipeline/config_parse.c 
b/examples/ip_pipeline/config_parse.c
index e5efd03..6710ee6 100644
--- a/examples/ip_pipeline/config_parse.c
+++ b/examples/ip_pipeline/config_parse.c
@@ -964,154 +964,6 @@ parse_eal(struct app_params *app,
 }

 static int
-parse_pipeline_pcap_source(struct app_params *app,
-   struct app_pipeline_params *p,
-   const char *file_name, const char *cp_size)
-{
-   const char *next = NULL;
-   char *end;
-   uint32_t i;
-   int parse_file = 0;
-
-   if (file_name && !cp_size) {
-   next = file_name;
-   parse_file = 1; /* parse file path */
-   } else if (cp_size && !file_name) {
-   next = cp_size;
-   parse_file = 0; /* parse copy size */
-   } else
-   return -EINVAL;
-
-   char name[APP_PARAM_NAME_SIZE];
-   size_t name_len;
-
-   if (p->n_pktq_in == 0)
-   return -EINVAL;
-
-   i = 0;
-   while (*next != '\0') {
-   uint32_t id;
-
-   if (i >= p->n_pktq_in)
-   return -EINVAL;
-
-   id = p->pktq_in[i].id;
-
-   end = strchr(next, ' ');
-   if (!end)
-   name_len = strlen(next);
-   else
-   name_len = end - next;
-
-   if (name_len == 0 || name_len == sizeof(name))
-   return -EINVAL;
-
-   strncpy(name, next, name_len);
-   name[name_len] = '\0';
-   next += name_len;
-   if (*next != '\0')
-   next++;
-
-   if (parse_file) {
-   app->source_params[id].file_name = strdup(name);
-   if (app->source_params[id].file_name == NULL)
-   return -ENOMEM;
-   } else {
-   if (parser_read_uint32(
-   >source_params[id].n_bytes_per_pkt,
-   name) != 0) {
-   if (app->source_params[id].
-   file_name != NULL)
-   free(app->source_params[id].
-   file_name);
-   return -EINVAL;
-   }
-   }
-
-   i++;
-
-   if (i == p->n_pktq_in)
-   return 0;
-   }
-
-   return -EINVAL;
-}
-
-static int
-parse_pipeline_pcap_sink(struct app_params *app,
-   struct app_pipeline_params *p,
-   const char *file_name, const char *n_pkts_to_dump)
-{
-   const char *next = NULL;
-   char *end;
-   uint32_t i;
-   int parse_file = 0;
-
-   if (file_name && !n_pkts_to_dump) {
-   next = file_name;
-   parse_file = 1; /* parse file path */
-   } else if (n_pkts_to_dump && !file_name) {
-   next = n_pkts_to_dump;
-   parse_file = 0; /* parse copy size */
-   } else
-   return -EINVAL;
-
-   char name[APP_PARAM_NAME_SIZE];
-   size_t name_len;
-
-   if (p->n_pktq_out == 0)
-   return -EINVAL;
-
-   i = 0;
-   while (*next != '\0') {
-   uint32_t id;
-
-   if (i >= p->n_pktq_out)
-   return -EINVAL;
-
-   id = p->pktq_out[i].id;
-
-   end = strchr(next, ' ');
-   if (!end)
-   name_len = strlen(next);
-   else
-   name_len = end - next;
-
-   if (name_len == 0 || name_len == sizeof(name))
-   return -EINVAL;
-
-   strncpy(name, next, name_len);
-   name[name_len] = '\0';
-   next += name_len;
-   if (*next != '\0')
-   next++;
-
-   if (parse_file) {
-   app->sink_params[id].file_name = strdup(name);
-   if (app->sink_params[id].file_name == NULL)
-   return -ENOMEM;
-   } else {
-   if (parser_read_uint32(
-   >sink_params[id].n_pkts_to_dump,
-   name) != 0) {
-   if (app->sink_params[id].file_name !=
-

[dpdk-dev] [PATCH 19/20] thunderx/nicvf: updated driver documentation and release notes

2016-05-17 Thread Mcnamara, John

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Jerin Jacob
> Sent: Saturday, May 7, 2016 4:17 PM
> To: dev at dpdk.org
> Cc: thomas.monjalon at 6wind.com; Richardson, Bruce
> ; Jerin Jacob
> ; Slawomir Rosek
> 
> Subject: [dpdk-dev] [PATCH 19/20] thunderx/nicvf: updated driver
> documentation and release notes

Hi,

Very good documentation. The content is quite clear and almost no RST issues.
The only comment is on some of the long lines. In general console blocks
have to be wrapped at 80 chars or else they go off the page in the PDF docs.
I see that you did that in some places but not in others.

It is worth building the pdf docs to check for that:

make doc-guides-pdf
mupdf build/doc/pdf/guides/nics.pdf &

Some minor comments below:


> +
> +#. Start ``testpmd`` with basic parameters:
> +
> +   .. code-block:: console
> +
> +  ./arm64-thunderx-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w
> + 0002:01:00.2 -- -i --disable-hw-vlan-filter --crc-strip --no-flush-rx
> + --port-topology=loop

Would be better wrapped as something like this:

   .. code-block:: console

  ./arm64-thunderx-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0002:01:00.2 \
  -- -i --disable-hw-vlan-filter --crc-strip --no-flush-rx
 --port-topology=loop


> +
> +   Example output:
> +
> +   .. code-block:: console
> +
> +  ...
> +
> +  PMD: rte_nicvf_pmd_init(): librte_pmd_thunderx nicvf version 1.0
> +
> +  ...
> +  EAL:   probe driver: 177d:11 rte_nicvf_pmd
> +  EAL:   using IOMMU type 1 (Type 1)
> +  EAL:   PCI memory mapped at 0x3ffade5
> +  EAL: Trying to map BAR 4 that contains the MSI-X table. Trying
> offsets: 0x400:0x, 0x1:0x1f
> +  EAL:   PCI memory mapped at 0x3ffadc6
> +  PMD: nicvf_eth_dev_init(): nicvf: device (177d:11) 2:1:0:2
> +  PMD: nicvf_eth_dev_init(): node=0 vf=1 mode=tns-bypass sqs=false
> loopback_supported=true
> +  PMD: nicvf_eth_dev_init(): Port 0 (177d:11) mac=a6:c6:d9:17:78:01
> +  Interactive-mode selected
> +  Configuring Port 0 (socket 0)


Also, this should be wrapped (even though it is the actual output):

  ...
  EAL:   probe driver: 177d:11 rte_nicvf_pmd
  EAL:   using IOMMU type 1 (Type 1)
  EAL:   PCI memory mapped at 0x3ffade5
  EAL: Trying to map BAR 4 that contains the MSI-X table.
   Trying offsets: 0x400:0x, 0x1:0x1f
  EAL:   PCI memory mapped at 0x3ffadc6
  PMD: nicvf_eth_dev_init(): nicvf: device (177d:11) 2:1:0:2
  PMD: nicvf_eth_dev_init(): node=0 vf=1 mode=tns-bypass sqs=false
   loopback_supported=true
  PMD: nicvf_eth_dev_init(): Port 0 (177d:11) mac=a6:c6:d9:17:78:01
  Interactive-mode selected
  Configuring Port 0 (socket 0)
  ...


> +SR-IOV: Prerequisites and sample Application Notes
> +~~
> +
> +Current ThunderX NIC PF/VF kernel modules maps each physical Ethernet
> +port automatically to virtual function (VF) and presented as PCIe-like
> SR-IOV device.


Slightly better as:

Current ThunderX NIC PF/VF kernel modules maps each physical Ethernet port
automatically to virtual functions (VF) and presents them as PCIe-like SR-IOV 
device.


> +   Example qemu guest launch command:
> +
> +   .. code-block:: console
> +
> +  sudo qemu-system-aarch64 -name vm1 -machine
> virt,gic_version=3,accel=kvm,usb=off \
> +  -cpu host -m 4096 \
> +  -smp 4,sockets=1,cores=8,threads=1 \
> +  -nographic -nodefaults \
> +  -kernel  \

Also wrap the first line:

   .. code-block:: console

  sudo qemu-system-aarch64 -name vm1 \
  -machine virt,gic_version=3,accel=kvm,usb=off \
  -cpu host -m 4096 \
  ...


Apart from those small changes:

Acked-by: John McNamara

[dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching

2016-05-17 Thread Thomas Monjalon

2016-05-17 13:44, Ananyev, Konstantin:
> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> > 2016-05-17 12:59, Ananyev, Konstantin:
> > > > > The rte_pktmbuf_detach() function should decrease refcnt on a direct
> > > > > buffer.
> > > >
> > > > As you have noticed, "whenever the indirect buffer is detached,
> > > > the reference counter on the direct buffer is decremented."
> > > > So the current behaviour of rte_pktmbuf_detach() is buggy.
> > > > Why not fix it without renaming?
> > > > If you consider this behavioral bug is part of the API, we
> > > > can fix it in a new function unattach and deprecate detach.
> > > > But Konstantin, why do you want to keep a restore function?
> > > > What is the need?
> > >
> > > I think it might be a useful functionality in some situations:
> > > some users can attach/detach to external memory buffers (no mbufs)
> > > and similar functionality is required.
> > 
> > Attach to external memory buffer (raw buffer) is not currently supported.
> > 
> > > Let say right now examples/vhost/main.c has its own pktmbuf_detach_zcp()
> > 
> > You should look at the commit http://dpdk.org/commit/68363d85
> > "examples/vhost: remove the non-working zero copy code"
> > 
> > > which is doing pretty much the same - restore original values, after 
> > > detaching
> > > mbuf from external (virtio) memory buffer.
> > > Would be good if we'll use a standard API function here.
> > 
> > You are welcome to implement mbuf attach to raw buffer.
> > But it is not a requirement for this fix.
> 
> Hmm, still not sure why we can't keep an existing function?

Because it does not do what its name (and doc) suggest.

> Obviously it wouldn't cost anything and I still think might be useful.

It costs to overcomplicate API for only a half support.
If you need the feature "attach to raw", please implement it completely.

[dpdk-dev] What is the status of mempool manager?

2016-05-17 Thread Olivier MATZ

Hi Jan,

On 05/17/2016 02:34 PM, Jan Viktorin wrote:
> Hello,
>
> I was trying to find out the status of the mempool rework and found
> this 36-pieces long patch set:
>
> http://dpdk.org/ml/archives/dev/2016-April/037464.html
>
> (I failed to apply it to 16.04, 2.1.0, current HEAD.)

It should be integrated soon, as there was no comment during
some time. I'll send a new version that applies on head
+ minor cosmetic changes.

> and some bits around:
>
> http://dpdk.org/ml/archives/dev/2016-May/038390.html

I need to review this one, it's on my todo list.

> http://dpdk.org/ml/archives/dev/2016-April/037509.html

I think the code is ready to be integrated, but I was expecting
some reactions on the v4, especially some precisions about the
use cases and the needs. See in the link "Things that should
still be discussed".

> http://dpdk.org/ml/archives/dev/2016-April/037457.html

It is integrated since today ;)

> http://dpdk.org/ml/archives/dev/2016-April/036979.html

Some comments have been done by Konstantin and myself, I
think a new version will be submitted. It will be integrated
in 16.07.

> ...but I am confused.
>
> I am trying to find out how to write a custom memory pool based on
> the uio_dmem_genirq driver. It provides DMA memory via the UIO API
> (maps).
>
> If I have a PMD running of top of this UIO, I need to implement a
> custom allocator that gets memory from the dev->mem_resource (with
> the mappings preloaded from the UIO by EAL).
>
> This is related to the SoC infra as given here:
>
> http://dpdk.org/ml/archives/dev/2016-May/038486.html
>
> Any idea how to start with this?

I think your feedback would be very valuable.

The "external mempool manager" series allows to register new
handler that replaces the internal ring storing the objects. It
does not change the way the memory is allocated.

The big patchset introduce new way of allocating memory used by
memory pool (no need to be virtually contiguous anymore). You
could throw an eye on rte_mempool_populate*() functions that
populate a mempool with memory for storing the objects.

I'm not sure I'm getting your exact needs.
Do you need to populate a ring-based mempool with specific memory,
retrieved from UIO API?
Do you need to replace the default internal ring by something else?
Do you need to access the mempool from the hardware?

Regards,
Olivier

[dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching

2016-05-17 Thread Ananyev, Konstantin



> -Original Message-
> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> Sent: Tuesday, May 17, 2016 3:19 PM
> To: Ananyev, Konstantin
> Cc: dev at dpdk.org; Hiroyuki Mikita; olivier.matz at 6wind.com
> Subject: Re: [dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching
> 
> 2016-05-17 13:44, Ananyev, Konstantin:
> > From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> > > 2016-05-17 12:59, Ananyev, Konstantin:
> > > > > > The rte_pktmbuf_detach() function should decrease refcnt on a direct
> > > > > > buffer.
> > > > >
> > > > > As you have noticed, "whenever the indirect buffer is detached,
> > > > > the reference counter on the direct buffer is decremented."
> > > > > So the current behaviour of rte_pktmbuf_detach() is buggy.
> > > > > Why not fix it without renaming?
> > > > > If you consider this behavioral bug is part of the API, we
> > > > > can fix it in a new function unattach and deprecate detach.
> > > > > But Konstantin, why do you want to keep a restore function?
> > > > > What is the need?
> > > >
> > > > I think it might be a useful functionality in some situations:
> > > > some users can attach/detach to external memory buffers (no mbufs)
> > > > and similar functionality is required.
> > >
> > > Attach to external memory buffer (raw buffer) is not currently supported.
> > >
> > > > Let say right now examples/vhost/main.c has its own pktmbuf_detach_zcp()
> > >
> > > You should look at the commit http://dpdk.org/commit/68363d85
> > >   "examples/vhost: remove the non-working zero copy code"
> > >
> > > > which is doing pretty much the same - restore original values, after 
> > > > detaching
> > > > mbuf from external (virtio) memory buffer.
> > > > Would be good if we'll use a standard API function here.
> > >
> > > You are welcome to implement mbuf attach to raw buffer.
> > > But it is not a requirement for this fix.
> >
> > Hmm, still not sure why we can't keep an existing function?
> 
> Because it does not do what its name (and doc) suggest.
> 
> > Obviously it wouldn't cost anything and I still think might be useful.
> 
> It costs to overcomplicate API for only a half support.

I still think it is better to have it then not, but wouldn't insist here.
Konstantin

> If you need the feature "attach to raw", please implement it completely.

[dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching

2016-05-17 Thread Thomas Monjalon

2016-05-17 12:59, Ananyev, Konstantin:
> > > The rte_pktmbuf_detach() function should decrease refcnt on a direct
> > > buffer.
> > 
> > As you have noticed, "whenever the indirect buffer is detached,
> > the reference counter on the direct buffer is decremented."
> > So the current behaviour of rte_pktmbuf_detach() is buggy.
> > Why not fix it without renaming?
> > If you consider this behavioral bug is part of the API, we
> > can fix it in a new function unattach and deprecate detach.
> > But Konstantin, why do you want to keep a restore function?
> > What is the need?
> 
> I think it might be a useful functionality in some situations:
> some users can attach/detach to external memory buffers (no mbufs)
> and similar functionality is required.

Attach to external memory buffer (raw buffer) is not currently supported.

> Let say right now examples/vhost/main.c has its own pktmbuf_detach_zcp()

You should look at the commit http://dpdk.org/commit/68363d85
"examples/vhost: remove the non-working zero copy code"

> which is doing pretty much the same - restore original values, after detaching
> mbuf from external (virtio) memory buffer.
> Would be good if we'll use a standard API function here.

You are welcome to implement mbuf attach to raw buffer.
But it is not a requirement for this fix.

[dpdk-dev] [PATCH] examples/ip_pipeline: fix NULL packet processing in source port

2016-05-17 Thread Thomas Monjalon

2016-05-17 13:37, Fan Zhang:
> This patch fixes the NULL packet processing problem. Originally,
> the write to NULL packets generated by source port will crash
> the ip pipeline application. This patch enforces, unless
> specified, a default pcap file path to be read.

I failed to understand.
Please describe clearly the bug in 1 or 2 sentences.
Then use a line break and describe the fix.
Thanks

[dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching

2016-05-17 Thread Thomas Monjalon

2016-05-17 01:53, Hiroyuki Mikita:
> The rte_pktmbuf_detach() function should decrease refcnt on a direct
> buffer.
> 
> Signed-off-by: Hiroyuki Mikita 
> ---
> v2:
> * introduced a new function rte_pktmbuf_detach2() which decrease refcnt.

As you have noticed, "whenever the indirect buffer is detached,
the reference counter on the direct buffer is decremented."
So the current behaviour of rte_pktmbuf_detach() is buggy.
Why not fix it without renaming?
If you consider this behavioral bug is part of the API, we
can fix it in a new function unattach and deprecate detach.
But Konstantin, why do you want to keep a restore function?
What is the need?

Please explicit the function name for the detach operation in
doc/guides/prog_guide/mbuf_lib.rst (whatever detach2 or unattach).

> * marked rte_pktmbuf_detach() as deprecated.
> * added comments about refcnt to rte_pktmbuf_attach() and 
> rte_pktmbuf_detach().
> * checked refcnt when detaching in unit tests.
> * added this issue to release notes.

[dpdk-dev] What is the status of mempool manager?

2016-05-17 Thread Jan Viktorin

Hello,

I was trying to find out the status of the mempool rework and found
this 36-pieces long patch set:

http://dpdk.org/ml/archives/dev/2016-April/037464.html

(I failed to apply it to 16.04, 2.1.0, current HEAD.)

and some bits around:

http://dpdk.org/ml/archives/dev/2016-May/038390.html
http://dpdk.org/ml/archives/dev/2016-April/037509.html
http://dpdk.org/ml/archives/dev/2016-April/037457.html
http://dpdk.org/ml/archives/dev/2016-April/036979.html

...but I am confused.

I am trying to find out how to write a custom memory pool based on
the uio_dmem_genirq driver. It provides DMA memory via the UIO API
(maps).

If I have a PMD running of top of this UIO, I need to implement a
custom allocator that gets memory from the dev->mem_resource (with
the mappings preloaded from the UIO by EAL).

This is related to the SoC infra as given here:

http://dpdk.org/ml/archives/dev/2016-May/038486.html

Any idea how to start with this?

Regards
Jan

-- 
   Jan Viktorin  E-mail: Viktorin at RehiveTech.com
   System Architect  Web:www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

[dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching

2016-05-17 Thread Ananyev, Konstantin



> -Original Message-
> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> Sent: Tuesday, May 17, 2016 2:40 PM
> To: Ananyev, Konstantin
> Cc: dev at dpdk.org; Hiroyuki Mikita; olivier.matz at 6wind.com
> Subject: Re: [dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching
> 
> 2016-05-17 12:59, Ananyev, Konstantin:
> > > > The rte_pktmbuf_detach() function should decrease refcnt on a direct
> > > > buffer.
> > >
> > > As you have noticed, "whenever the indirect buffer is detached,
> > > the reference counter on the direct buffer is decremented."
> > > So the current behaviour of rte_pktmbuf_detach() is buggy.
> > > Why not fix it without renaming?
> > > If you consider this behavioral bug is part of the API, we
> > > can fix it in a new function unattach and deprecate detach.
> > > But Konstantin, why do you want to keep a restore function?
> > > What is the need?
> >
> > I think it might be a useful functionality in some situations:
> > some users can attach/detach to external memory buffers (no mbufs)
> > and similar functionality is required.
> 
> Attach to external memory buffer (raw buffer) is not currently supported.
> 
> > Let say right now examples/vhost/main.c has its own pktmbuf_detach_zcp()
> 
> You should look at the commit http://dpdk.org/commit/68363d85
>   "examples/vhost: remove the non-working zero copy code"
> 
> > which is doing pretty much the same - restore original values, after 
> > detaching
> > mbuf from external (virtio) memory buffer.
> > Would be good if we'll use a standard API function here.
> 
> You are welcome to implement mbuf attach to raw buffer.
> But it is not a requirement for this fix.

Hmm, still not sure why we can't keep an existing function?
Obviously it wouldn't cost anything and I still think might be useful.
Konstantin

[dpdk-dev] [PATCH] examples/ip_pipeline: fix NULL packet processing in source port

2016-05-17 Thread Fan Zhang

This patch fixes the NULL packet processing problem. Originally,
the write to NULL packets generated by source port will crash
the ip pipeline application. This patch enforces, unless
specified, a default pcap file path to be read.

Fixes: 0e1e7d53 ("add pcap file source")

Signed-off-by: Fan Zhang 
Acked-by: Cristian Dumitrescu 
---
 examples/ip_pipeline/config_parse.c | 235 ++--
 examples/ip_pipeline/init.c |  48 +++-
 2 files changed, 24 insertions(+), 259 deletions(-)

diff --git a/examples/ip_pipeline/config_parse.c 
b/examples/ip_pipeline/config_parse.c
index e5efd03..6710ee6 100644
--- a/examples/ip_pipeline/config_parse.c
+++ b/examples/ip_pipeline/config_parse.c
@@ -964,154 +964,6 @@ parse_eal(struct app_params *app,
 }

 static int
-parse_pipeline_pcap_source(struct app_params *app,
-   struct app_pipeline_params *p,
-   const char *file_name, const char *cp_size)
-{
-   const char *next = NULL;
-   char *end;
-   uint32_t i;
-   int parse_file = 0;
-
-   if (file_name && !cp_size) {
-   next = file_name;
-   parse_file = 1; /* parse file path */
-   } else if (cp_size && !file_name) {
-   next = cp_size;
-   parse_file = 0; /* parse copy size */
-   } else
-   return -EINVAL;
-
-   char name[APP_PARAM_NAME_SIZE];
-   size_t name_len;
-
-   if (p->n_pktq_in == 0)
-   return -EINVAL;
-
-   i = 0;
-   while (*next != '\0') {
-   uint32_t id;
-
-   if (i >= p->n_pktq_in)
-   return -EINVAL;
-
-   id = p->pktq_in[i].id;
-
-   end = strchr(next, ' ');
-   if (!end)
-   name_len = strlen(next);
-   else
-   name_len = end - next;
-
-   if (name_len == 0 || name_len == sizeof(name))
-   return -EINVAL;
-
-   strncpy(name, next, name_len);
-   name[name_len] = '\0';
-   next += name_len;
-   if (*next != '\0')
-   next++;
-
-   if (parse_file) {
-   app->source_params[id].file_name = strdup(name);
-   if (app->source_params[id].file_name == NULL)
-   return -ENOMEM;
-   } else {
-   if (parser_read_uint32(
-   >source_params[id].n_bytes_per_pkt,
-   name) != 0) {
-   if (app->source_params[id].
-   file_name != NULL)
-   free(app->source_params[id].
-   file_name);
-   return -EINVAL;
-   }
-   }
-
-   i++;
-
-   if (i == p->n_pktq_in)
-   return 0;
-   }
-
-   return -EINVAL;
-}
-
-static int
-parse_pipeline_pcap_sink(struct app_params *app,
-   struct app_pipeline_params *p,
-   const char *file_name, const char *n_pkts_to_dump)
-{
-   const char *next = NULL;
-   char *end;
-   uint32_t i;
-   int parse_file = 0;
-
-   if (file_name && !n_pkts_to_dump) {
-   next = file_name;
-   parse_file = 1; /* parse file path */
-   } else if (n_pkts_to_dump && !file_name) {
-   next = n_pkts_to_dump;
-   parse_file = 0; /* parse copy size */
-   } else
-   return -EINVAL;
-
-   char name[APP_PARAM_NAME_SIZE];
-   size_t name_len;
-
-   if (p->n_pktq_out == 0)
-   return -EINVAL;
-
-   i = 0;
-   while (*next != '\0') {
-   uint32_t id;
-
-   if (i >= p->n_pktq_out)
-   return -EINVAL;
-
-   id = p->pktq_out[i].id;
-
-   end = strchr(next, ' ');
-   if (!end)
-   name_len = strlen(next);
-   else
-   name_len = end - next;
-
-   if (name_len == 0 || name_len == sizeof(name))
-   return -EINVAL;
-
-   strncpy(name, next, name_len);
-   name[name_len] = '\0';
-   next += name_len;
-   if (*next != '\0')
-   next++;
-
-   if (parse_file) {
-   app->sink_params[id].file_name = strdup(name);
-   if (app->sink_params[id].file_name == NULL)
-   return -ENOMEM;
-   } else {
-   if (parser_read_uint32(
-   >sink_params[id].n_pkts_to_dump,
-   name) != 0) {
-   if (app->sink_params[id].file_name !=
-   NULL)
-

[dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching

2016-05-17 Thread Ananyev, Konstantin


Hi Thomas,

> > The rte_pktmbuf_detach() function should decrease refcnt on a direct
> > buffer.
> >
> > Signed-off-by: Hiroyuki Mikita 
> > ---
> > v2:
> > * introduced a new function rte_pktmbuf_detach2() which decrease refcnt.
> 
> As you have noticed, "whenever the indirect buffer is detached,
> the reference counter on the direct buffer is decremented."
> So the current behaviour of rte_pktmbuf_detach() is buggy.
> Why not fix it without renaming?
> If you consider this behavioral bug is part of the API, we
> can fix it in a new function unattach and deprecate detach.
> But Konstantin, why do you want to keep a restore function?
> What is the need?

I think it might be a useful functionality in some situations:
some users can attach/detach to external memory buffers (no mbufs)
and similar functionality is required.
Let say right now examples/vhost/main.c has its own pktmbuf_detach_zcp()
which is doing pretty much the same - restore original values, after detaching
mbuf from external (virtio) memory buffer.
Would be good if we'll use a standard API function here.
Konstantin  

> 
> Please explicit the function name for the detach operation in
> doc/guides/prog_guide/mbuf_lib.rst (whatever detach2 or unattach).
> 
> > * marked rte_pktmbuf_detach() as deprecated.
> > * added comments about refcnt to rte_pktmbuf_attach() and 
> > rte_pktmbuf_detach().
> > * checked refcnt when detaching in unit tests.
> > * added this issue to release notes.

[dpdk-dev] [PATCH v2 7/7] config: enable virtio-net pmd for ppc64

2016-05-17 Thread Olivier Matz

Now that virtio pmd is supported on ppc, enable it.

Signed-off-by: Olivier Matz 
---
 config/defconfig_ppc_64-power8-linuxapp-gcc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/config/defconfig_ppc_64-power8-linuxapp-gcc 
b/config/defconfig_ppc_64-power8-linuxapp-gcc
index 9eb0cc4..bef8f49 100644
--- a/config/defconfig_ppc_64-power8-linuxapp-gcc
+++ b/config/defconfig_ppc_64-power8-linuxapp-gcc
@@ -50,7 +50,7 @@ CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=n
 # Will turn on them only after the successful testing on Power
 CONFIG_RTE_LIBRTE_IXGBE_PMD=n
 CONFIG_RTE_LIBRTE_I40E_PMD=n
-CONFIG_RTE_LIBRTE_VIRTIO_PMD=n
+CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
 CONFIG_RTE_LIBRTE_VMXNET3_PMD=n
 CONFIG_RTE_LIBRTE_PMD_BOND=n
 CONFIG_RTE_LIBRTE_ENIC_PMD=n
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 6/7] virtio: fix pci accesses for ppc64 in legacy mode

2016-05-17 Thread Olivier Matz

From: David Marchand 

Although ppc supports both endianesses, qemu supposes that the cpu is
big endian and enforces this for the virtio-net stuff.

Fix PCI accesses in legacy mode. Only ppc64le is supported at the moment.

Signed-off-by: David Marchand 
Signed-off-by: Olivier Matz 
---
 drivers/net/virtio/virtio_pci.c | 68 +
 1 file changed, 68 insertions(+)

diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 9cdca06..ebf4cf7 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -55,20 +55,88 @@
  */
 #define VIRTIO_PCI_CONFIG(hw) (((hw)->use_msix) ? 24 : 20)

+/*
+ * Since we are in legacy mode:
+ * http://ozlabs.org/~rusty/virtio-spec/virtio-0.9.5.pdf
+ *
+ * "Note that this is possible because while the virtio header is PCI (i.e.
+ * little) endian, the device-specific region is encoded in the native endian 
of
+ * the guest (where such distinction is applicable)."
+ *
+ * For powerpc which supports both, qemu supposes that cpu is big endian and
+ * enforces this for the virtio-net stuff.
+ */
+
 static void
 legacy_read_dev_config(struct virtio_hw *hw, size_t offset,
   void *dst, int length)
 {
+#ifdef RTE_ARCH_PPC_64
+   int size;
+
+   while (length > 0) {
+   if (length >= 4) {
+   size = 4;
+   rte_eal_pci_ioport_read(>io, dst, size,
+   VIRTIO_PCI_CONFIG(hw) + offset);
+   *(uint32_t *)dst = rte_be_to_cpu_32(*(uint32_t *)dst);
+   } else if (length >= 2) {
+   size = 2;
+   rte_eal_pci_ioport_read(>io, dst, size,
+   VIRTIO_PCI_CONFIG(hw) + offset);
+   *(uint16_t *)dst = rte_be_to_cpu_16(*(uint16_t *)dst);
+   } else {
+   size = 1;
+   rte_eal_pci_ioport_read(>io, dst, size,
+   VIRTIO_PCI_CONFIG(hw) + offset);
+   }
+
+   dst = (char *)dst + size;
+   offset += size;
+   length -= size;
+   }
+#else
rte_eal_pci_ioport_read(>io, dst, length,
VIRTIO_PCI_CONFIG(hw) + offset);
+#endif
 }

 static void
 legacy_write_dev_config(struct virtio_hw *hw, size_t offset,
const void *src, int length)
 {
+#ifdef RTE_ARCH_PPC_64
+   union {
+   uint32_t u32;
+   uint16_t u16;
+   } tmp;
+   int size;
+
+   while (length > 0) {
+   if (length >= 4) {
+   size = 4;
+   tmp.u32 = rte_cpu_to_be_32(*(const uint32_t *)src);
+   rte_eal_pci_ioport_write(>io, , size,
+   VIRTIO_PCI_CONFIG(hw) + offset);
+   } else if (length >= 2) {
+   size = 2;
+   tmp.u16 = rte_cpu_to_be_16(*(const uint16_t *)src);
+   rte_eal_pci_ioport_write(>io, , size,
+   VIRTIO_PCI_CONFIG(hw) + offset);
+   } else {
+   size = 1;
+   rte_eal_pci_ioport_write(>io, src, size,
+   VIRTIO_PCI_CONFIG(hw) + offset);
+   }
+
+   src = (const char *)src + size;
+   offset += size;
+   length -= size;
+   }
+#else
rte_eal_pci_ioport_write(>io, src, length,
 VIRTIO_PCI_CONFIG(hw) + offset);
+#endif
 }

 static uint64_t
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

2016-05-17 Thread Olivier Matz

On PPC64, the ioports are mapped in memory. Implement the missing part
of ioport API for PPC64 when using uio. This may also work on other
architectures but it has not been tested.

Signed-off-by: David Marchand 
Signed-off-by: Olivier Matz 
---
 lib/librte_eal/common/include/rte_pci.h|   4 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c  |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci_init.h |  10 +++
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c  | 119 +++--
 4 files changed, 108 insertions(+), 27 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h 
b/lib/librte_eal/common/include/rte_pci.h
index fd049d1..565bf38 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -105,9 +105,6 @@ extern struct pci_device_list pci_device_list; /**< Global 
list of PCI devices.
 /** Nb. of values in PCI resource format. */
 #define PCI_RESOURCE_FMT_NVAL 3

-/** IO resource type: memory address space */
-#define IORESOURCE_MEM0x0200
-
 /**
  * A structure describing a PCI resource.
  */
@@ -518,6 +515,7 @@ int rte_eal_pci_write_config(const struct rte_pci_device 
*device,
 struct rte_pci_ioport {
struct rte_pci_device *dev;
uint64_t base;
+   uint64_t len; /* only filled for memory mapped ports */
 };

 /**
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c 
b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 1a93725..30e2328 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -193,7 +193,7 @@ pci_find_max_end_va(void)
 /* parse one line of the "resource" sysfs file (note that the 'line'
  * string is modified)
  */
-static int
+int
 pci_parse_one_sysfs_resource(char *line, size_t len, uint64_t *phys_addr,
uint64_t *end_addr, uint64_t *flags)
 {
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h 
b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
index 7011753..f72a254 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
@@ -36,12 +36,22 @@

 #include "eal_vfio.h"

+/** IO resource type: */
+#define IORESOURCE_IO 0x0100
+#define IORESOURCE_MEM0x0200
+
 /*
  * Helper function to map PCI resources right after hugepages in virtual memory
  */
 extern void *pci_map_addr;
 void *pci_find_max_end_va(void);

+/* parse one line of the "resource" sysfs file (note that the 'line'
+ * string is modified)
+ */
+int pci_parse_one_sysfs_resource(char *line, size_t len, uint64_t *phys_addr,
+   uint64_t *end_addr, uint64_t *flags);
+
 int pci_uio_alloc_resource(struct rte_pci_device *dev,
struct mapped_pci_resource **uio_res);
 void pci_uio_free_resource(struct rte_pci_device *dev,
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c 
b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index ac449c5..077ad96 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -35,6 +35,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -368,11 +369,11 @@ error:
return -1;
 }

+#if defined(RTE_ARCH_X86)
 int
 pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
   struct rte_pci_ioport *p)
 {
-#if defined(RTE_ARCH_X86)
char dirname[PATH_MAX];
char filename[PATH_MAX];
int uio_num;
@@ -411,81 +412,153 @@ pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%lx\n", start);

p->base = start;
+   p->len = 0;
return 0;
+}
 #else
-   RTE_SET_USED(dev);
-   RTE_SET_USED(bar);
-   RTE_SET_USED(p);
+int
+pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
+  struct rte_pci_ioport *p)
+{
+   FILE *f;
+   char buf[BUFSIZ];
+   char filename[PATH_MAX];
+   uint64_t phys_addr, end_addr, flags;
+   int fd, i;
+   void *addr;
+
+   /* open and read addresses of the corresponding resource in sysfs */
+   snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource",
+   SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
+dev->addr.devid, dev->addr.function);
+   f = fopen(filename, "r");
+   if (f == NULL) {
+   RTE_LOG(ERR, EAL, "Cannot open sysfs resource: %s\n",
+   strerror(errno));
+   return -1;
+   }
+   for (i = 0; i < bar + 1; i++) {
+   if (fgets(buf, sizeof(buf), f) == NULL) {
+   RTE_LOG(ERR, EAL, "Cannot read sysfs resource\n");
+   goto error;
+   }
+   }
+   if (pci_parse_one_sysfs_resource(buf, sizeof(buf), _addr,
+   _addr, ) < 0)
+   goto error;
+   if ((flags & IORESOURCE_IO) == 0) {
+   RTE_LOG(ERR, EAL, "BAR %d is not an IO resource\n", bar);
+   goto error;
+   }
+   snprintf(filename,

[dpdk-dev] [PATCH v2 4/7] eal/linux: split function parsing pci resources in sysfs

2016-05-17 Thread Olivier Matz

Split pci_parse_sysfs_resource() and introduce
pci_parse_one_sysfs_resource() that parses one line of sysfs resource
file.

This new function will be exported and used in next commits when
mapping the ioports resources.

No functional change.

Signed-off-by: Olivier Matz 
---
 lib/librte_eal/linuxapp/eal/eal_pci.c | 50 ++-
 1 file changed, 32 insertions(+), 18 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c 
b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bdc08a0..1a93725 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -190,12 +190,13 @@ pci_find_max_end_va(void)
return RTE_PTR_ADD(last->addr, last->len);
 }

-/* parse the "resource" sysfs file */
+/* parse one line of the "resource" sysfs file (note that the 'line'
+ * string is modified)
+ */
 static int
-pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
+pci_parse_one_sysfs_resource(char *line, size_t len, uint64_t *phys_addr,
+   uint64_t *end_addr, uint64_t *flags)
 {
-   FILE *f;
-   char buf[BUFSIZ];
union pci_resource_info {
struct {
char *phys_addr;
@@ -204,6 +205,31 @@ pci_parse_sysfs_resource(const char *filename, struct 
rte_pci_device *dev)
};
char *ptrs[PCI_RESOURCE_FMT_NVAL];
} res_info;
+
+   if (rte_strsplit(line, len, res_info.ptrs, 3, ' ') != 3) {
+   RTE_LOG(ERR, EAL,
+   "%s(): bad resource format\n", __func__);
+   return -1;
+   }
+   errno = 0;
+   *phys_addr = strtoull(res_info.phys_addr, NULL, 16);
+   *end_addr = strtoull(res_info.end_addr, NULL, 16);
+   *flags = strtoull(res_info.flags, NULL, 16);
+   if (errno != 0) {
+   RTE_LOG(ERR, EAL,
+   "%s(): bad resource format\n", __func__);
+   return -1;
+   }
+
+   return 0;
+}
+
+/* parse the "resource" sysfs file */
+static int
+pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
+{
+   FILE *f;
+   char buf[BUFSIZ];
int i;
uint64_t phys_addr, end_addr, flags;

@@ -220,21 +246,9 @@ pci_parse_sysfs_resource(const char *filename, struct 
rte_pci_device *dev)
"%s(): cannot read resource\n", __func__);
goto error;
}
-
-   if (rte_strsplit(buf, sizeof(buf), res_info.ptrs, 3, ' ') != 3) 
{
-   RTE_LOG(ERR, EAL,
-   "%s(): bad resource format\n", __func__);
+   if (pci_parse_one_sysfs_resource(buf, sizeof(buf), _addr,
+   _addr, ) < 0)
goto error;
-   }
-   errno = 0;
-   phys_addr = strtoull(res_info.phys_addr, NULL, 16);
-   end_addr = strtoull(res_info.end_addr, NULL, 16);
-   flags = strtoull(res_info.flags, NULL, 16);
-   if (errno != 0) {
-   RTE_LOG(ERR, EAL,
-   "%s(): bad resource format\n", __func__);
-   goto error;
-   }

if (flags & IORESOURCE_MEM) {
dev->mem_resource[i].phys_addr = phys_addr;
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 3/7] eal/linux: remove invalid comment

2016-05-17 Thread Olivier Matz

In a previous commit, the file used to map the PCI resources changed
from "/dev/uio" to "/sys/bus/pci/devices//resource", making
the comment wrong. Remove it.

Fixes: 9e67561acd1a ("eal/linux: mmap uio resources using resourceX files")
Signed-off-by: Olivier Matz 
---
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c 
b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 068694d..ac449c5 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -309,7 +309,7 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, 
int res_idx,
struct mapped_pci_resource *uio_res, int map_idx)
 {
int fd;
-   char devname[PATH_MAX]; /* contains the /dev/uioX */
+   char devname[PATH_MAX];
void *mapaddr;
struct rte_pci_addr *loc;
struct pci_map *maps;
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 2/7] eal/linux: only call iopl on x86

2016-05-17 Thread Olivier Matz

>From iopl(2) man page: "This call is mostly for the x86 architecture. On
many other architectures it does not exist or will always return an
error".

This patch removes the call to iopl() in rte_eal_iopl_init() for
architectures other than x86, and always return 0 (success). This was
already done for ARM in
commit 0291476ae364 ("eal/linux: never check iopl for arm")

Next patches will introduce the support of memory mapped IO resources
for architectures != x86.

On BSD, there is nothing to do as open("/dev/io") already does the
proper thing. See man IO(4).

Signed-off-by: Olivier Matz 
---
 lib/librte_eal/linuxapp/eal/eal.c | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal.c 
b/lib/librte_eal/linuxapp/eal/eal.c
index 8aafd51..bba8fea 100644
--- a/lib/librte_eal/linuxapp/eal/eal.c
+++ b/lib/librte_eal/linuxapp/eal/eal.c
@@ -715,12 +715,8 @@ rte_eal_iopl_init(void)
 #if defined(RTE_ARCH_X86)
if (iopl(3) != 0)
return -1;
-   return 0;
-#elif defined(RTE_ARCH_ARM) || defined(RTE_ARCH_ARM64)
-   return 0; /* iopl syscall not supported for ARM/ARM64 */
-#else
-   return -1;
 #endif
+   return 0;
 }

 /* Launch threads, called at application init(). */
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 1/7] eal: fix typos in ioport API doxygen comments

2016-05-17 Thread Olivier Matz

Fix some typos and add missing comments related to ioports API in
rte_pci.h.

Fixes: 756ce64b1 ("eal: introduce PCI ioport API")
Signed-off-by: Olivier Matz 
---
 lib/librte_eal/common/include/rte_pci.h | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h 
b/lib/librte_eal/common/include/rte_pci.h
index 8fa2712..fd049d1 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -521,12 +521,13 @@ struct rte_pci_ioport {
 };

 /**
- * Initialises a rte_pci_ioport object for a pci device io resource.
+ * Initialize a rte_pci_ioport object for a pci device io resource.
+ *
  * This object is then used to gain access to those io resources (see below).
  *
  * @param dev
- *   A pointer to a rte_pci_device structure describing the device.
- *   to use
+ *   A pointer to a rte_pci_device structure describing the device
+ *   to use.
  * @param bar
  *   Index of the io pci resource we want to access.
  * @param p
@@ -542,6 +543,8 @@ int rte_eal_pci_ioport_map(struct rte_pci_device *dev, int 
bar,
  *
  * @param p
  *   The rte_pci_ioport object to be uninitialized.
+ * @return
+ *  0 on success, negative on error.
  */
 int rte_eal_pci_ioport_unmap(struct rte_pci_ioport *p);

-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 0/7] virtio-net support on ppc64

2016-05-17 Thread Olivier Matz

This patchset allows virtio-net pmd to run on ppc64 processors.
The main thing that as missing was the support of ioports in EAL.
It also fixes some endianess issues in PCI config accesses in
legacy mode.

v1 -> v2:
  - fix read/write lengths management in virtio (this fixes
the reading of mac address)
  - fix garbage in comment
  - move linux specific definition into a linux specific
header
  - replay the test plan


This is validated with test-pmd:

=== HOST

mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 128 > 
/sys/devices/system/node/node0/hugepages/hugepages-16384kB/nr_hugepages 
qemu-system-ppc64le --enable-kvm -m 1G -mem-path "/mnt/huge" -cpu host -smp 3 \
  -serial telnet::58028,server,nowait -serial null \
  -device ne2k_pci,mac=de:ad:de:01:02:03,netdev=user.0,addr=03 \
  -netdev user,id=user.0,hostfwd=tcp::49680-:22 \
  -netdev type=tap,id=vhostnet0,script=no,vhost=on,queues=8 \
  -device virtio-net-pci,netdev=vhostnet0,ioeventfd=on,mq=on,vectors=17 \
  -hda /path/to/ubuntu-14.04-ppc64el.qcow2 \
  -vga none -display none
ip l set tap0 up
ip a a 1.1.1.1/24 dev tap0
# start guest, then:
ping 1.1.1.2

=== GUEST

cd dpdk.org/
make config T=ppc_64-power8-linuxapp-gcc
make
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 64 > 
/sys/devices/system/node/node0/hugepages/hugepages-16384kB/nr_hugepages 
modprobe uio_pci_generic
python tools/dpdk_nic_bind.py -b uio_pci_generic :00:00.0
./build/app/testpmd -l 0,1 --log-level 8 -- --total-num-mbufs=16384 -i 
--port-topology=chained
EAL: Detected 3 lcore(s)
EAL: Probing VFIO support...
EAL: PCI device :00:00.0 on NUMA socket -1
EAL:   probe driver: 1af4:1000 rte_virtio_pmd
Interactive-mode selected
Configuring Port 0 (socket 0)
Port 0: 52:54:00:12:34:56
Checking link statuses...
Port 0 Link Up - speed 1 Mbps - full-duplex
Done
testpmd> set fwd icmpecho
Set icmpecho packet forwarding mode
testpmd> set verbose 1
Change verbose level from 0 to 1
testpmd> start
icmp_echo_config_setup fwd_cores=1 fwd_ports=1 fwd_streams=1
  core=0:
  stream=0 port=0 rxq=0 txq=0
  icmpecho packet forwarding - CRC stripping disabled - packets/burst=32
  nb forwarding cores=1 - nb forwarding ports=1
  RX queues=1 - RX desc=128 - RX free threshold=0
  RX threshold registers: pthresh=0 hthresh=0 wthresh=0
  TX queues=1 - TX desc=512 - TX free threshold=0
  TX threshold registers: pthresh=0 hthresh=0 wthresh=0
  TX RS bit threshold=0 - TXQ flags=0xf00
testpmd>
Port 0 pkt-len=98 nb-segs=1
  ETH:  src=46:A9:44:17:62:F1 dst=52:54:00:12:34:56 type=0x0800
  IPV4: src=1.1.1.1 dst=1.1.1.2 proto=1 (ICMP)
  ICMP: echo request seq id=1

Port 0 pkt-len=98 nb-segs=1
  ETH:  src=46:A9:44:17:62:F1 dst=52:54:00:12:34:56 type=0x0800
  IPV4: src=1.1.1.1 dst=1.1.1.2 proto=1 (ICMP)
  ICMP: echo request seq id=2


David Marchand (1):
  virtio: fix pci accesses for ppc64 in legacy mode

Olivier Matz (6):
  eal: fix typos in ioport API doxygen comments
  eal/linux: only call iopl on x86
  eal/linux: remove invalid comment
  eal/linux: split function parsing pci resources in sysfs
  eal/linux: mmap ioports on ppc64
  config: enable virtio-net pmd for ppc64

 config/defconfig_ppc_64-power8-linuxapp-gcc |   2 +-
 drivers/net/virtio/virtio_pci.c |  68 
 lib/librte_eal/common/include/rte_pci.h |  13 +--
 lib/librte_eal/linuxapp/eal/eal.c   |   6 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c   |  52 +++-
 lib/librte_eal/linuxapp/eal/eal_pci_init.h  |  10 +++
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c   | 121 ++--
 7 files changed, 217 insertions(+), 55 deletions(-)

-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching

2016-05-17 Thread Bruce Richardson

On Tue, May 17, 2016 at 01:53:20AM +0900, Hiroyuki Mikita wrote:
> The rte_pktmbuf_detach() function should decrease refcnt on a direct
> buffer.
> 
> Signed-off-by: Hiroyuki Mikita 
> ---
> v2:
> * introduced a new function rte_pktmbuf_detach2() which decrease refcnt.
> * marked rte_pktmbuf_detach() as deprecated.
> * added comments about refcnt to rte_pktmbuf_attach() and 
> rte_pktmbuf_detach().
> * checked refcnt when detaching in unit tests.
> * added this issue to release notes.
> 
Rather than detach2, would unattach be better?

/Bruce

[dpdk-dev] possible kni bug and proposed fix

2016-05-17 Thread Ferruh Yigit

On 5/16/2016 4:31 PM, ALeX Wang wrote:
> Hi Ferruh,
> 
> Thx for pointing out the 'fill alloc_q with these mubfs _until it gets
> full_.',
> 
> I saw the size of all queues are 'KNI_FIFO_COUNT_MAX (1024)'...
> The corresponding memory required is more than what I specify as
> 'socket_mem' (since i'm using VM)...
> 

If the mempool is not big enough to fill the ring, this explains the
error log. Logic is to fill the alloc_q, but if you are missing the
required mbufs, in each rte_kni_rx_burst() it will complain about memory.

But the required memory for mbufs to fill the ring is not too much. It
should be ~2Mbytes, are you sure you are missing this much memory?

> Also, in my use case, I only `tcpreplay` through the kni interface, and
> my application only do rx and then free the 'mbufs'.  So there is no tx
> at all.
> 
> So, in my case, I still think this is a bug/defect, or somewhere i still
> misunderstand,
> 
> P.S. The description here seems to be inverted,
> http://dpdk.org/doc/api/rte__kni_8h.html#a0cdd727cdc227d005fef22c0189f3dfe
> 'rte_kni_rx_burst' does the 'It handles allocating the mbufs for KNI
> interface alloc queue.'
> 

You are right. That part of the description for rte_kni_rx_burst and
rte_kni_tx_burst needs to be replaced. Do you want to send a patch?

> Thanks,
> Alex Wang,
> 
> On 16 May 2016 at 04:20, Ferruh Yigit  > wrote:
> 
> On 5/15/2016 5:48 AM, ALeX Wang wrote:
> > Hi,
> >
> > When using the kni module to test my application inside
> > debian (virtualbox) VM (kernel version 4.4), I get the
> >
> > "KNI: Out of memory"
> >
> > from syslog every time I `tcpreply` packets through
> > the kni interface.
> >
> > After checking source code, I saw that when I call
> > 'rte_kni_rx_burst()', no matter how many packets
> > are actually retrieved, we always call 'kni_allocate_mbufs()'
> > and try allocate 'MAX_MBUF_BURST_NUM' more
> > mbufs...  I fix the issue via using this patch below,
> >
> > Could you confirm if this is an actual bug?
> >
> 
> Hi Alex,
> 
> I don't think this is a bug.
> 
> kni_allocate_mbufs() will allocate MAX_MBUF_BURST_NUM mbufs as you
> mentioned. And will fill alloc_q with these mubfs _until it gets full_.
> And if the alloc_q is full and there are remaining mbufs, they will be
> freed. So for some cases this isn't the most optimized way, but there is
> no defect.
> 
> Since you are getting "KNI: Out of memory", somewhere else can be
> missing freeing mbufs.
> 
> mbufs freeing done in rte_kni_tx_burst(), I can guess two cases that can
> cause problem:
> a) not calling rte_kni_tx_burst() frequent, so that all free mbufs
> consumed.
> b) calling rte_kni_tx_burst() with number of mbufs bigger than
> MAX_MBUF_BURST_NUM, because this function frees at most
> MAX_MBUF_BURST_NUM of mbufs, but if you are calling calling
> rte_kni_tx_burst() with bigger numbers, this will cause mbufs to stuck
> in free_q
> 
> 
> Regards,
> ferruh
> 
> 
> 
> 
> 
> -- 
> Alex Wang,
> Open vSwitch developer

[dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching

2016-05-17 Thread Ananyev, Konstantin

Hi Hiroyuki,

> 
> The rte_pktmbuf_detach() function should decrease refcnt on a direct
> buffer.
> 
> Signed-off-by: Hiroyuki Mikita 
> ---
> v2:
> * introduced a new function rte_pktmbuf_detach2() which decrease refcnt.
> * marked rte_pktmbuf_detach() as deprecated.
> * added comments about refcnt to rte_pktmbuf_attach() and 
> rte_pktmbuf_detach().
> * checked refcnt when detaching in unit tests.
> * added this issue to release notes.
> 
>  app/test/test_mbuf.c   |  9 +--
>  doc/guides/rel_notes/release_16_07.rst | 11 -
>  lib/librte_mbuf/rte_mbuf.h | 43 
> +++---
>  3 files changed, 51 insertions(+), 12 deletions(-)
> 
> diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> index 98ff93a..2bf05eb 100644
> --- a/app/test/test_mbuf.c
> +++ b/app/test/test_mbuf.c
> @@ -438,6 +438,7 @@ test_attach_from_different_pool(void)
>   struct rte_mbuf *clone = NULL;
>   struct rte_mbuf *clone2 = NULL;
>   char *data, *c_data, *c_data2;
> + uint16_t refcnt;
> 
>   /* alloc a mbuf */
>   m = rte_pktmbuf_alloc(pktmbuf_pool);
> @@ -508,13 +509,17 @@ test_attach_from_different_pool(void)
>   GOTO_FAIL("invalid refcnt in m\n");
> 
>   /* detach the clones */
> - rte_pktmbuf_detach(clone);
> + refcnt = rte_pktmbuf_detach2(clone);
>   if (c_data != rte_pktmbuf_mtod(clone, char *))
>   GOTO_FAIL("clone was not detached properly\n");
> + if (refcnt != 2 || rte_mbuf_refcnt_read(m) != 2)
> + GOTO_FAIL("invalid refcnt in m\n");
> 
> - rte_pktmbuf_detach(clone2);
> + refcnt = rte_pktmbuf_detach2(clone2);
>   if (c_data2 != rte_pktmbuf_mtod(clone2, char *))
>   GOTO_FAIL("clone2 was not detached properly\n");
> + if (refcnt != 1 || rte_mbuf_refcnt_read(m) != 1)
> + GOTO_FAIL("invalid refcnt in m\n");
> 
>   /* free the clones and the initial mbuf */
>   rte_pktmbuf_free(clone2);
> diff --git a/doc/guides/rel_notes/release_16_07.rst 
> b/doc/guides/rel_notes/release_16_07.rst
> index f6d543c..9678c1f 100644
> --- a/doc/guides/rel_notes/release_16_07.rst
> +++ b/doc/guides/rel_notes/release_16_07.rst
> @@ -77,13 +77,10 @@ Other
>  Known Issues
>  
> 
> -This section should contain new known issues in this release. Sample format:
> -
> -* **Add title in present tense with full stop.**
> -
> -  Add a short 1-2 sentence description of the known issue in the present
> -  tense. Add information on any known workarounds.
> -
> +* The ``rte_pktmbuf_detach()`` function does not decrement the direct
> +  mbuf's reference counter. It leads a memory leak of the direct
> +  mbuf. The workaround is to explicitly decrement the reference
> +  counter or use ``rte_pktmbuf_detach2()``.
> 
>  API Changes
>  ---
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> index 529debb..c0a592d 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -1408,6 +1408,7 @@ static inline int rte_pktmbuf_alloc_bulk(struct 
> rte_mempool *pool,
>   *
>   * After attachment we refer the mbuf we attached as 'indirect',
>   * while mbuf we attached to as 'direct'.
> + * The direct mbuf's reference counter is incremented.
>   * Right now, not supported:
>   *  - attachment for already indirect mbuf (e.g. - mi has to be direct).
>   *  - mbuf we trying to attach (mi) is used by someone else
> @@ -1459,15 +1460,50 @@ static inline void rte_pktmbuf_attach(struct rte_mbuf 
> *mi, struct rte_mbuf *m)
>  /**
>   * Detach an indirect packet mbuf.
>   *
> + * Note: It is deprecated.
> + * The direct mbuf's reference counter is not decremented.
> + *
> + *  - restore original mbuf address and length values.
> + *  - reset pktmbuf data and data_len to their default values.
> + *  All other fields of the given packet mbuf will be left intact.
> + *
> + * @param m
> + *   The indirect attached packet mbuf.
> + */
> +static inline void __rte_deprecated rte_pktmbuf_detach(struct rte_mbuf *m)
> +{
> + struct rte_mempool *mp = m->pool;
> + uint32_t mbuf_size, buf_len, priv_size;
> +
> + priv_size = rte_pktmbuf_priv_size(mp);
> + mbuf_size = sizeof(struct rte_mbuf) + priv_size;
> + buf_len = rte_pktmbuf_data_room_size(mp);
> +
> + m->priv_size = priv_size;
> + m->buf_addr = (char *)m + mbuf_size;
> + m->buf_physaddr = rte_mempool_virt2phy(mp, m) + mbuf_size;
> + m->buf_len = (uint16_t)buf_len;
> + m->data_off = RTE_MIN(RTE_PKTMBUF_HEADROOM, (uint16_t)m->buf_len);
> + m->data_len = 0;
> + m->ol_flags = 0;
> +}

I still think it would be good to have a separate function for what 
rte_pktmbuf_detach()
Is doing right now: restore original values of indirect mbuf.
Probably rename it to rte_pktmbuf_restore() or so and make _detach2() 
(unatach() ?)
to call it internally. 

> +
> +/**
> + * Detach an indirect packet mbuf.
> + *
>   *  - restore original mbuf address and length

[dpdk-dev] [PATCH v2] cmdline: check return value at initialization

2016-05-17 Thread Olivier Matz

From: Marcin Kerlin 

The value returned by rdline_init() was not checked in cmdline_new().
On error, free the allocated memory and return NULL.

This condition should not happen today, but it's safer to do the check
in case rdline_init() is updated.

Fixes: af75078fece3 ("first public release")
Coverity ID 13204

Signed-off-by: Marcin Kerlin 
Signed-off-by: Olivier Matz 
---

Hi Marcin,

I updated the commit log and title to be clearer.

Regards,
Olivier


v1 -> v2
  rework title and commit log

 lib/librte_cmdline/cmdline.c | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/lib/librte_cmdline/cmdline.c b/lib/librte_cmdline/cmdline.c
index c405878..a9c47be 100644
--- a/lib/librte_cmdline/cmdline.c
+++ b/lib/librte_cmdline/cmdline.c
@@ -130,6 +130,7 @@ struct cmdline *
 cmdline_new(cmdline_parse_ctx_t *ctx, const char *prompt, int s_in, int s_out)
 {
struct cmdline *cl;
+   int ret;

if (!ctx || !prompt)
return NULL;
@@ -142,8 +143,13 @@ cmdline_new(cmdline_parse_ctx_t *ctx, const char *prompt, 
int s_in, int s_out)
cl->s_out = s_out;
cl->ctx = ctx;

-   rdline_init(>rdl, cmdline_write_char,
-   cmdline_valid_buffer, cmdline_complete_buffer);
+   ret = rdline_init(>rdl, cmdline_write_char, cmdline_valid_buffer,
+   cmdline_complete_buffer);
+   if (ret != 0) {
+   free(cl);
+   return NULL;
+   }
+
cl->rdl.opaque = cl;
cmdline_set_prompt(cl, prompt);
rdline_newline(>rdl, cl->prompt);
--
2.8.0.rc3

[dpdk-dev] test-pmd: Free of address-of expression

2016-05-17 Thread Olivier MATZ

Hi,


On 05/17/2016 09:55 AM, Mrozowicz, SlawomirX wrote:
> Hi,
>
> Noticed is that in the file:
>
> app/test-pmd/mempool.c
>
> using of the function munmap() could cause a problem.
>
> Coverity static code analyzer provide error (CID 13184) in line 158:
>
> munmap frees incorrect pointer uv.
>
> I noticed information on the net:
>
> https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.bpxbd00/mumap.htm
>
> ?If addr is not the address of a mapping established by a prior call to
> mmap(), the behavior is undefined?
>
> I have analyzed the code and I have done some test with gcc.
>
> It seems that it is possible to free subrange of the mapping memory.
>
> In the mempool.c code the address is calculated independently.
>
> Anyway in my opinion the address variable uv is calculated correctly.
>
> So we should classify this issue as a  False Positive.
>
> Please accept the conclusion.

This file (app/test-pmd/mempool_anon.c) will be removed soon.
Please see http://dpdk.org/dev/patchwork/patch/12067/

So I think we can ignore this alert, as the new code does not
have the same issue.

Regards,
Olivier

[dpdk-dev] possible kni bug and proposed fix

2016-05-17 Thread ALeX Wang

On 17 May 2016 at 03:07, Ferruh Yigit  wrote:

> On 5/16/2016 4:31 PM, ALeX Wang wrote:
> > Hi Ferruh,
> >
> > Thx for pointing out the 'fill alloc_q with these mubfs _until it gets
> > full_.',
> >
> > I saw the size of all queues are 'KNI_FIFO_COUNT_MAX (1024)'...
> > The corresponding memory required is more than what I specify as
> > 'socket_mem' (since i'm using VM)...
> >
>
> If the mempool is not big enough to fill the ring, this explains the
> error log. Logic is to fill the alloc_q, but if you are missing the
> required mbufs, in each rte_kni_rx_burst() it will complain about memory.
>
> But the required memory for mbufs to fill the ring is not too much. It
> should be ~2Mbytes, are you sure you are missing this much memory?
>
>

Thx for reminding me about this, by default, i only allocate 2048 mbufs for
my
pool...  once i raise it to 4096, the issue is gone...  will the
`KNI_FIFO_COUNT_MAX `
ever change?  I want to try adding some documentation,...


> > Also, in my use case, I only `tcpreplay` through the kni interface, and
> > my application only do rx and then free the 'mbufs'.  So there is no tx
> > at all.
> >
> > So, in my case, I still think this is a bug/defect, or somewhere i still
> > misunderstand,
> >
> > P.S. The description here seems to be inverted,
> >
> http://dpdk.org/doc/api/rte__kni_8h.html#a0cdd727cdc227d005fef22c0189f3dfe
> > 'rte_kni_rx_burst' does the 'It handles allocating the mbufs for KNI
> > interface alloc queue.'
> >
>
> You are right. That part of the description for rte_kni_rx_burst and
> rte_kni_tx_burst needs to be replaced. Do you want to send a patch?
>
>
Sure, will do that,

Thanks,
Alex Wang,



> > Thanks,
> > Alex Wang,
> >
> > On 16 May 2016 at 04:20, Ferruh Yigit  > > wrote:
> >
> > On 5/15/2016 5:48 AM, ALeX Wang wrote:
> > > Hi,
> > >
> > > When using the kni module to test my application inside
> > > debian (virtualbox) VM (kernel version 4.4), I get the
> > >
> > > "KNI: Out of memory"
> > >
> > > from syslog every time I `tcpreply` packets through
> > > the kni interface.
> > >
> > > After checking source code, I saw that when I call
> > > 'rte_kni_rx_burst()', no matter how many packets
> > > are actually retrieved, we always call 'kni_allocate_mbufs()'
> > > and try allocate 'MAX_MBUF_BURST_NUM' more
> > > mbufs...  I fix the issue via using this patch below,
> > >
> > > Could you confirm if this is an actual bug?
> > >
> >
> > Hi Alex,
> >
> > I don't think this is a bug.
> >
> > kni_allocate_mbufs() will allocate MAX_MBUF_BURST_NUM mbufs as you
> > mentioned. And will fill alloc_q with these mubfs _until it gets
> full_.
> > And if the alloc_q is full and there are remaining mbufs, they will
> be
> > freed. So for some cases this isn't the most optimized way, but
> there is
> > no defect.
> >
> > Since you are getting "KNI: Out of memory", somewhere else can be
> > missing freeing mbufs.
> >
> > mbufs freeing done in rte_kni_tx_burst(), I can guess two cases that
> can
> > cause problem:
> > a) not calling rte_kni_tx_burst() frequent, so that all free mbufs
> > consumed.
> > b) calling rte_kni_tx_burst() with number of mbufs bigger than
> > MAX_MBUF_BURST_NUM, because this function frees at most
> > MAX_MBUF_BURST_NUM of mbufs, but if you are calling calling
> > rte_kni_tx_burst() with bigger numbers, this will cause mbufs to
> stuck
> > in free_q
> >
> >
> > Regards,
> > ferruh
> >
> >
> >
> >
> >
> > --
> > Alex Wang,
> > Open vSwitch developer
>
>


-- 
Alex Wang,
Open vSwitch developer

[dpdk-dev] [PATCH 3/4] ixgbe: automatic link recovery on VF

2016-05-17 Thread Olivier MATZ

Hi Wenzhuo,

On 05/17/2016 03:11 AM, Lu, Wenzhuo wrote:
>> -Original Message-
>> From: Olivier Matz [mailto:olivier.matz at 6wind.com]
>> If I understand well, ixgbevf_dev_link_up_down_handler() is called by
>> ixgbevf_recv_pkts_fake() on a dataplane core. It means that the core that
>> acquired the lock will loop during 100us + 1sec at least.
>> If this core was also in charge of polling other queues of other ports, or 
>> timers,
>> many packets will be dropped (even with a 100us loop). I don't think it is
>> acceptable to actively wait inside a rx function.
>>
>> I think it would avoid many issues to delegate this work to the application,
>> maybe by notifying it that the port is in a bad state and must be restarted. 
>> The
>> application could then properly stop polling the queues, and stop and 
>> restart the
>> port in a separate thread, without bothering the dataplane cores.
> Thanks for the comments.
> Yes, you're right. I had a wrong assumption that every queue is handled by 
> one core.
> But surely it's not right, we cannot tell how the users will deploy their 
> system.
>
> I plan to update this patch set. The solution now is, first let the users 
> choose if they want this
> auto-reset feature. If so, we will apply another series rx/tx functions which 
> have lock. So we
> can stop the rx/tx of the bad ports.
> And we also apply a reset API for users. The APPs should call this API in 
> their management thread or so.
> It means APPs should guarantee the thread safe for the API.
> You see, there're 2 things,
> 1, Lock the rx/tx to stop them for users.
> 2, Apply a resetting API for users, and every NIC can do their own job. APPs 
> need not to worry about the difference
> between different NICs.
>
> Surely, it's not *automatic* now. The reason is DPDK doesn't guarantee the 
> thread safe. So the operations have to be
> left to the APPs and let them to guarantee the thread safe.
>
> And if the users choose not using auto-reset feature, we will leave this work 
> to the APP :)

Yes, I think having 2 modes is a good approach:

- the first mode would let the application know a reset has to
   be performed, without active loop or lock in the rx/tx funcs.
- the second mode would transparently manage the reset in the driver,
   but may lock the core during some time.

By the way, you talk about a reset API, why not just using the
usual stop/start functions? I think it would work the same.

Regards,
Olivier

[dpdk-dev] [PATCH] Add rte_mempool_free

2016-05-17 Thread Olivier MATZ

Hello Benjamin,

On 05/16/2016 09:56 PM, Walker, Benjamin wrote:
> On Mon, 2016-05-16 at 16:57 +, Wiles, Keith wrote:
>>>
>>> There is no inverse of rte_mempool_create, so this patch adds one.
>>> The typical usage of rte_mempool_create is to create a pool at
>>> initialization time and only to free it upon program exit, so an
>>> rte_mempool_free function at first seems to be of little value.
>>> However, it is very useful as a sanity check for a clean shutdown
>>> when used in conjunction with tools like AddressSanitizer. Further,
>>> the call itself verifies that all elements have been returned to
>>> the pool or it fails.
>>>
>>> Signed-off-by: Ben Walker 

I already submitted a patch introducing a similar feature.
Please see:
http://dpdk.org/dev/patchwork/patch/12062/

It is part of a larger patchset:
http://dpdk.org/ml/archives/dev/2016-April/037464.html


Regards,
Olivier

[dpdk-dev] [PATCH v2] app/test: fix log check when default level is high

2016-05-17 Thread Thomas Monjalon

The log unit test was checking display of low priority messages.
It was not working if RTE_LOG_LEVEL is not RTE_LOG_DEBUG.
It is even easier to see since the default level is INFO (9b9d7ca).

Now the test use ERR and CRIT levels which should be always enabled
while not trigerring syslog output on the console.

Signed-off-by: Thomas Monjalon 
---
v2:
- fix comments for the log levels
- add comments to explain the test

 app/test/autotest_test_funcs.py | 21 +
 app/test/test_logs.c| 27 +--
 2 files changed, 22 insertions(+), 26 deletions(-)

diff --git a/app/test/autotest_test_funcs.py b/app/test/autotest_test_funcs.py
index 0f012f6..adbd8c9 100644
--- a/app/test/autotest_test_funcs.py
+++ b/app/test/autotest_test_funcs.py
@@ -144,19 +144,16 @@ def logs_autotest(child, test_name):
i = 0
child.sendline(test_name)

+   # logs sequence is printed twice because of history dump
log_list = [
-   "TESTAPP1: this is a debug level message",
-   "TESTAPP1: this is a info level message",
-   "TESTAPP1: this is a warning level message",
-   "TESTAPP2: this is a info level message",
-   "TESTAPP2: this is a warning level message",
-   "TESTAPP1: this is a debug level message",
-   "TESTAPP1: this is a debug level message",
-   "TESTAPP1: this is a info level message",
-   "TESTAPP1: this is a warning level message",
-   "TESTAPP2: this is a info level message",
-   "TESTAPP2: this is a warning level message",
-   "TESTAPP1: this is a debug level message",
+   "TESTAPP1: error message",
+   "TESTAPP1: critical message",
+   "TESTAPP2: critical message",
+   "TESTAPP1: error message",
+   "TESTAPP1: error message",
+   "TESTAPP1: critical message",
+   "TESTAPP2: critical message",
+   "TESTAPP1: error message",
]

for log_msg in log_list:
diff --git a/app/test/test_logs.c b/app/test/test_logs.c
index 18a3b6a..05aa862 100644
--- a/app/test/test_logs.c
+++ b/app/test/test_logs.c
@@ -65,26 +65,25 @@ test_logs(void)
rte_set_log_type(RTE_LOGTYPE_TESTAPP1, 1);
rte_set_log_type(RTE_LOGTYPE_TESTAPP2, 1);

-   /* log in debug level */
-   rte_set_log_level(RTE_LOG_DEBUG);
-   RTE_LOG(DEBUG, TESTAPP1, "this is a debug level message\n");
-   RTE_LOG(INFO, TESTAPP1, "this is a info level message\n");
-   RTE_LOG(WARNING, TESTAPP1, "this is a warning level message\n");
+   /* log in error level */
+   rte_set_log_level(RTE_LOG_ERR);
+   RTE_LOG(ERR, TESTAPP1, "error message\n");
+   RTE_LOG(CRIT, TESTAPP1, "critical message\n");

-   /* log in info level */
-   rte_set_log_level(RTE_LOG_INFO);
-   RTE_LOG(DEBUG, TESTAPP2, "debug level message (not displayed)\n");
-   RTE_LOG(INFO, TESTAPP2, "this is a info level message\n");
-   RTE_LOG(WARNING, TESTAPP2, "this is a warning level message\n");
+   /* log in critical level */
+   rte_set_log_level(RTE_LOG_CRIT);
+   RTE_LOG(ERR, TESTAPP2, "error message (not displayed)\n");
+   RTE_LOG(CRIT, TESTAPP2, "critical message\n");

/* disable one log type */
rte_set_log_type(RTE_LOGTYPE_TESTAPP2, 0);

-   /* log in debug level */
-   rte_set_log_level(RTE_LOG_DEBUG);
-   RTE_LOG(DEBUG, TESTAPP1, "this is a debug level message\n");
-   RTE_LOG(DEBUG, TESTAPP2, "debug level message (not displayed)\n");
+   /* log in error level */
+   rte_set_log_level(RTE_LOG_ERR);
+   RTE_LOG(ERR, TESTAPP1, "error message\n");
+   RTE_LOG(ERR, TESTAPP2, "error message (not displayed)\n");

+   /* print again the previous logs */
rte_log_dump_history(stdout);

return 0;
-- 
2.7.0

[dpdk-dev] [PATCH] app/test: fix log check when default level is high

2016-05-17 Thread David Marchand

Hello Thomas,

On Wed, May 4, 2016 at 3:34 PM, Thomas Monjalon
 wrote:
> The log unit test was checking display of low priority messages.
> It was not working if RTE_LOG_LEVEL is not RTE_LOG_DEBUG.
> It is even easier to see since the default level is INFO (9b9d7ca).
>
> Now the test use ERR and CRIT levels which should be always enabled
> while not trigerring syslog output on the console.
>
> Signed-off-by: Thomas Monjalon 
> ---
>  app/test/autotest_test_funcs.py | 20 
>  app/test/test_logs.c| 20 +---
>  2 files changed, 17 insertions(+), 23 deletions(-)
>
> diff --git a/app/test/autotest_test_funcs.py b/app/test/autotest_test_funcs.py
> index 0f012f6..f04909d 100644
> --- a/app/test/autotest_test_funcs.py
> +++ b/app/test/autotest_test_funcs.py
> @@ -145,18 +145,14 @@ def logs_autotest(child, test_name):
> child.sendline(test_name)
>
> log_list = [
> -   "TESTAPP1: this is a debug level message",
> -   "TESTAPP1: this is a info level message",
> -   "TESTAPP1: this is a warning level message",
> -   "TESTAPP2: this is a info level message",
> -   "TESTAPP2: this is a warning level message",
> -   "TESTAPP1: this is a debug level message",
> -   "TESTAPP1: this is a debug level message",
> -   "TESTAPP1: this is a info level message",
> -   "TESTAPP1: this is a warning level message",
> -   "TESTAPP2: this is a info level message",
> -   "TESTAPP2: this is a warning level message",
> -   "TESTAPP1: this is a debug level message",
> +   "TESTAPP1: error message",
> +   "TESTAPP1: critical message",
> +   "TESTAPP2: critical message",
> +   "TESTAPP1: error message",
> +   "TESTAPP1: error message",
> +   "TESTAPP1: critical message",
> +   "TESTAPP2: critical message",
> +   "TESTAPP1: error message",

Since this list is tighly coupled to test_logs.c, a little comment on
why the messages are declared twice in this list would help.

> ]
>
> for log_msg in log_list:
> diff --git a/app/test/test_logs.c b/app/test/test_logs.c
> index 18a3b6a..861cdff 100644
> --- a/app/test/test_logs.c
> +++ b/app/test/test_logs.c
> @@ -66,24 +66,22 @@ test_logs(void)
> rte_set_log_type(RTE_LOGTYPE_TESTAPP2, 1);
>
> /* log in debug level */
> -   rte_set_log_level(RTE_LOG_DEBUG);
> -   RTE_LOG(DEBUG, TESTAPP1, "this is a debug level message\n");
> -   RTE_LOG(INFO, TESTAPP1, "this is a info level message\n");
> -   RTE_LOG(WARNING, TESTAPP1, "this is a warning level message\n");
> +   rte_set_log_level(RTE_LOG_ERR);
> +   RTE_LOG(ERR, TESTAPP1, "error message\n");
> +   RTE_LOG(CRIT, TESTAPP1, "critical message\n");

Comment is wrong, should be about error level.

>
> /* log in info level */
> -   rte_set_log_level(RTE_LOG_INFO);
> -   RTE_LOG(DEBUG, TESTAPP2, "debug level message (not displayed)\n");
> -   RTE_LOG(INFO, TESTAPP2, "this is a info level message\n");
> -   RTE_LOG(WARNING, TESTAPP2, "this is a warning level message\n");
> +   rte_set_log_level(RTE_LOG_CRIT);
> +   RTE_LOG(ERR, TESTAPP2, "error message (not displayed)\n");
> +   RTE_LOG(CRIT, TESTAPP2, "critical message\n");

Idem.

>
> /* disable one log type */
> rte_set_log_type(RTE_LOGTYPE_TESTAPP2, 0);
>
> /* log in debug level */
> -   rte_set_log_level(RTE_LOG_DEBUG);
> -   RTE_LOG(DEBUG, TESTAPP1, "this is a debug level message\n");
> -   RTE_LOG(DEBUG, TESTAPP2, "debug level message (not displayed)\n");
> +   rte_set_log_level(RTE_LOG_ERR);
> +   RTE_LOG(ERR, TESTAPP1, "error message\n");
> +   RTE_LOG(ERR, TESTAPP2, "error message (not displayed)\n");

Idem.


-- 
David Marchand

[dpdk-dev] [PATCH] mbuf: new public function to allocate raw mbufs

2016-05-17 Thread Thomas Monjalon

2016-05-11 16:43, Olivier Matz:
> Many drivers provide their own implementation of rte_mbuf_raw_alloc(),
> duplicating the code. Introduce a new public function in rte_mbuf to
> allocate a raw mbuf (uninitialized).
> 
> Signed-off-by: Olivier Matz 

Applied, thanks

[dpdk-dev] [PATCH 3/4] ixgbe: automatic link recovery on VF

2016-05-17 Thread Lu, Wenzhuo

Hi Olivier,

> -Original Message-
> From: Olivier MATZ [mailto:olivier.matz at 6wind.com]
> Sent: Tuesday, May 17, 2016 3:51 PM
> To: Lu, Wenzhuo; dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 3/4] ixgbe: automatic link recovery on VF
> 
> Hi Wenzhuo,
> 
> On 05/17/2016 03:11 AM, Lu, Wenzhuo wrote:
> >> -Original Message-
> >> From: Olivier Matz [mailto:olivier.matz at 6wind.com] If I understand
> >> well, ixgbevf_dev_link_up_down_handler() is called by
> >> ixgbevf_recv_pkts_fake() on a dataplane core. It means that the core
> >> that acquired the lock will loop during 100us + 1sec at least.
> >> If this core was also in charge of polling other queues of other
> >> ports, or timers, many packets will be dropped (even with a 100us
> >> loop). I don't think it is acceptable to actively wait inside a rx 
> >> function.
> >>
> >> I think it would avoid many issues to delegate this work to the
> >> application, maybe by notifying it that the port is in a bad state
> >> and must be restarted. The application could then properly stop
> >> polling the queues, and stop and restart the port in a separate thread,
> without bothering the dataplane cores.
> > Thanks for the comments.
> > Yes, you're right. I had a wrong assumption that every queue is handled by 
> > one
> core.
> > But surely it's not right, we cannot tell how the users will deploy their 
> > system.
> >
> > I plan to update this patch set. The solution now is, first let the
> > users choose if they want this auto-reset feature. If so, we will
> > apply another series rx/tx functions which have lock. So we can stop the 
> > rx/tx
> of the bad ports.
> > And we also apply a reset API for users. The APPs should call this API in 
> > their
> management thread or so.
> > It means APPs should guarantee the thread safe for the API.
> > You see, there're 2 things,
> > 1, Lock the rx/tx to stop them for users.
> > 2, Apply a resetting API for users, and every NIC can do their own
> > job. APPs need not to worry about the difference between different NICs.
> >
> > Surely, it's not *automatic* now. The reason is DPDK doesn't guarantee
> > the thread safe. So the operations have to be left to the APPs and let them 
> > to
> guarantee the thread safe.
> >
> > And if the users choose not using auto-reset feature, we will leave
> > this work to the APP :)
> 
> Yes, I think having 2 modes is a good approach:
> 
> - the first mode would let the application know a reset has to
>be performed, without active loop or lock in the rx/tx funcs.
> - the second mode would transparently manage the reset in the driver,
>but may lock the core during some time.
For the second mode, at first we want to let the driver manage the reset 
transparently. But the bad news is
in driver layer the operations is not thread safe. If we want the reset to be 
transparent,
we need a whole new mechanism to guarantee the thread safe for the operations 
in driver layer.
Obviously, it need to be discussed and cannot be finished in this release.
So now we write a reset API for APP, and let APP call this API and guarantee 
the thread safe for all the operations.
It's not transparent. But seems it's what we can do at this stage.

> 
> By the way, you talk about a reset API, why not just using the usual 
> stop/start
> functions? I think it would work the same.
For ixgbe/igb, stop/start is enough. But for i40e, some other work should be 
done. (For example, the resource of the queues should be re-init.)
So we think about introducing a new API, then different NICs can do what they 
have to do.

> 
> Regards,
> Olivier

[dpdk-dev] [PATCH] Add rte_mempool_free

2016-05-17 Thread Simon Kågström

Thanks for adding this, I've been missing this function!

On 2016-05-16 21:56, Walker, Benjamin wrote:
> On Mon, 2016-05-16 at 16:57 +, Wiles, Keith wrote:

>> The big question is how do you know the mempool is not being used someplace?
> 
> That's the user's responsibility. Use after free is certainly possible if the 
> user doesn't take
> care, just like any alloc/free in C. This is the same situation as 
> rte_ring_free or
> rte_memzone_free. To help prevent users from shooting themselves in the foot 
> I did add a check that
> all of the elements have been freed back to the pool at the top of the 
> function. There are certainly
> potential race conditions if the user is freeing this on one thread and using 
> it from another that I
> haven't handled. I'm not sure these cases need to be handled though - they're 
> not handled by
> rte_ring_free, for example.

Also, the user can use rte_mempool_full() to see if there are entries
still allocated from it.

And perhaps rte_mempool_free() should at least check if the pool is full
before releasing it and warn or panic.

// Simon

[dpdk-dev] test-pmd: Free of address-of expression

2016-05-17 Thread Mrozowicz, SlawomirX

Hi,

Noticed is that in the file:
app/test-pmd/mempool.c
using of the function munmap() could cause a problem.

Coverity static code analyzer provide error (CID 13184) in line 158:
munmap frees incorrect pointer uv.

I noticed information on the net:
https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.bpxbd00/mumap.htm
"If addr is not the address of a mapping established by a prior call to mmap(), 
the behavior is undefined"

I have analyzed the code and I have done some test with gcc.
It seems that it is possible to free subrange of the mapping memory.
In the mempool.c code the address is calculated independently.
Anyway in my opinion the address variable uv is calculated correctly.

So we should classify this issue as a  False Positive.
Please accept the conclusion.

Slawomir Mrozowicz
Sii Engineer
Delivering outsourced services to Intel
Intel Technology Poland sp. z o.o. - KRS 101882 - ul. Slowackiego 173, 80-298 
Gdansk

[dpdk-dev] [PATCH] doc: move rel_notes instructions as comments

2016-05-17 Thread Thomas Monjalon

> > We don't want to have this instructions in the generated docs, so use
> > comments. It's also less confusing for people adding entries in the
> > documentation.
> > 
> > Signed-off-by: Olivier Matz 
> 
> Good idea.
> 
> Reviewed-by: John McNamara 
> Acked-by: John McNamara 

Applied, thanks

[dpdk-dev] [PATCH v5] mempool: reduce rte_mempool structure size

2016-05-17 Thread Thomas Monjalon

2016-04-14 11:42, Olivier Matz:
> From: Keith Wiles 
> 
> The rte_mempool structure is changed, which will cause an ABI change
> for this structure. Providing backward compat is not reasonable
> here as this structure is used in multiple defines/inlines.

The deprecation notice must be removed by this patch.

[...]
> +/**
>   * Calculate the size of the mempool header.
>   *
>   * @param mp
> @@ -254,9 +256,9 @@ struct rte_mempool {
>   * @param pgn
>   *   Number of pages used to store mempool objects.

A new parameter has been forgotten:
 * @param cs
 *   Size of the per-lcore cache.   
 

>   */
> -#define  MEMPOOL_HEADER_SIZE(mp, pgn)(sizeof(*(mp)) + \
> - RTE_ALIGN_CEIL(((pgn) - RTE_DIM((mp)->elt_pa)) * \
> - sizeof ((mp)->elt_pa[0]), RTE_CACHE_LINE_SIZE))
> +#define MEMPOOL_HEADER_SIZE(mp, pgn, cs) \
> + (sizeof(*(mp)) + __PA_SIZE(mp, pgn) + (((cs) == 0) ? 0 : \
> + (sizeof(struct rte_mempool_cache) * RTE_MAX_LCORE)))

Applied with above changes

[dpdk-dev] FW: [PATCH v1 0/3] Keep-alive enhancements

2016-05-17 Thread Liu, Shucong


Shucong:
The validation for patches has been done.

-Original Message-
From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Horton, Remy
Sent: Friday, April 29, 2016 1:41 PM
To: dev at dpdk.org
Subject: [dpdk-dev] [PATCH v1 0/3] Keep-alive enhancements

This patchset adds enhancements to the keepalive core monitoring and reporting 
sub-system. The first is support for idled (sleeping and
frequency-stepped) CPU cores, and the second is support for applications to be 
notified of active as well as faulted cores. The latter is to allow core state 
to be relayed to external (secondary) processes, which is demonstrated by 
changes to the l2fed-keepalive example.

Remy Horton (3):
  eal: add new keepalive state & callback hook
  examples/l2fwd-keepalive: add IPC liveness reporting
  doc: add keepalive enhancement documentation

 doc/guides/rel_notes/release_16_07.rst  |   5 +
 examples/Makefile   |   1 +
 examples/l2fwd-keepalive/Makefile   |   4 +-
 examples/l2fwd-keepalive/ka-agent/Makefile  |  51 ++
 examples/l2fwd-keepalive/ka-agent/main.c| 128 +++
 examples/l2fwd-keepalive/main.c |  22 +++-
 examples/l2fwd-keepalive/shm.c  | 130 
 examples/l2fwd-keepalive/shm.h  | 102 +++
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |   7 ++
 lib/librte_eal/common/include/rte_keepalive.h   |  40 
 lib/librte_eal/common/rte_keepalive.c   |  35 ++-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |   7 ++
 12 files changed, 526 insertions(+), 6 deletions(-)  create mode 100644 
examples/l2fwd-keepalive/ka-agent/Makefile
 create mode 100644 examples/l2fwd-keepalive/ka-agent/main.c
 create mode 100644 examples/l2fwd-keepalive/shm.c  create mode 100644 
examples/l2fwd-keepalive/shm.h

--
2.5.5

[dpdk-dev] [PATCH v2] mbuf: decrease refcnt when detaching

2016-05-17 Thread Hiroyuki Mikita

The rte_pktmbuf_detach() function should decrease refcnt on a direct
buffer.

Signed-off-by: Hiroyuki Mikita 
---
v2:
* introduced a new function rte_pktmbuf_detach2() which decrease refcnt.
* marked rte_pktmbuf_detach() as deprecated.
* added comments about refcnt to rte_pktmbuf_attach() and rte_pktmbuf_detach().
* checked refcnt when detaching in unit tests.
* added this issue to release notes.

 app/test/test_mbuf.c   |  9 +--
 doc/guides/rel_notes/release_16_07.rst | 11 -
 lib/librte_mbuf/rte_mbuf.h | 43 +++---
 3 files changed, 51 insertions(+), 12 deletions(-)

diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
index 98ff93a..2bf05eb 100644
--- a/app/test/test_mbuf.c
+++ b/app/test/test_mbuf.c
@@ -438,6 +438,7 @@ test_attach_from_different_pool(void)
struct rte_mbuf *clone = NULL;
struct rte_mbuf *clone2 = NULL;
char *data, *c_data, *c_data2;
+   uint16_t refcnt;

/* alloc a mbuf */
m = rte_pktmbuf_alloc(pktmbuf_pool);
@@ -508,13 +509,17 @@ test_attach_from_different_pool(void)
GOTO_FAIL("invalid refcnt in m\n");

/* detach the clones */
-   rte_pktmbuf_detach(clone);
+   refcnt = rte_pktmbuf_detach2(clone);
if (c_data != rte_pktmbuf_mtod(clone, char *))
GOTO_FAIL("clone was not detached properly\n");
+   if (refcnt != 2 || rte_mbuf_refcnt_read(m) != 2)
+   GOTO_FAIL("invalid refcnt in m\n");

-   rte_pktmbuf_detach(clone2);
+   refcnt = rte_pktmbuf_detach2(clone2);
if (c_data2 != rte_pktmbuf_mtod(clone2, char *))
GOTO_FAIL("clone2 was not detached properly\n");
+   if (refcnt != 1 || rte_mbuf_refcnt_read(m) != 1)
+   GOTO_FAIL("invalid refcnt in m\n");

/* free the clones and the initial mbuf */
rte_pktmbuf_free(clone2);
diff --git a/doc/guides/rel_notes/release_16_07.rst 
b/doc/guides/rel_notes/release_16_07.rst
index f6d543c..9678c1f 100644
--- a/doc/guides/rel_notes/release_16_07.rst
+++ b/doc/guides/rel_notes/release_16_07.rst
@@ -77,13 +77,10 @@ Other
 Known Issues
 

-This section should contain new known issues in this release. Sample format:
-
-* **Add title in present tense with full stop.**
-
-  Add a short 1-2 sentence description of the known issue in the present
-  tense. Add information on any known workarounds.
-
+* The ``rte_pktmbuf_detach()`` function does not decrement the direct
+  mbuf's reference counter. It leads a memory leak of the direct
+  mbuf. The workaround is to explicitly decrement the reference
+  counter or use ``rte_pktmbuf_detach2()``.

 API Changes
 ---
diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
index 529debb..c0a592d 100644
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h
@@ -1408,6 +1408,7 @@ static inline int rte_pktmbuf_alloc_bulk(struct 
rte_mempool *pool,
  *
  * After attachment we refer the mbuf we attached as 'indirect',
  * while mbuf we attached to as 'direct'.
+ * The direct mbuf's reference counter is incremented.
  * Right now, not supported:
  *  - attachment for already indirect mbuf (e.g. - mi has to be direct).
  *  - mbuf we trying to attach (mi) is used by someone else
@@ -1459,15 +1460,50 @@ static inline void rte_pktmbuf_attach(struct rte_mbuf 
*mi, struct rte_mbuf *m)
 /**
  * Detach an indirect packet mbuf.
  *
+ * Note: It is deprecated.
+ * The direct mbuf's reference counter is not decremented.
+ *
+ *  - restore original mbuf address and length values.
+ *  - reset pktmbuf data and data_len to their default values.
+ *  All other fields of the given packet mbuf will be left intact.
+ *
+ * @param m
+ *   The indirect attached packet mbuf.
+ */
+static inline void __rte_deprecated rte_pktmbuf_detach(struct rte_mbuf *m)
+{
+   struct rte_mempool *mp = m->pool;
+   uint32_t mbuf_size, buf_len, priv_size;
+
+   priv_size = rte_pktmbuf_priv_size(mp);
+   mbuf_size = sizeof(struct rte_mbuf) + priv_size;
+   buf_len = rte_pktmbuf_data_room_size(mp);
+
+   m->priv_size = priv_size;
+   m->buf_addr = (char *)m + mbuf_size;
+   m->buf_physaddr = rte_mempool_virt2phy(mp, m) + mbuf_size;
+   m->buf_len = (uint16_t)buf_len;
+   m->data_off = RTE_MIN(RTE_PKTMBUF_HEADROOM, (uint16_t)m->buf_len);
+   m->data_len = 0;
+   m->ol_flags = 0;
+}
+
+/**
+ * Detach an indirect packet mbuf.
+ *
  *  - restore original mbuf address and length values.
  *  - reset pktmbuf data and data_len to their default values.
  *  All other fields of the given packet mbuf will be left intact.
+ *  - decrement the direct mbuf's reference counter.
  *
  * @param m
  *   The indirect attached packet mbuf.
+ * @return
+ *   The updated value of the direct mbuf's reference counter.
  */
-static inline void rte_pktmbuf_detach(struct rte_mbuf *m)
+static inline uint16_t rte_pktmbuf_detach2(struct rte_mbuf *m)
 {
+

[dpdk-dev] [PATCH] mbuf: decrease refcnt when detaching

2016-05-17 Thread Hiroyuki MIKITA

Hi all,

Thanks for suggestions.
I think the Oliver's first option is good.
I introduce the new function and will replace rte_pktmbuf_detach()
with it in a future release.


2016-05-16 18:13 GMT+09:00 Thomas Monjalon :
> 2016-05-16 11:46, Hiroyuki MIKITA:
>> Now, the attach operation increases refcnt, but the detach does not decrease 
>> it.
>> I think both operations should affect refcnt or not.
>> Which design is intended?
>>
>> In "6.7. Direct and Indirect Buffers" of Programmer's Guide,
>> it is mentioned that "...whenever an indirect buffer is attached to
>> the direct buffer,
>> the reference counter on the direct buffer is incremented.
>> Similarly, whenever the indirect buffer is detached,
>> the reference counter on the direct buffer is decremented."
>
> The doc is the reference. The doxygen comment should explicit every
> details of the behaviour.
> And the unit tests must validate every parts of the behaviour.
> Probably there is a bug which is not (yet) tested.
> Please see the function testclone_testupdate_testdetach. Thanks
>

[dpdk-dev] [PATCH 3/4] ixgbe: automatic link recovery on VF

2016-05-17 Thread Lu, Wenzhuo

Hi Olivier,

> -Original Message-
> From: Olivier Matz [mailto:olivier.matz at 6wind.com]
> Sent: Monday, May 16, 2016 8:01 PM
> To: Lu, Wenzhuo; dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 3/4] ixgbe: automatic link recovery on VF
> 
> Hi Wenzhuo,
> 
> On 05/04/2016 11:10 PM, Wenzhuo Lu wrote:
> > When the physical link is down and recover later, the VF link cannot
> > recover until the user stop and start it manually.
> > This patch implements the automatic recovery of VF port.
> > The automatic recovery bases on the link up/down message received from
> > PF. When VF receives the link up/down message, it will replace the
> > RX/TX and operation functions with fake ones to stop RX/TX and any
> > future operation. Then reset the VF port.
> > After successfully resetting the port, recover the RX/TX and operation
> > functions.
> >
> > Signed-off-by: Wenzhuo Lu 
> >
> > [...]
> >
> > +void
> > +ixgbevf_dev_link_up_down_handler(struct rte_eth_dev *dev) {
> > +   struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
> > +   struct ixgbe_adapter *adapter =
> > +   (struct ixgbe_adapter *)dev->data->dev_private;
> > +   int diag;
> > +   uint32_t vteiam;
> > +
> > +   /* Only one working core need to performance VF reset */
> > +   if (rte_spinlock_trylock(>vf_reset_lock)) {
> > +   /**
> > +* When fake rec/xmit is replaced, working thread may is
> running
> > +* into real RX/TX func, so wait long enough to assume all
> > +* working thread exit. The assumption is it will spend less
> > +* than 100us for each execution of RX and TX func.
> > +*/
> > +   rte_delay_us(100);
> > +
> > +   do {
> > +   dev->data->dev_started = 0;
> > +   ixgbevf_dev_stop(dev);
> > +   rte_delay_us(100);
> 
> If I understand well, ixgbevf_dev_link_up_down_handler() is called by
> ixgbevf_recv_pkts_fake() on a dataplane core. It means that the core that
> acquired the lock will loop during 100us + 1sec at least.
> If this core was also in charge of polling other queues of other ports, or 
> timers,
> many packets will be dropped (even with a 100us loop). I don't think it is
> acceptable to actively wait inside a rx function.
> 
> I think it would avoid many issues to delegate this work to the application,
> maybe by notifying it that the port is in a bad state and must be restarted. 
> The
> application could then properly stop polling the queues, and stop and restart 
> the
> port in a separate thread, without bothering the dataplane cores.
Thanks for the comments.
Yes, you're right. I had a wrong assumption that every queue is handled by one 
core.
But surely it's not right, we cannot tell how the users will deploy their 
system.

I plan to update this patch set. The solution now is, first let the users 
choose if they want this
auto-reset feature. If so, we will apply another series rx/tx functions which 
have lock. So we
can stop the rx/tx of the bad ports.
And we also apply a reset API for users. The APPs should call this API in their 
management thread or so.
It means APPs should guarantee the thread safe for the API.
You see, there're 2 things,
1, Lock the rx/tx to stop them for users.
2, Apply a resetting API for users, and every NIC can do their own job. APPs 
need not to worry about the difference 
between different NICs.

Surely, it's not *automatic* now. The reason is DPDK doesn't guarantee the 
thread safe. So the operations have to be
left to the APPs and let them to guarantee the thread safe.

And if the users choose not using auto-reset feature, we will leave this work 
to the APP :)

> 
> 
> Regards,
> Olivier

71 matches

Mail list logo