Re: [dpdk-users] kni compile failed with Linux-3.10.0-514.6.1.el7.x86_64

[Thomas Monjalon]

2017-02-08 11:52, Cao Tong:
> hi,
> 
> KNI compile failed with kernel "Linux-3.10.0-514.6.1.el7.x86_64" under 
> CentOS 7.
> 
> DPDK version is 'dpdk-stable-16.07.2'

In order to build with recent kernel versions, you must use a recent DPDK 
version.

Please try 16.11 or master.

Re: [dpdk-users] [dpdk-dev] Problem running dpdk-qat example in SR-IOV mode

[Thomas Monjalon]

2017-02-20 11:53, Trahe, Fiona:
> From: Pankaj Joshi
> > 
> > Hello All,
> > 
> > I am trying to run dpdk-qat example code in SR-IOV mode. Steps done by me
> > so far are :
> 
> The dpdk-qat example app is an earlier mechanism of wrapping a QAT driver 
> into DPDK - it's not using the cryptodev API, and depends on both a kernel 
> driver and user-space libs from a QAT package.
> Unless you have a really good reason for using this (like you wanted to use 
> the older Cave Creek device) I would recommend you use the cryptodev API. 

Could we remove this obsolete example?

Re: [dpdk-users] [dpdk-dev] Problem running dpdk-qat example in SR-IOV mode

[Thomas Monjalon]

2017-02-23 16:18, De Lara Guarch, Pablo:
> > From: Thomas Monjalon
> > 2017-02-20 11:53, Trahe, Fiona:
> > > The dpdk-qat example app is an earlier mechanism of wrapping a QAT
> > driver into DPDK - it's not using the cryptodev API, and depends on both a
> > kernel driver and user-space libs from a QAT package.
> > > Unless you have a really good reason for using this (like you wanted to
> > use the older Cave Creek device) I would recommend you use the cryptodev
> > API.
> > 
> > Could we remove this obsolete example?
> 
> This is a good point. How can we proceed with this? As it is a sample 
> application,
> does it require a deprecation notice?

I don't think so.
You just have to send a patch.
Thanks

Re: [dpdk-users] Is Elastic Flow Distributor thread-safe?

[Thomas Monjalon]

2017-02-14 14:14, De Lara Guarch, Pablo:
> From: Rob
> >   Playing around with EFD and seeing some really bizarre behavior in a
> > multi-core environment. Is it safe for multiple logical cores to be making
> > EFD table updates while another logical core is performing lookups? Side
> > note: the core which is making lookups is where the offline table resides.
> > 
> > Intuition tells me this is probably not safe, but the docs make no mention
> > of thread safety.
> 
> The library is not thread-safe. The only operation that is thread safe is the 
> lookup.
> 
> However, although update is not thread safe, it is interesting to know the 
> following.
> Updating the EFD table consists of two steps:
> 
> 1 - Compute Update: the new key is added to the offline group, which is not 
> thread safe.
>  Then, a new perfect hash for the group is computed (thread safe and 
> where most of time is spent).
> 
> 2 - Apply Update: Once the update is computed, it has to be applied to the 
> online table (fast, but not thread safe).
> 
> Multi-writer support might be added in the future, as it was added in the 
> Hash library.
> 
> For now, several readers can work simultaneously, as long as there is no 
> writer working on it (you would need locks for this),
> and only a single writer is supported if there is no other core doing lookups.

Should it be added to the documentation?

Re: [dpdk-users] My questions and Suggestions about the stable DPDK version number

[Thomas Monjalon]

2016-12-10 10:43, 成桦者别:
> Hi all:
>  Anyone can tell me why does the DPDK version number dpdk-stable-16.07.1 
> change to dpdk-stable-16.07.2  and what about the difference between them?
>  In my opinion, A software's stability begins from its version name. As 
> users, we is sensitive to the version number dpdk-stable-16.07.1.
>  "dpdk-stable-16.07.1" has been applied to our production environment, 
> but it cannot be found from DPDK website now .

It is available at the same URL as before:
http://fast.dpdk.org/rel/

Re: [dpdk-users] Network - Drivers

[Thomas Monjalon]

Hi,

2016-12-14 15:08, Kumaraparameshwaran Rathnavel:
> Hi All,
> 
> I have been trying to use network drivers provided by the DPDK to write a 
> small client - server application. I will explain the configuration

I suggest to start your tests with testpmd in txonly mode.

> I used uio_pci_generic and igb_uio both modules in the Kernel for DPDK 
> Network drivers in userspace. The client and server were run on Centos VM 
> with virtual box as the Host and made sure that Virtual NIC had the driver.
> 
> The sequence of calling the functions as 
> 
> rte_eth_dev_count
> rte_eth_dev_configure
> tx_queue_setup
> rte_pkt_mbuf_alloc
> 
> I filled the packet with Ethernet Header and Ethertype as 0x8870. I did fill 
> the pkt_len and l2_hdr len. I did not fill the VLAN Tag, Outer VLAN tag,
> flags. Transmission is showing success but I am not receiving any packet in 
> the other side wheree i run the receiver setup.
> 
> Can you suggest what should be done and how it should be done.
> 
> Does Drivers work in Virtual Box? If not which is prefered ESX or KVM and 
> does Virtual NICs work?

The supported NICs are listed here: http://dpdk.org/doc/nics
Virtualbox is not known to be fully compatible, but virtio works with Qemu/KVM
and vmxnet3 works with ESX.

Re: [dpdk-users] inshmem build in dpdk-16.11

[Thomas Monjalon]

2016-12-07 08:07, Avi Cohen:
> Hello,
> Why the defconfig file for ivshmem is missing from the /config dir in 
> dpdk-16.11 -  cannot make install for the ivshmem build

Because it has been removed:
http://dpdk.org/commit/c711ccb
There were some design issues and nobody wanted to improve it.

Re: [dpdk-users] PMD For Any Device

[Thomas Monjalon]

2016-12-16 18:36, Cliff Burdick:
> Hi, I wanted to do something like when I'm working on a machine that has an
> ixgbe device I can load that driver, but when using machine with no special
> NIC, just use a generic "PMD" that would go through the Linux stack. Does
> such a thing exist? If not, is there a standard way to test dpdk
> applications on servers without a custom PMD?

Yes you can try af_packet or pcap PMD:
http://dpdk.org/doc/quick-start

[dpdk-users] Why packet_type is zero?

[Thomas Monjalon]

Thanks for reporting.

2015-11-10 15:46, Arseniy Zaostrovnykh:
> Is the pcap driver obsolete?

No

> L3fwd example(http://dpdk.org/doc/guides/sample_app_ug/l3_forward.html) 
> check the mbuf field packet_type, and in the zero case (which is a 
> default value, as far as I know) it does nothing. At the same time, only 
> few drivers even mention this field:
> 
> dpdk-2.1.0 $ grep packet_type drivers -Rl
>  drivers/net/enic/enic_main.c
>  drivers/net/e1000/igb_rxtx.c
>  drivers/net/ixgbe/ixgbe_rxtx.c
>  drivers/net/mlx4/mlx4.c
>  drivers/net/i40e/i40e_rxtx.c
>  drivers/net/mpipe/mpipe_tilegx.c
>  drivers/net/vmxnet3/vmxnet3_rxtx.c
>  drivers/net/fm10k/fm10k_rxtx.c
>  drivers/net/cxgbe/sge.c
> 
> And a PCap driver (drivers/net/pcap/rte_eth_pcap.c) specifically, does 
> not alter the field, so L3fwd application drops all packets.

The zero value is acceptable.
#define RTE_PTYPE_UNKNOWN   0x

Maybe a fix is required in the l3fwd example?
Or maybe it should be explicit that it works with only few drivers.

More generally, the packet type is not mandatory for drivers.
At a time we were talking about implementing a generic callback to
fill it. Or it can be filled in an application fallback with parsing.

[dpdk-users] dpdk - for video frames too ?

[Thomas Monjalon]

2015-11-12 09:14, Ran Shalit:
> I need to develop a driver/application which capture and output video
> frames from PCIe device , and is using Intel cpu (i7), Centand Intel's
> media sdk server framework for the video compression.
> 
> I am not sure what will be a better choice between the following 2 options:
> 1. application which use dpdk for capture and output to the PCIe device
> 2. v4l driver for the PCIe device
> 
> dpdk is supposed to be able to read/write from PCIe device too.
> I tried to see the prons/cons of dpdk compared to v4l.
> 
> prons of dpdk, as I understand them:
> 1. userspace application (easier debugging compared to kernel
> debugging of v4l device driver)
> 2. supposed better performance
> 
> cons of dpdk compared to v4l:
> 1. I could not find examples for PCIe device usage , or samples for
> showing how application (such as media sdk) use dpdk video frames.

There is no API for video currently.
There is an API for networking drivers and a crypto API is coming.
If you feel a DPDK poll mode driver would be a good design, you are
welcome to do it. In case it brings some performance improvement,
it can be decided (no guarantee) to integrate it in the scope of the DPDK.

[dpdk-users] Link Status and Interrupts in 2.2.0

[Thomas Monjalon]

2016-04-25 11:49, martin_curran-gray at keysight.com:
> But if the link was "active" when dpdk and my app starts up,
> then no interrupt is generated, which then means that global
> structure is not filled in

You are describing the case of interrupts enabled but never called.
It has been fixed in DPDK 2.0:
http://dpdk.org/commit/99610782

But a race condition has been seen if interrupt fires during
rte_eth_dev_start() call. The decision was to remove link update if
interrupt is enabled (DPDK 2.2):
http://dpdk.org/commit/d5790b03
This "fix" was a bit strange:
- Is it possible to fix the race condition with link_update in the drivers?
- Is it possible to update the link status in dev_start of each driver
without race condition with the interrupt?
- Why calling link_update() from rte_eth_dev_start() in the polling case?

> I saw a submission to the patchwork 7160, dated Sep 24 2015
> but it was marked as not applicable , archived.

It has been re-submitted and applied (hence the problem):
http://dpdk.org/patch/8112

[dpdk-users] rte_zmalloc() returning non-zeroed memory on FreeBSD

[Thomas Monjalon]

Hi,

2016-08-10 23:30, Verkamp, Daniel:
> It seems that with DPDK 16.07, rte_zmalloc() and related functions no
> longer return zeroed memory reliably on FreeBSD.
> 
> I notice that commit b78c9175118f7d61022ddc5c62ce54a1bd73cea5 ("mem: do
> not zero out memory on zmalloc") removed the explicit memset() that used
> to ensure the buffer was zeroed; its log message says:
> 
> "Zeroing out memory on rte_zmalloc_socket is not required anymore since
> all allocated memory is already zeroed."

On Linux, the memory is zeroed by the kernel.
Then the zero value is maintained in the rte_malloc pool by rte_free.

> However, I don't see how this is guaranteed (at least for FreeBSD), and
> it is not true in practice.  I've attached a minimized reproducer program -
> running it twice in a row fails reliably for me.
> 
> Is there a missing step in FreeBSD, or is it a more general problem for
> other platforms?

I guess the initial value from the kernel has been verified only on Linux.
We could re-add a memset for FreeBSD.

[dpdk-users] unable to configure EAL command line options

[Thomas Monjalon]

Hi all,

General comment for the mailing list:
This kind of footer is not relevant and may sometimes imply
to reject the mail.
Please remove it. Thanks

2016-02-03 17:55, Nagaraj Trivedi:
> =-=-=
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain 
> confidential or privileged information. If you are 
> not the intended recipient, any dissemination, use, 
> review, distribution, printing or copying of the 
> information contained in this e-mail message 
> and/or attachments to it are strictly prohibited. If 
> you have received this communication in error, 
> please notify us by reply e-mail or telephone and 
> immediately and permanently delete the message 
> and any attachments. Thank you

[dpdk-users] [dpdk-dev] DPDK i40evf problem in receving packet

[Thomas Monjalon]

Formatting note:
Please avoid top posting (inline replies are preferred),
and do not include such disclaimer, not relevant on a mailing list.
Thanks

2016-02-10 16:00, Muhammad Zain-ul-Abideen:
> Hi Saurabh,
> Can you tell me what card you are uing
> 
> 
> *Regards:*
> Zain ul Abideen
> 
> 
> *Disclaimer**: The information contained in this e-mail and any attachments
> is confidential; it is intended only for use of the individual or entity
> named above. If the reader of this message is not the intended recipient,
> you are notified that any dissemination, distribution or use of this
> information is strictly prohibited. If you have received this communication
> in error, please delete it and email confirmation to the sender. Thank You.*
> 
> 
> On Wed, Feb 10, 2016 at 6:30 AM, Saurabh Mishra 
> wrote:
> 
> > Hi Qian --
> >
> > Any suggestions? This is bit urgent.
> >
> > /Saurabh
> >
> > On Sat, Feb 6, 2016 at 9:22 AM, Saurabh Mishra 
> > wrote:
> >
> > > Hi Qian --
> > >
> > >
> > > Here's the data from Host:
> > >
> > > [root at oscompute3 ~]# ethtool -i p3p1
> > >
> > > driver: i40e
> > >
> > > version: 1.0.11-k
> > >
> > > firmware-version: f4.40 a1.4 n04.53 e80001dc0
> > >
> > > bus-info: :04:00.0
> > >
> > > supports-statistics: yes
> > >
> > > supports-test: yes
> > >
> > > supports-eeprom-access: yes
> > >
> > > supports-register-dump: yes
> > >
> > > supports-priv-flags: no
> > >
> > > [root at oscompute3 ~]# ethtool -i p3p2
> > >
> > > driver: i40e
> > >
> > > version: 1.0.11-k
> > >
> > > firmware-version: f4.40 a1.4 n04.53 e80001dc0
> > >
> > > bus-info: :04:00.1
> > >
> > > supports-statistics: yes
> > >
> > > supports-test: yes
> > >
> > > supports-eeprom-access: yes
> > >
> > > supports-register-dump: yes
> > >
> > > supports-priv-flags: no
> > >
> > > [root at oscompute3 ~]#
> > >
> > > EthApp> drvinfo
> > >
> > > Port 0 driver: rte_i40evf_pmd (ver: RTE 2.2.0)
> > >
> > > Port 1 driver: rte_i40evf_pmd (ver: RTE 2.2.0)
> > >
> > > EthApp>
> > >
> > > On Fri, Feb 5, 2016 at 4:59 PM, Xu, Qian Q  wrote:
> > >
> > >> What's your current firmware info, can u run ethtool -i port_interface
> > to
> > >> check?
> > >>
> > >> Thanks
> > >> Qian
> > >>
> > >> -Original Message-
> > >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Saurabh Mishra
> > >> Sent: Saturday, February 06, 2016 6:33 AM
> > >> To: dev at dpdk.org; users at dpdk.org
> > >> Subject: [dpdk-dev] DPDK i40evf problem in receving packet
> > >>
> > >> Hi,
> > >>
> > >> I'm seeing two problems:
> > >>
> > >> 1)  when use our kernel '3.10.88-8.0.0.0.6', we only receive first
> > >> packet but not subsequent ones at all after that. However, when I use
> > >> centos7.0, then l2fwd is able to receive all the packets.
> > >>
> > >> 2) I've also seen that on centos7.0, symmetric_mp itself is not working.
> > >> dev start fails with 280 error.
> > >>
> > >> i40evf is giving us lot of headache. The i40evf kernel driver works
> > >> without any problem. Host is a centos7 KVM. I've already upgraded
> > firmware
> > >> to latest.
> > >>
> > >> [root at localhost ~]# uname -a
> > >>
> > >> Linux localhost.localdomain 3.10.0-327.4.5.el7.x86_64 #1 SMP Mon Jan 25
> > >> 22:07:14 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
> > >>
> > >> [root at localhost ~]#
> > >>
> > >> [root at localhost ~]# ./symmetric_mp fakeelf --file-prefix=virtaddr -c2
> > >> -m2048 -n1 --base-virtaddr=0x7fa0 --proc-type=primary -- -p 3
> > >> --num-procs=1 --proc-id=0
> > >>
> > >> EAL: Detected lcore 0 as core 0 on socket 0
> > >>
> > >> EAL: Detected lcore 1 as core 0 on socket 0
> > >>
> > >> EAL: Support maximum 128 logical core(s) by configuration.
> > >>
> > >> EAL: Detected 2 lcore(s)
> > >>
> > >> EAL: No free hugepages reported in hugepages-1048576kB
> > >>
> > >> EAL: VFIO modules not all loaded, skip VFIO support...
> > >>
> > >> EAL: Setting up physically contiguous memory...
> > >>
> > >> EAL: Ask a virtual area of 0x2a80 bytes
> > >>
> > >> EAL: Virtual area found at 0x7fa0 (size = 0x2a80)
> > >>
> > >> EAL: Ask a virtual area of 0x20 bytes
> > >>
> > >> EAL: Virtual area found at 0x7fa02a80 (size = 0x20)
> > >>
> > >> EAL: Ask a virtual area of 0x5400 bytes
> > >>
> > >> EAL: Virtual area found at 0x7fa02aa0 (size = 0x5400)
> > >>
> > >> EAL: Ask a virtual area of 0x40 bytes
> > >>
> > >> EAL: Virtual area found at 0x7fa07ea0 (size = 0x40)
> > >>
> > >> EAL: Ask a virtual area of 0x20 bytes
> > >>
> > >> EAL: Virtual area found at 0x7fa07ee0 (size = 0x20)
> > >>
> > >> EAL: Ask a virtual area of 0x20 bytes
> > >>
> > >> EAL: Virtual area found at 0x7fa07f00 (size = 0x20)
> > >>
> > >> EAL: Ask a virtual area of 0x60 bytes
> > >>
> > >> EAL: Virtual area found at 0x7fa07f20 (size = 0x60)
> > >>
> > >> EAL: Ask a virtual area of 0x20 bytes
> > >>
> > >> EAL: Virtual area found at 0x7fa07f80 (size = 0x20)
> > >>
> > >> EAL: Ask a 

[dpdk-users] Not able to detect Intel EC I218-V (rev 03)

[Thomas Monjalon]

Hi,

2016-02-26 18:59, Nagaprabhanjan Bellaru:
> Hello,
> 
> Using setup.sh (available in tools folder) I have bound eth0 (which is of
> Intel EC I218-V make) to igb_uio module (which I can see with
> dpdk_nic_bind.py --status output):
> 
> --
> Network devices using DPDK-compatible driver
> 
> :00:19.0 'Ethernet Connection (3) I218-V' drv=igb_uio unused=
> 
> Network devices using kernel driver
> ===
> 
> --
> 
> However, when I run any sample application, it complains that there are no
> ethernet ports:
> 
> --
> EAL: Error - exiting with code: 1
>   Cause: No available NIC ports!
> --
> 
> If I give a --vdev=eth_pcap0,iface=eth0, then it picks it. Am I missing
> something? I hope this Intel card is supported by DPDK.

I think this card is not supported.
We are waiting a review for this patch:
http://dpdk.org/dev/patchwork/patch/7161/
Do not hesitate to test it.

[dpdk-users] SyntaxError in setup.sh during binding ethernet device

[Thomas Monjalon]

Hi

2016-01-19 10:45, dawid_jurek:
> Hello DPDK developers,
> I experienced issue when I ran tools/setup.sh and chose option [23]: Bind 
> Ethernet device to IGB UIO module.
> Script Output:
> File ".../dpdk/dpdk-2.2.0/tools/dpdk_nic_bind.py", line 113
> """ % locals() # replace items from local variables
>   ^
> SyntaxError: invalid syntax
> It turned out that python 2.7 set as default in system was needed.
> So I added python2 to every line with dpdk_nic_bind.py call in tools/setup.sh 
> and now it works fine.
> But the question is: is it expected behaviour? Shouldn't it be fixed by 
> simple patch in way I did it?

Yes a patch would be appreciated to make it python 3 friendly.
Thanks

[dpdk-users] Running DPDK as non-root

[Thomas Monjalon]

Hi

2016-07-07 16:47, Jez Higgins:
> Is it possible to get DPDK up and running as non-root - if so, can
> anyone guide me to what I'm missing? Or should I be giving this up as a
> bad job?

You can try the --no-huge option.
But most of drivers won't work without hugepage currently.
A rework of the memory allocation is needed to make it work better.

[dpdk-users] Issue with virtio PMD (using dpdk-2.2.0)

[Thomas Monjalon]

2016-06-22 15:58, Chinmaya Dwibedy:
> It appears that, the virtio PMD takes over each virtio device.

It has been fixed in recent DPDK.
You can use --pci-blacklist to avoid this behaviour in old DPDK.

[dpdk-users] Get interface speed through IOCTL

[Thomas Monjalon]

2016-06-22 17:49, Gadre Nayan:
> I need to get the interface speed though standard IOCTL call:
[...]
> 
> However, for 10G interface I do not read a correct speed, since it may
> not be supported.

Yes

> Out 1G cards are I350 and 10G card is I40.
[...]
> There is no support for 1base. So to get the ioctl working for 10G
> card, is it a trivial change of adding few more support options and
> adding another case SPEED_1 clause, or is it more involved ?

You are talking about KNI which has some old ethtool support for
some igb and ixgbe NICs.
My opinion is that we should drop this ethtool support which do not
compile everywhere and has a very limited support.

If you want to get the speed information, you have several other ways
to explore:
- communicating with the DPDK application
- make a secondary application process
- add the feature in the secondary process app/proc_info
- use examples/ethtool (and check if it can be promoted in lib/)
- implement a kernel module upstream

If you just need to get the speed info, I feel app/proc_info would be
a good idea.

[dpdk-users] R: Error in rte_eal_init using dpdk 16.04 and Qlogic (Broadcom) bnx2x driver in a SECONDARY process

[Thomas Monjalon]

2016-05-05 13:02, David Coen:
> Looking at Intel and Mellanox driver sources ("ixgbe_ethdev.c" and "mlx4.c" 
> respectively), 
> I found that, in the driver function called by rte_eal_init to initialize pci 
> devs, is mandatory
> to distinguish if the process is running as SECONDARY (--proc-type=secondary 
> in EAL prameters).
> 
> For example:
> eth_ixgbe_dev_init() function of the ixgbe Intel driver, at line 988 :
> 
>   /*
>* For secondary processes, we don't initialise any further as primary
>* has already done this work. Only check we don't need a different
>* RX and TX function.
>*/
>   if (rte_eal_process_type() != RTE_PROC_PRIMARY){
>   ... some code lines here ...
>   return 0;
>   }
> 
> While in Mellanox mlx4 driver there is a check in ALL driver functions that 
> makes them return
> immediatelly if the running process is of type "secondary".
> 
> That means that a driver has to take into account that the process could be a 
> SECONDARY process...

This is called "Multiprocess aware" in doc/guides/nics/overview.rst.

> but in Qlogic (Broadcom) bnx2x driver I didn't find anything about that.

It seems the driver do not comply with this multiprocess requirement.

> Do you think I've missed something ?

You are probably right :)

[dpdk-users] Link status?

[Thomas Monjalon]

2016-05-25 12:14, Jason Lixfeld:
> Is there an easy way to do that with an existing dpdk application, or does 
> one need to be an super uber programmer type (which I am not anywhere close 
> to being :)) and write some application to be able to do that?

Yes you need to access to a variable inside the application to know that.
You can have a secondary application which looks into the primary one.
You can try to write one or check these apps:
app/proc_info (stats but no link status yet)
examples/ethtool/ethtool-app/ (more complicate than a secondary app)

[dpdk-users] 3rd party PMDs

[Thomas Monjalon]

2016-11-01 14:19, Radley Kneeland:
> Does dpdk have a way to integrate/link third party PMDs that does not
> require placing them within the dpdk source tree?
> 
> currently using dpdk-16.07

Yes it works.

However it is generally better to submit your PMD here.

[dpdk-users] Compiling DPDK 16.07 with shared option is failing..

[Thomas Monjalon]

2016-11-10 10:37, Nagaprabhanjan Bellaru:
>  I found that librte_timer was not enabled and so the compilation failed.
> When I enabled the same, it went ahead and failed at a different place
> while compiling librte_pmd:
> 
> --
> gcc -pie -fPIC
> -L/home/ubuntu/development/libfwdd/src/platform/vpp/vpp/build-root/install-vpp_debug-native/dpdk/lib
> -Wl,-g -shared rte_eth_bond_api.o rte_eth_bond_pmd.o rte_eth_bond_args.o
> rte_eth_bond_8023ad.o rte_eth_bond_alb.o -z defs -lrte_mbuf -lethdev
> -lrte_eal -lrte_kvargs -lrte_cmdline -lrte_mempool -lrte_ring
> -Wl,-soname,librte_pmd_bond.so.1.1 -o librte_pmd_bond.so.1.1
> /usr/bin/ld: librte_pmd_bond.so.1.1: version node not found for symbol
> rte_eth_bond_8023ad_conf_get at DPDK_2.0
> /usr/bin/ld: failed to set dynamic section sizes: Bad value
> collect2: error: ld returned 1 exit status
> --

What are you changing in the default configuration?
Which commands are you using to compile?

[dpdk-users] Problem building DPDK libraries

[Thomas Monjalon]

2016-11-17 15:51, David Aldrich:
> make[6]: *** /lib/modules/4.1.21-rt13-WR8.0.0.10_preempt-rt/build: No such 
> file or directory.  Stop.

You need the kernel headers, or just disable compilation of kernel modules:
sed -ri 's,(KNI_KMOD=).*,\1n,' build/.config
sed -ri 's,(IGB_UIO=).*,\1n,' build/.config

[dpdk-users] Problem building DPDK libraries

[Thomas Monjalon]

2016-11-17 17:02, David Aldrich:
> You are all very helpful. Thank you.
> 
> I'm afraid 'make install' isn't working:
> 
> # make install
> make[1]: Nothing to be done for 'pre_install'.
> == Installing /usr/local/
> cp: cannot stat 'build/lib/*': No such file or directory
> /root/dpdk-2.2.0/mk/rte.sdkinstall.mk:116: recipe for target 
> 'install-runtime' failed
> make[3]: *** [install-runtime] Error 1

The canonical commands are:

make config T=x86_64-native-linuxapp-gcc
make -j3
make install DESTDIR=myinstall

You can change build/.config if needed.
The install step is optional.

There is also
make help

[dpdk-users] Problem building DPDK libraries

[Thomas Monjalon]

2016-11-17 17:23, David Aldrich:
> # make install T=x86_64-native-linuxapp-gcc install
> Installation cannot run with T defined and DESTDIR undefined

Extract of "make help":
install T=   configure, build and install a target in DESTDIR
install  install optionally staged in DESTDIR

So "install T=" is equivalent to
make config
make
make install
But install cannot be done without DESTDIR.

[dpdk-users] DPDK Linux kernel version requirement

[Thomas Monjalon]

2016-11-17 11:46, James Huang:
> CentOS release 6.5 (Final)
[...]
> According to the test results, could I say DPDK 32bit applications are
> GOOD to run on Linux 64bit kernel v2.6.32?

CentOS has a lot of backports so it is not really a 2.6.32 kernel.
It is good to know that DPDK 16.07 runs on CentOS 6.5,
but we cannot conclude on other 2.6.32 distributions.

[dpdk-users] Sphinx Errors on CentOS 7

[Thomas Monjalon]

2016-10-05 08:09, Aly Dharshi:
> Hello DPDK Users,
> 
> I am building an rpm set using the bundled SPEC file. Everything
> compiles fine except the sphinx part:
[...]
> I've had to do this in the spec file to generate the build:
> 
> BuildRequires: kernel-devel, kernel-headers, libpcap-devel, xen-devel
> BuildRequires: doxygen, python-sphinx, inkscape
> #BuildRequires: texlive-collection-latexextra
> 
> This is because it seems that texlive-collection-latexextra does not
> exist on CentOS 7.2. Is there anyway to proceed:
> 
> 1. How do I resolve the sphinx error ?

It is really hard to know what's going wrong with PDF generation.
You can try "make doc V=1"

> 2. Can I simply build this kit without docs  ?

Yes you can remove the line "make O=%{target} doc" from the spec file.

[dpdk-users] [dpdk-dev] Project Governance and Linux Foundation

[Thomas Monjalon]

Hi,

This is a really interesting use case.
Questions below,

2016-10-12 13:44, qin.chunhua at zte.com.cn:
> ZTE is supportive of improving the DPDK project governance including 
> moving the project to the Linux Foundation. 
> DPDK has been used in lots of ZTE's equipments and solutions  such as 
> BBU,RNC,EPC,vEPC,vBRAS,vCPE,vRouter,vSwitch,and so on for many years. 
> We have also done some optimization in DPDK.

You mean you made some optimizations to DPDK but keep them secret?

> If  the DPDK  project governance moves to the Linux Foundation,
> we think this will promote ZTE and other companies to contribute in DPDK.

Why it would help you to contribute?
Why not contribute today?
The project is truly open, a lot of major companies are contributing,
why not yours?

[dpdk-users] Project Governance and Linux Foundation

[Thomas Monjalon]

2016-10-17 14:40, O'Driscoll, Tim:
> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> > 2016-10-17 11:52, O'Driscoll, Tim:
> > > From: Hobywan Kenoby
> > > > The current DPDK version can run on virtually all processors (Intel,
> > > > IBM and ARM) and leverage all NICs: is there **really** anyone
> > > > questionning openness of the community?
> > >
> > > I still hear concerns on this, and based on discussions with others
> > > who put their names to the post below, they do too.
> > > I think it's a perception that we need to address.
> > 
> > It is simple to address this perception with fact checking.
> > The next releases will provide even more code for ARM and NPUs.
> > If someone submits some good code and is ignored, it is easy enough
> > to ping the mailing list and make it visible.
> > If someone sees any regression on his architecture, we care.
> > Please let's stop maintaining confusion on this topic.
> > 
> > DPDK *is* truly open.
> 
> Well, to be a little more specific, the concern I've heard on many occasions 
> is that 6WIND control the infrastructure for the project and so effectively 
> have a veto over what's accepted into DPDK. Your argument is that you've 
> never exercised that veto, which is true, but you still have the ability to 
> do so. That's not a characteristic of a truly open project. As stated in the 
> original post on this:
> 
> > - The infrastructure for a project like DPDK should not be owned and 
> > controlled by any single company.

Technically yes, we can improve that part, at the cost of more coordination
with more administrators, and without being sure that everybody will trust
this new organization.
I would like to highlight that this supposed veto cannot really be exercised
because feedbacks are open on the mailing list.

I'm worried that we are talking too much about a veto situation which
does not happen, and would mean ignoring some comments, whereas the real
issue is the lack of reviews.
Apart that, I still think such organization can be interesting for other
(legals and budget) reasons.

At this point, I must admit that moving the project infrastructure will have
at least one big benefit: stopping this kind of discussion.
And such discussion will probably never happen again because nobody will
take the risk of annoying the big vendors supporting the new organization.
One can wonder whether it is an improvement.

[dpdk-users] [dpdk-dev] Project Governance and Linux Foundation

[Thomas Monjalon]

Hi,
Thanks Dave for the report.

I suggest to continue on the new mailing list:
moving at dpdk.org
Please register if you are interested in the structure move:
http://dpdk.org/ml/listinfo/moving


2016-10-21 15:00, Dave Neary:
> Hi all,
> 
> We had a great session yesterday on this topic, I took some notes - does
> anyone who was there have any corrections, or anyone who was not have
> any comments?
> 
> Thanks,
> Dave.
> 
> Tim led the discussion, and started by outlining that he saw there were
> 3 different questions which we should treat independently:
> 
> 1. Is there a benefit to moving DPDK to a foundation?
> 2. If the answer is yes: there are two options currently proposed - a
> low overhead, independent project under the Linux Foundation (LF Lite),
> or joining fd.io as a sub-project. Which one of these is preferable, or
> is there another option to consider?
> 3. Are there any related changes we should consider in technical
> infrastructure and project governance?
> 
> I outlined some advantages I see to the Linux Foundation:
> * Pool resources for events
> * Provides some legal foresight
> * LF standing behind a project gives some companies assurances that
> there is good, open technical governance and a level playing field for
> participants
> 
> Stephen Hemminger asked if there was a sponsorship requirement. Tim
> responded that it is possible to do what Open vSwitch has done, and have
> no membership funding requirement. What that means is that any funds the
> project community wants to spend needs to be budgeted ad hoc.
> 
> A number of others (Shreyansh Jain, Matt Spencer) said they would like
> to see a formal model for non-technical engagement, legal protection for
> patent and copyright, and more clarity on the technical governance.
> 
> Vincent Jardin said that whatever happens, it is vital that DPDK remain
> an open, community-run project.
> 
> A number of people expressed interest in the change, but could not
> commit to funding.
> 
> Jerome Tollet said that he felt it was important to have better test and
> CI infrastructure, and that these cost money. He proposed that since
> fd.io already has infrastructure and a lab, that this would be an
> affordable option for doing this.
> 
> Vincent and Thomas Monjalon suggested that distributed testing was a
> better option - creating an opportunity for different people to send
> test results to a central gathering point. Thomas mentioned that
> Patchwork has a feature which allows aggregation of test results for
> specific patches now.
> 
> Tim asked if there was agreement on a move, and there was no opposition.
> Vincent suggested opening a call for proposals to have a wider range of
> choices than LF Lite or fd.io. Jim St. Leger said we have already had a
> group who evaluated options and made a proposal, and we should not re-do
> the process.
> 
> Jerome recommended that we focus on requirements and criteria for
> determining the choice: timing, governance requirements, budget, and
> hardware/infrastructure requirements. Keith Wiles suggested that there
> was a need for some budgetary requirement to show commitment of
> participating companies.
> 
> When asked about transferring the ownership of the domain name to Linux
> Foundation, Vincent reiterated that his main concern was keeping the
> project open, and that he did not anticipate that transferring the
> domain ownership would be an issue.
> 
> Moving on to question 2:
> 
> I said that Red Hat is happy with the technical operation of the
> project, and we don't want to see the community disrupted with toolset
> changes - and it's possible to work with projects like fd.io, OVS, and
> OPNFV to do testing of DPDK.
> 
> Representatives from Brocade, Cavium, and Linaro all voiced a preference
> for a stand-alone lightweight project - one concern voiced was that
> there is a potential perception issue with fd.io too.
> 
> Maciek K and Jerome encouraged everyone not to underestimate the
> difficulty in setting up good CI and testing processes.
> 
> To close out the meeting, Tim summarised the consensus decisions:
> 
> * We agreed to move to a foundation
> * A group will work on re-doing a budget proposal with the Linux
> Foundation - target of 4 weeks to come up with a budget proposal for the
> community
> * There is a preference for an independent project rather than being a
> sub-project
> 
> Budget group:
> * Matt Spencer, ARM
> * Jerome Tollet, Cisco
> * Ed Warnicke, Cisco
> * Shreyansh Jain, NXP
> * Dave Neary, Red Hat
> * Jan Blunk, Brocade
> * Vincent Jardin, 6WIND
> * Thomas Monjalon, 6WIND
> * Tim O'Driscoll, Intel
> * Francois Ozog, Linaro
> * John Bromhead (sp?), Cavium

[dpdk-users] Cross compile for ARM64 fails due to librte_vhost and pmdinfogen issues

[Thomas Monjalon]

Hi,

2016-09-12 22:20, Michael Wildt:
> I'm attempting to cross compile DPDK on an x86 for an ARM64 target. This
> fails in the following areas, using latest dpdk as of 9/12. When compiling
> natively there are no issues.

Your analysis below seems good.
Interestingly, I do not see such error (don't know why).
Please could you share the commands you are using?

> - librte_vhost, fails with:
> 
> /projects/dpdk_latest/lib/librte_vhost/vhost_user/virtio-net-user.c:250:23:
> error: array subscript is above array bounds [-Werror=array-bounds]
>rvq = dev->virtqueue[i * VIRTIO_QNUM + VIRTIO_RXQ];
[...]
> - buildtools/pmdinfogen, fails with:
> 
> == Build buildtools/pmdinfogen
>   HOSTCC pmdinfogen.o
> /projects/dpdk_test_wget/dpdk-16.07/build/include/rte_byteorder.h:
> Assembler messages:
> /projects/dpdk_test_wget/dpdk-16.07/build/include/rte_byteorder.h:53:
> Error: no such instruction: `rev16 %bx,%bx'
[...]
>   - The issue is due to the rte_byteorder.h file which gets symlink'ed with
> the ARM version at the beginning of the build. The pmdinfogen is always
> compiled for x86 thus the asm is failing. I see two ways to address this.
> 
> - Have pmdinfogen always pickup the local OS rte_byteorder.h file, not the
> ARCH symlinked one.

Yes we should not use DPDK headers in pmdinfogen or have a host-include/.

> - Fix pmdinfogen such that it gets cross compiled correctly as the rest of
> the code. Did a quick attempt by setting the HOST_OS to 0, , but then it
> fails at the HOST_LD.

No it must be native because it is part of the build system.

Re: [dpdk-users] Too much time in mmap()

[Thomas Monjalon]

2017-01-04 14:38, Chae-yong Chong:
> I am using 16.07. Is it already 'too old'? -_-;;;

No it is not too old :)

> Almost time is spent on mmap() not the memset()

Ah OK interesting to know.
Which kernel is it?
Could you profile what is done in kernel side?

PS: please reply below the questions

Re: [dpdk-users] Too much time in mmap()

[Thomas Monjalon]

2017-01-04 08:08, Chae-yong Chong:
> Hi,
> 
> Too much time is spent on mmap() in rte_eal_init().
> When running DPDK application with 35 1G huge pages, it takes about 12sec
> in mapping().
> Is there any way to reduce this time?

I guess you are using an old DPDK.
You just need to upgrade :)

Re: [dpdk-users] dpdk.org server maintenance

[Thomas Monjalon]

2016-12-19 15:52, Thomas Monjalon:
> Hi all,
> 
> On Wednesday December 21, 2016, between 6:00 and 9:00 UTC, there will be
> an upgrade of our hosting platform.
> Unfortunately it is announced a downtime of an hour or more.
> It will affect every hosted services: web, mail, git, patchwork.
> 
> It is unavoidable so we will have to be patient.


There was no disturbance in the Force as the operation has been cancelled :)
Hard to predict hosting service operations, is it. Yes hmmm

Re: [dpdk-users] [dpdk-dev] Reg DPDK with unsupported NIC

[Thomas Monjalon]

2017-03-14 14:58, raman geetha gopalakrishnan:
> In what way i can still develop DPDK APP with non supported NIC till get
> the DPDK NIC.

You should check the bottom of this page (section Ohers):
http://dpdk.org/doc/nics
The simplest is to use the ring PMD which do not require a NIC at all.
Or you can try the other solutions (pcap, tap, af_packet).

Re: [dpdk-users] difference between rte_eth_tx_buffer and rte_eth_tx_burst

[Thomas Monjalon]

2017-03-05 15:22, Wiles, Keith:
> 
> > On Mar 4, 2017, at 7:46 PM, Yuliang Li  wrote:
> > 
> > Hi all,
> > 
> > I am new to DPDK. I found 2 functions that can send packets:
> > rte_eth_tx_buffer and rte_eth_tx_burst. I am confused about why we need
> > these two different functions. To me, I can use only rte_eth_tx_burst, and
> > rte_eth_tx_buffer does add more functional capability. Are there any other
> > concerns like performance or others that we need these two functions?
> 
> The best place to find out this type of information is to read the emails 
> around the rte_eth_tx_buffer APIs in the archives. You may want to refine the 
> search.
> 
> https://mail-archive.com/search?a=1=dev%40dpdk.org=newest=rte_eth_tx_buffer

The best place should be the documentation :)
If someone is missing in the doc, please tell us what or make a patch.
Thanks

Re: [dpdk-users] If shared libraries are used vdev doesn't work anymore

[Thomas Monjalon]

31/07/2017 16:58, Dirk-Holger Lenz:
> If dpdk is built with 'CONFIG_RTE_BUILD_SHARED_LIB=y' then
> using the vdev feature (args: e.g. -c 3 -n 4 --vdev="crypto_openssl")
> the rte_eal_init() returns 'ERROR: failed to parse device "crypto_openssl"'.
> It looks to me that rte_eal_devargs_add() calling rte_eal_devargs_parse()
> is trying to check the device name before the shared libraries are read
> and the internal data arrays are setup.

Yes, you're right: eal_parse_args() is called before eal_plugins_init().
The fix is not small: we should split the args parsing to parse the
device arguments after loading shared libraries.

It is a release blocker.

Re: [dpdk-users] If shared libraries are used vdev doesn't work anymore

[Thomas Monjalon]

01/08/2017 10:17, Gaëtan Rivet:
> Hi,
> 
> On Mon, Jul 31, 2017 at 10:23:50PM +0200, Thomas Monjalon wrote:
> > 31/07/2017 16:58, Dirk-Holger Lenz:
> > > If dpdk is built with 'CONFIG_RTE_BUILD_SHARED_LIB=y' then
> > > using the vdev feature (args: e.g. -c 3 -n 4 --vdev="crypto_openssl")
> > > the rte_eal_init() returns 'ERROR: failed to parse device 
> > > "crypto_openssl"'.
> > > It looks to me that rte_eal_devargs_add() calling rte_eal_devargs_parse()
> > > is trying to check the device name before the shared libraries are read
> > > and the internal data arrays are setup.
> > 
> > Yes, you're right: eal_parse_args() is called before eal_plugins_init().
> > The fix is not small: we should split the args parsing to parse the
> > device arguments after loading shared libraries.
> > 
> > It is a release blocker.
> 
> I saw that yesterday, tried to investigate a bit.
> I have currently an issue when launching testpmd when
> BUILD_SHARED_LIB=y. Mbufs fail to be allocated. I was not able to find
> the root cause for this.

Have you loaded the mempool driver?

> Anyway, I guess there are two possible solutions:
> 
>   - Delayed device validation
>   - Earlier plugins init
> 
> Thomas you seem to propose the first one, I agree that it will probably
> be a little involved to implement but I guess it's feasible in time.
> However, I don't yet understand why the second one is not possible from
> the get-go. It makes sense in any case that the system should be
> stabilized as soon as possible - i.e. that underlying subsystems such as
> plugins and capabilities are loaded first to expose a stable set of
> capabilities to any subsequent initializations.

The problem is that the plugins initialization depends on tailqs, log
and memory initialization.

> So, I will lack time to investigate the issue with testpmd and shared
> libs. If anyone has any idea, I will gladly hear it. In the meantime, I
> will test those two solutions, see what would be feasible, and try to
> propose one shortly.

Thank you

Re: [dpdk-users] [dpdk-maintainers] Issue->Dpdk for arm cortex-a15 compilation

[Thomas Monjalon]

16/05/2017 12:51, Jan Viktorin:
> DPDK does not support OpenWRT because (as far as I know) nobody from
> the DPDK community is using it in this way. I build DPDK via Buildroot
> but this is unsupported by the DPDK upstream.

Why means "unsupported upstream"? Are we missing some patches?
Should we integrate Buildroot compilation in our CI?

Re: [dpdk-users] Invalid TCP/UDP checksum for IPV4 packets

[Thomas Monjalon]

Hi,

31/05/2017 10:16, Julien Castets:
> I had a running application on DPDK 2.2.0 which offloads IP/TCP/UDP
> checksums to the NIC. A few days ago, I started the migration to 17.02
> and started to see invalid TCP/UDP checksum.
> 
> I called the new API rte_eth_tx_prepare() but checksums were still invalid.
> 
> After digging for a few hours in DPDK source code, I discovered the
> flag PKT_TX_IPV4 needs to be set in ol_flags.
> 
> Basically, the function rte_net_intel_cksum_flags_prepare of rte_net.h
> (http://dpdk.org/doc/api/rte__net_8h_source.html) does the following:
> 
> if (offload TCP checksum flag is set) {
>   if (PKT_TX_IPV4 in ol_flags) {
>  // compute ipv4 checksum
>}
>else {
>  // compute ipv6 checksum
>}
> }
> 
> Because I didn't set PKT_TX_IPV4 in ol_flags, I was going into the
> "else" clause and the checksum was computed as if my packet was an
> IPV6 packet.
> 
> Don't you think it would be preferable to assert instead of
> considering a non-explicitely-declared-IPV4-packet as being an IPV6
> packet?

Yes, you're right.
I think we were avoiding wasting 1 cycle here.
Usability was the cost for best performance.
A good trade-off may be to use RTE_ASSERT which will be triggered only
in debug mode.

Re: [dpdk-users] [ovs-dev] adding dpdk ports sharing same pci address to ovs-dpdk bridge

[Thomas Monjalon]

20/09/2017 19:33, Kevin Traynor:
> On 09/08/2017 10:56 AM, Loftus, Ciara wrote:
> >> Hi,
> >>
> >> I have compiled and built ovs-dpdk using DPDK v17.08 and OVS v2.8.0. The
> >> NIC that I am using is Mellanox ConnectX-3 Pro, which is a dual port 10G
> >> NIC. The problem with this NIC is that it provides only one PCI address for
> >> both the 10G ports.
> >>
> >> So when I am trying to add the two DPDK ports to my br0 bridge
> >>
> >> # ovs-vsctl --no-wait add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
> >> options:dpdk-devargs=0002:01:00.0
> >>
> >> # ovs-vsctl --no-wait add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk
> >> options:dpdk-devargs=0002:01:00.0
> >>
> >> The port dpdk1 is added successfully and able to transfer data, but adding
> >> dpdk0 to br0 fails:
[...]
> >> With OVS v2.6.1 I never had this problem as dpdk-devargs was not
> >> mandatory
> >> and just specifying port name was enough to add that port to bridge.
> >>
> >> Is there a way to add port both ports to bridge ?
> > 
> > It seems the DPDK function rte_eth_dev_get_port_by_name() will always 
> > return the port ID of the first port on your NIC, when you specify the 
> > single PCI address and that's where the problem is. There doesn't seem to 
> > be a way currently to indicate to the calling application that in fact two 
> > (or more) port IDs are associated with the one PCI address.

We have two ports (with the same PCI address) so we should have
two different names.
Where the names passed to rte_eth_dev_get_port_by_name() come from?
It is the user parameter from options:dpdk-devargs=0002:01:00.0, right?

> > I am cc-ing DPDK users mailing list for hopefully some input. Are there any 
> > plans for the rte_eth_dev_get_port_by_name function to be compatible with 
> > NICs with multiple ports under the same PCI address?

We cannot return two different ports for the same name.
There are two issues here:
- the input should not be the PCI address
- the ethdev function should look at ethdev name, not rte_device one

The idea is that we have only one rte_device object and we instantiate
two rte_eth_dev ports.
An ethdev port can be identified with its id (a number) or its unique name.
Unfortunately, the user cannot guess the port id or the name set by the PMD.

> Hi Adrien/Nelio,
> 
> Is this something you can answer? We're wondering how to handle this in
> OVS and whether a temporary or long term solution is needed.

I suggest to rely on ethdev name.
You will need to show to the user the mapping between the bus information
(PCI id here) and the device names.

Another alternative is to add a new function returning all ethdev ports
associated to a given rte_device resource.
So you would get two ports and you could pick one on the first "add-port",
and the other one for the second "add-port" command.
It means the user would be forced to add them in the right order if he
wants a reproducible result.

Re: [dpdk-users] [ovs-dev] adding dpdk ports sharing same pci address to ovs-dpdk bridge

[Thomas Monjalon]

21/09/2017 10:04, Loftus, Ciara:
> > 20/09/2017 19:33, Kevin Traynor:
> > > On 09/08/2017 10:56 AM, Loftus, Ciara wrote:
> > > > It seems the DPDK function rte_eth_dev_get_port_by_name() will
> > > > always return the port ID of the first port on your NIC,
> > > > when you specify the single PCI address and that's where the
> > > > problem is. There doesn't seem to be a way currently to
> > > > indicate to the calling application that in fact two
> > > > (or more) port IDs are associated with the one PCI address.
> > 
> > We have two ports (with the same PCI address) so we should have
> > two different names.
> > Where the names passed to rte_eth_dev_get_port_by_name() come from?
> > It is the user parameter from options:dpdk-devargs=0002:01:00.0, right?
> 
> Yes, we're using the PCI address specified by the user in dpdk-devargs.
> 
> > > > I am cc-ing DPDK users mailing list for hopefully some input.
> > > > Are there any plans for the rte_eth_dev_get_port_by_name function
> > > > to be compatible with NICs with multiple ports under the same PCI 
> > > > address?
> > 
> > We cannot return two different ports for the same name.
> > There are two issues here:
> > - the input should not be the PCI address
> > - the ethdev function should look at ethdev name, not rte_device
> > one
> 
> This would require the user having to "guess" the DPDK ethdev name
> which is something we'd like to avoid.

Yes, but you can provide a way to list the ports with their names
and characteristics.

> We had the same problem using DPDK port IDs and decided not to use
> them anymore, and use the PCI instead as it took the guesswork out.
> Ethdev names and port IDs can change between tests, unlike the PCI
> address which tends to remain constant for a device.

We can add a requirement on ethdev names and make sure they remain
constant for a given port.

> > The idea is that we have only one rte_device object and we instantiate
> > two rte_eth_dev ports.
> > An ethdev port can be identified with its id (a number) or its unique name.
> > Unfortunately, the user cannot guess the port id or the name set by the
> > PMD.
> 
> Exactly. Thanks for clarifying what's going on under the hood.
> 
> Ciara
> 
> > 
> > > Hi Adrien/Nelio,
> > >
> > > Is this something you can answer? We're wondering how to handle this in
> > > OVS and whether a temporary or long term solution is needed.
> > 
> > I suggest to rely on ethdev name.
> > You will need to show to the user the mapping between the bus information
> > (PCI id here) and the device names.
> > 
> > Another alternative is to add a new function returning all ethdev ports
> > associated to a given rte_device resource.
> > So you would get two ports and you could pick one on the first "add-port",
> > and the other one for the second "add-port" command.
> > It means the user would be forced to add them in the right order if he
> > wants a reproducible result.

Re: [dpdk-users] How to customize the files in DPDK installed dir?

[Thomas Monjalon]

12/09/2017 03:53, Yan, Liming (NSB - CN/Hangzhou):
> Hi,
>  I have a question about DPDK installation.   Could anyone please help me?   
> Thanks.
> After I run command 'make install T=${RTE_TARGET} 
> DESTDIR=/work/dpdk1705/install', all files are installed in 
> /work/dpdk1705/install as following.   But if I don't need the 
> 'share/dpdk/examples', how can I remove it?   I didn't find the place where I 
> can change this.Changing dpdk.spec seems does not take effect.

See this file:
http://dpdk.org/browse/dpdk/tree/mk/rte.sdkinstall.mk

Examples are part of install-doc which is part of "make install".

Note: you probably do not want to use T= with install.
See also "make help"

Re: [dpdk-users] [vpp-dev] vpp-verify-master-opensuse build failure triage

[Thomas Monjalon]

Hi,

It is an error on DPDK side.
It seems the tarball for DPDK 17.08 has been overwritten by the stable release 
process.
I have restored the original tarball and we are checking what went wrong.

fast.dpdk.org is a CDN for static.dpdk.org.
It may take time to get the restored tarball on all CDN nodes.

Sorry for the noise


28/11/2017 12:31, Marco Varlese:
> Hi Gabriel,
> 
> I just submitted a patch (https://gerrit.fd.io/r/#/c/9597/) to VPP to fix 
> thisissue. 
> I've added you to the review so you can take a look.
> 
> In order to verify that the new patch fixes the issue, people should first 
> run:
> 
> $ rpm -e vpp-dpdk-devel && rm dpdk/dpdk-17.08.tar.xz
> 
> to trigger a rebuild and reinstall of the dpdk package...
> 
> 
> Cheers,
> Marco
> 
> On Tue, 2017-11-28 at 10:34 +, Gabriel Ganne wrote:
> > Adding dpdk-user ML.
> > 
> > I had a look with an older dpdk archive I found.
> > The folder archived has been renamed from *dpdk-17.08* to 
> > *dpdk-stable-17.08*
> > This is the only difference, but it is enough to make the md5sum fail.
> > 
> > --
> > Gabriel Ganne
> > From: Gabriel Ganne
> > Sent: Tuesday, November 28, 2017 11:13:23 AM
> > To: Marco Varlese; Dave Wallace; Gonzalez Monroy, Sergio
> > Cc: vpp-...@lists.fd.io
> > Subject: Re: [vpp-dev] vpp-verify-master-opensuse build failure triage
> >  
> > Hi Marco,
> > 
> > I believe http://fast.dpdk.org/rel redirects to http://static.dpdk.org/rel/
> > 
> > I disagree on the md5 hashs.
> > I have the following (NOK on 17.08, and OK on 17.11) :
> > 
> > $ wget http://static.dpdk.org/rel/dpdk-17.08.tar.xz
> > $ openssl md5 dpdk-17.08.tar.xz # is 0641f59ea8ea98afefa7cfa2699f6241 in
> > dpdk/Makefile
> > MD5(dpdk-17.08.tar.xz)= 537ff038915fefd0f210905fafcadb4b 
> > 
> > $ wget http://static.dpdk.org/rel/dpdk-17.11.tar.xz
> > $ openssl md5 dpdk-17.11.tar.xz
> > MD5(dpdk-17.11.tar.xz)= 53ee9e054a8797c9e67ffa0eb5d0c701 
> > 
> > Though I agree that if the "recheck" button made the build pass, there must 
> > be
> > something wrong on my side.
> >  ... what did I miss ?
> > 
> > --
> > Gabriel Ganne
> > From: Marco Varlese 
> > Sent: Tuesday, November 28, 2017 10:55:49 AM
> > To: Gabriel Ganne; Dave Wallace; Gonzalez Monroy, Sergio
> > Cc: vpp-...@lists.fd.io
> > Subject: Re: [vpp-dev] vpp-verify-master-opensuse build failure triage
> >  
> > Hi Gabriel,
> > 
> > On Tue, 2017-11-28 at 09:19 +, Gabriel Ganne wrote:
> > > Hi,
> > > 
> > > I also have this issue on my machine, and I see on 
> > > http://static.dpdk.org/re
> > > l/ that dpdk-17.08.tar.xz  was written yesterday (27-Nov-2017 13:00)
> > > Wouldn't it be possible that the archive was overwritten ?
> > 
> > The DPDK tarball in VPP is downloaded from http://fast.dpdk.org/rel
> > According to http://dpdk.org/rel the MD5 used in VPP for the DPDK 17.08
> > release is correct.
> > > In which case, the hash would need to be updated.
> > 
> > Right, if the tarball was a newer and different one then the MD5 hash should
> > be updated in VPP for the the checksum performed...
> > However, in the case described by Dave below, a simple 'recheck' which
> > triggers a new build (with the same code/scripts/etc. hence the same MD5 
> > hash)
> > solved it.
> > 
> > > Also, this would probably not be seen by people who had the 
> > > dpdk-install-dev 
> > > package already installed.
> > > 
> > > Who should I ask to check this ?
> > 
> > I've added Sergio who might have further thoughts on this one.
> > 
> > > Best regards
> > > 
> > > --
> > > Gabriel Ganne
> > > From: vpp-dev-boun...@lists.fd.io  on behalf 
> > > of
> > > Marco Varlese 
> > > Sent: Tuesday, November 28, 2017 9:19:37 AM
> > > To: Dave Wallace
> > > Cc: vpp-...@lists.fd.io
> > > Subject: Re: [vpp-dev] vpp-verify-master-opensuse build failure triage
> > >  
> > > Dear Dave,
> > > 
> > > By the look of it is seemed to have been an hiccup with the download or
> > > that something spurious was left on the filesystem...
> > > ===
> > > 12:08:13 Bad Checksum! Please remove /w/workspace/vpp-verify-master-
> > > opensuse/dpdk/dpdk-17.08.tar.xz and retry
> > > 12:08:13 Makefile:267: recipe for target '/w/workspace/vpp-verify-
> > > master-opensuse/build-root/build-vpp-native/dpdk/.download.ok' failed
> > > 12:08:13 make[3]: *** [/w/workspace/vpp-verify-master-opensuse/build-
> > > root/build-vpp-native/dpdk/.download.ok] Error 1
> > > 12:08:13 make[3]: Leaving directory '/w/workspace/vpp-verify-master-
> > > opensuse/dpdk'
> > > 12:08:13 Makefile:460: recipe for target 'ebuild-build' failed
> > > 12:08:13 make[2]: *** [ebuild-build] Error 2
> > > 12:08:13 make[2]: Leaving directory '/w/workspace/vpp-verify-master-
> > > opensuse/dpdk'
> > > 12:08:13 Makefile:682: recipe for target 'dpdk-build' failed
> > > 12:08:13 make[1]: *** [dpdk-build] Error 2
> > > 12:08:13 make[1]: Leaving directory '/w/workspace/vpp-verify-master-
> > > opensuse/build-root'
> > > 12:08:13 Makefile:333: recipe 

Re: [dpdk-users] DPDK Performance tips

[Thomas Monjalon]

13/12/2017 09:14, Anand Prasad:
>  Hi Dpdk team,
>Can anyone please share tips to get better DPDK performance? I have tried 
> to run DPDK test applications on various PC configurations with different CPU 
> Speeds, RAM Size/Speed and and PCIex16 2nd and 3rd generation connections... 
> but I don't get very consistent results.
>The same test application does not behave similarly on 2 PC's with same 
> hardware configuration.But general observation is that, performance is better 
> on Intel motherboard with 3rd generation PCIe slot. When tried on Gigabyte 
> motherboard (even with higher CPU and RAM speeds), performance was very poor.
> The performance issue I am facing is the Packet Drop issue on the Rx side.
>  Two PC's with exactly same hardware configuration, one PC drops packets 
> after few hours, but in the other PC I don't observe packet drop
> Highly appreciate a quick response
> Regards,Anand Prasad

This is very complicate because it really depends on the hardware.
Managing performance requires a very good knowledge of the hardware.

You can find some basic advices in this guide for some Intel hardware:
http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html
You will also find some informations in the drivers guide. Example:
http://dpdk.org/doc/guides/nics/mlx5.html#performance-tuning

Re: [dpdk-users] VF RSS availble in I350-T2?

[Thomas Monjalon]

12/12/2017 13:58, ..:
> I assume my message was ignored due to it not being related to dpdk
> software?

It is ignored because people have not read it or are not expert in
this hardware.
I am CC'ing the maintainer of igb/e1000.


> On 11 December 2017 at 10:14, ..  wrote:
> 
> > Hi,
> >
> > I have an intel I350-T2 which I use for SR-IOV, however, I am hitting some
> > rx_dropped on the card when I start increasing traffic. (I have got more
> > with the same software out of a identical bare metal system)
> >
> > I am using the Intel igb driver on Centos 7.2 (downloaded from Intel not
> > the driver installed with Centos), so the RSS parameters amongst others are
> > availbe to me
> >
> > This then led me to investigate the interrupts on the tx rx ring buffers
> > and I noticed that the interface (vfs enabled) only had on tx/rx queue. Its
> > distributed between   This is on the KVM Host
> >
> >  CPU0   CPU1   CPU2   CPU3   CPU4
> > CPU5   CPU6   CPU7   CPU8
> >  100:  1 33137  0  0
> > 0  0  0  0 IR-PCI-MSI-edge  ens2f1
> >  101:   2224  0  0   6309 178807
> > 0  0  0  0 IR-PCI-MSI-edge  ens2f1-TxRx-0
> >
> > Looking at my standard nic ethernet ports I see 1 rx and 4 rx queues
> >
> > On the VM I only get one tx one rx queue ( I know all the interrupts are
> > only using CPU0) but that is defined in our builds.
> >
> > egrep "CPU|ens11" /proc/interrupts
> >CPU0   CPU1   CPU2   CPU3   CPU4
> > CPU5   CPU6   CPU7
> >  34:  715885552  0  0  0  0
> > 0  0  0  0   PCI-MSI-edge  ens11-tx-0
> >  35:  559402399  0  0  0  0
> > 0  0  0  0   PCI-MSI-edge  ens11-rx-0
> >
> > I activated RSS in my card, and can set if, however if I use the param
> > max_vfs=n then it  defaults back to to 1 rx 1 tx queue per nic port
> >
> > [  392.833410] igb :07:00.0: Using MSI-X interrupts. 1 rx queue(s), 1
> > tx queue(s)
> > [  393.035408] igb :07:00.1: Using MSI-X interrupts. 1 rx queue(s), 1
> > tx queue(s)
> >
> > I have been reading some of the dpdk older posts and see that VF RSS is
> > implemented in some cards, does anybody know if its available in this card
> > (from reading it only seemed the 10GB cards)
> >
> > One of my plans aside from trying to create more RSS per VM is to add more
> > CPUS to the VM that are not isolated so that the rx and tx queues can
> > distribute their load a bit to see if this helps.
> >
> > Also is it worth investigating the VMDq options, however I understand this
> > to be less useful than SR-IOV which works well for me with KVM.
> >
> >
> > Thanks in advance,
> >
> > Rolando
> >
> 

Re: [dpdk-users] DPDK mlx4 PMD on Azure VM

[Thomas Monjalon]

Hi,

19/12/2017 08:14, Hui Ling:
> I installed DPDK 17.11 on Ubuntu 16.04. And I downloaded MLNX OFED
> 4.2-1.2.0.0 and installed up-stream libs with
> ./mlnxofedinstall --guest --dpdk --upstream-libs
> 
> MLX4 PMD in DPDK doesn't seem to work with lib from ubuntu repository
> and install OFED allows me to compile DPDK mlx4 PMD without any
> compilation problem.

The recommended setup is using Linux 4.14 with rdma-core v15, not OFED.
Please check this doc:

http://dpdk.org/doc/guides/nics/mlx4.html#current-rdma-core-package-and-linux-kernel-recommended

Re: [dpdk-users] DPDK mlx4 PMD on Azure VM

[Thomas Monjalon]

19/12/2017 08:14, Hui Ling:
> I installed DPDK 17.11 on Ubuntu 16.04. And I downloaded MLNX OFED
> 4.2-1.2.0.0 and installed up-stream libs with
> ./mlnxofedinstall --guest --dpdk --upstream-libs
> 
> MLX4 PMD in DPDK doesn't seem to work with lib from ubuntu repository
> and install OFED allows me to compile DPDK mlx4 PMD without any
> compilation problem.
> 
> Then I tried to see if the mlx4 PMD works or not by running:
> 
> root@myVM:
> ./build/app/testpmd -l 1-2 -n 4 -w 0003:00:02.0 -w 0004:00:02.0 --
> --rxq=2 --txq=2 -i
[...]
> Configuring Port 0 (socket 0)
> PMD: mlx4_rxq.c:811: mlx4_rx_queue_setup(): 0xde0740: MR creation
> failure: Operation not permitted

[...]
> I also tried to run DPDK 17.11 on Ubuntu 17.10. It didn't work either.
> the testpmd hangs during "configuring Port 0" forever.

So you see 2 different errors on Ubuntu 16.04 and 17.10.
What are the Linux kernel versions?

> Can someone from MS or Mellanox help me figure out why? and how to
> make mlx4 PMD work on Azure VM?

Mellanox will support you.

Re: [dpdk-users] attach/detach on secondary process

[Thomas Monjalon]

13/12/2017 18:09, Stephen Hemminger:
> Many DPDK drivers require that setup and initialization be done by
> the primary process. This is mostly to avoid dealing with concurrency since
> there can be multiple secondary processes.

I think we should consider this limitation as a bug.
We must allow a secondary process to initialize a device.
The race in device creation must be fixed.

Re: [dpdk-users] attach/detach on secondary process

[Thomas Monjalon]

13/12/2017 22:10, Stephen Hemminger:
> On Wed, 13 Dec 2017 22:00:48 +0100
> Thomas Monjalon <tho...@monjalon.net> wrote:
> 
> > 13/12/2017 18:09, Stephen Hemminger:
> > > Many DPDK drivers require that setup and initialization be done by
> > > the primary process. This is mostly to avoid dealing with concurrency 
> > > since
> > > there can be multiple secondary processes.  
> > 
> > I think we should consider this limitation as a bug.
> > We must allow a secondary process to initialize a device.
> > The race in device creation must be fixed.
> > 
> 
> Secondary processes should be able to do setup.
> But it is up to the application not to do it concurrently from multiple
> processes.

Yes there can be synchronization between processes.
But I think it is safer to fix the device creation race in ethdev.
Note that I am not talking about configuration concurrency,
but just race in probing.

Re: [dpdk-users] IPsec offload

[Thomas Monjalon]

Hi,

05/11/2017 09:32, Avi Cohen (A):
> Does the DPDK  support HW IPsec offload ?
> can DPDK configure the NIC/Network adapter to ipsec a specific flow ?

Yes, an experimental IPsec offload support is added in DPDK 17.11.
It is called rte_security:
http://dpdk.org/doc/guides/prog_guide/rte_security.html

The hardware support will probably be improved in DPDK 18.02.

Re: [dpdk-users] IPsec offload

[Thomas Monjalon]

06/11/2017 11:25, Avi Cohen (A):
> Thank you Thomas
> Is this was tested on a specific HW e.g.  Mellanox Innova  / other ?

Mellanox hardware will probably be supported in 18.02.
There are some experimental support for NXP DPAA2 and Intel ixgbe in DPDK 17.11.


> > -Original Message-
> > From: Thomas Monjalon [mailto:tho...@monjalon.net]
> > 
> > Hi,
> > 
> > 05/11/2017 09:32, Avi Cohen (A):
> > > Does the DPDK  support HW IPsec offload ?
> > > can DPDK configure the NIC/Network adapter to ipsec a specific flow ?
> > 
> > Yes, an experimental IPsec offload support is added in DPDK 17.11.
> > It is called rte_security:
> > http://dpdk.org/doc/guides/prog_guide/rte_security.html
> > 
> > The hardware support will probably be improved in DPDK 18.02.

Re: [dpdk-users] Building mlx4 drivers

[Thomas Monjalon]

Hi,

07/11/2017 21:49, Michael Sowka:
> Hello, I am experiencing some early difficulties in building the mlx4
> driver for a ConnectX-3 device.

Which version of DPDK?

[...]
> Again, i'm sticking with docs to not install anything outside of what
> MLNX_OFED provides in its packages, but where is that pesky mlx4dv.h header?

If you are compiling DPDK 17.11, the doc is not updated yet unfortunately.
You should try rdma-core and upstream kernel instead of MLNX_OFED.

Re: [dpdk-users] Does BONDING_MODE_8023AD is works in 17.11?

[Thomas Monjalon]

No reply after 2 months, adding maintainer Cc.

17/11/2017 23:24, Алексей Телятников:
> Greetings. I have issue with link-aggregation in DPDK. My HP A5820X 
> switch does not recive LACP from DPDK application.
> 
> Has anyone worked with BONDING_MODE_8023AD?
> 
> I have: DPDK 17.11 HEAD & 11.1-RELEASE-p4 FreeBSD
> 
> Also I build examples/bond with little changes:
> 
> --- a/examples/bond/main.c
> +++ b/examples/bond/main.c
> @@ -226,7 +226,7 @@ bond_port_init(struct rte_mempool *mbuf_pool)
>  uint16_t nb_rxd = RTE_RX_DESC_DEFAULT;
>  uint16_t nb_txd = RTE_TX_DESC_DEFAULT;
> 
> -   retval = rte_eth_bond_create("bond0", BONDING_MODE_ALB,
> +   retval = rte_eth_bond_create("net_bonding0", BONDING_MODE_8023AD,
>  0 /*SOCKET_ID_ANY*/);
>  if (retval < 0)
>  rte_exit(EXIT_FAILURE,
> 
> Run it:
> 
> $ sudo build/bond_app
> EAL: Sysctl reports 16 cpus
> EAL: Detected 16 lcore(s)
> EAL: Contigmem driver has 1 buffers, each of size 32GB
> EAL: Mapped memory segment 1 @ 0x8023ff000: physaddr:0x8, len 
> 34359738368
> EAL: PCI device :03:00.0 on NUMA socket 0
> EAL:   probe driver: 8086:1521 net_e1000_igb
> EAL:   :03:00.0 not managed by UIO driver, skipping
> EAL: PCI device :03:00.3 on NUMA socket 0
> EAL:   probe driver: 8086:1521 net_e1000_igb
> EAL:   :03:00.3 not managed by UIO driver, skipping
> EAL: PCI device :05:00.0 on NUMA socket 0
> EAL:   probe driver: 8086:10fb net_ixgbe
> EAL: PCI device :05:00.1 on NUMA socket 0
> EAL:   probe driver: 8086:10fb net_ixgbe
> User device list:
> PMD: ixgbe_dev_link_status_print():  Port 0: Link Down
> Port 0 MAC: 90:e2:ba:d0:a3:c4
> PMD: ixgbe_dev_link_status_print():  Port 1: Link Down
> Port 1 MAC: 90:e2:ba:d0:a3:c5
> EAL: Initializing pmd_bond for net_bonding0
> PMD: Using mode 4, it is necessary to do TX burst and RX burst at least 
> every 100ms.
> EAL: Create bonded device net_bonding0 on port 2 in mode 4 on socket 0.
> PMD: ixgbe_dev_link_status_print():  Port 0: Link Down
> PMD: ixgbe_dev_link_status_print():  Port 1: Link Down
> Port 2 MAC: 90:e2:ba:d0:a3:c4
> Starting lcore_main on core 1:0 Our IP:7.0.0.10
> bond6>show
> 90:e2:ba:d0:a3:c4
> 90:e2:ba:d0:a3:c5
> Active_slaves:0 packets received:Tot:0 Arp:0 IPv4:0
> 
> Active slaves always 0. I tried configure bond in testpmd. Same problem.
> 
> On the switch all ports is in up state but:
> 
> Received LACP Packets: 0 packet(s)
> 
> 

Re: [dpdk-users] Using Mellanox ConnectX-3 for DPDK

[Thomas Monjalon]

Hi,

06/02/2018 00:01, Rohan Gandhi:
> Hello All,
> 
> 1) I am trying to use Mellanox ConnectX-3 to use DPDK on my server
> (Ubuntu 16.04 kernel 4.13.0-32-generic).
> 
> 2) Using dpdk/usertools/dpdk-setup.sh, I can see that the Mellanox
> interface is dpdk compatible.
> 
> Network devices using DPDK-compatible driver
> 
> :01:00.0 'MT27520 Family [ConnectX-3 Pro] 1007' drv=vfio-pci 
> unused=igb_uio

Mellanox drivers do not use neither UIO nor VFIO.


> 3) When I try to use Moongen to use this interface, it returned an
> error that it cannot detect the device.
> 
> [INFO]  Found 0 usable devices:
> [FATAL] Lua error in task master
> ...ace/dpdk/moongen/MoonGen/build/../libmoon/lua/device.lua:100: there
> are only 0 ports, tried to configure port id 1
> 
> 
> The same setup works with my other Intel NIC. I am not sure what I am
> doing wrong. Can you please help?

Please read the documentation:
http://dpdk.org/doc/guides/nics/mlx4.html#usage-example

Re: [dpdk-users] Suggestion: DPDK "latest" Download Links

[Thomas Monjalon]

22/08/2018 11:58, Ferruh Yigit:
> On 8/20/2018 8:03 PM, Justin Parus wrote:
> > Hi,
> > 
> > I would like to propose adding DPDK "latest" download links. For example 
> > dpdk-latest-major and dpdk-latest-stable, would automatically update and 
> > point to the most recent date major and stable releases respectively. This 
> > would be useful for our tests as they would always be testing the latest 
> > packages automatically.
> 
> +1, looks good idea

I am not sure how useful it is.
When you change from one major version to another one,
you often have to adapt your code, so it cannot be automatic.

I see more values in having a link for the latest version of each stable branch.

Re: [dpdk-users] Apply patches from the mailing list

[Thomas Monjalon]

25/03/2018 16:31, long...@viettel.com.vn:
> From: "shreyansh jain" 
> > From: long...@viettel.com.vn
> > > A very basic question, but how do I apply some of the patches that
> > > were put on the dev mailing list to try it out? I already looked at
> > > the next- subtrees but apparently even major patch set such as the new
> > > packet framework/ip_pipeline is not in there (yet).
> > 
> > This is what I do:
> > 
> > 1. Access http://dpdk.org/dev/patchwork/project/dpdk/list/ and search for
> > patches from the author. This has all the patches posted to Mailing List
> > - with their state (that is, for example, superseded if a series has been
> > superseded with another version)> > 
> > 2. You have three options:
> >  a) Either select all patches (you will need to register/login) in a
> >  series and add to "bundle" and download that bundle as mbox b) Select
> >  individual patch and look for "download patch" or "download mbox" link
> >  and manually download them.> > 
> > OR, one I use most frequently:
> >  b) Copy the link to patch (for example,
> >  http://dpdk.org/dev/patchwork/patch/36473/) and append "mbox" to it
> >  (http://dpdk.org/dev/patchwork/patch/36473/mbox)> > 
> > Then,
> > 
> > $ wget  -O - | git am
> > 
> > One can easily make a script which can do the steps (1)>(2b) above based
> > on a given patch ID (last integer in the link to patch).


This script exists already. It is pwclient:
https://dpdk.org/dev/patchwork/help/pwclient/


> > Maybe there is a better and efficient way - this is just what I do. :)
> 
> After my email I figured out your 2a). But yeah having to register/login was 
> a nuisance 
> and not very intuitive especially for people like me who had had no 
> experience working
> with patchwork or mailing lists before.


The basic method is to have a decent email client which allows you to
download emails (i.e. patches). Then you can just apply them with git am.


> > > The contributor guideline only has sections for submitting patches to
> > > the mailing list, not pulling and applying patches for local testing.
> > > I know of dpdk patchwork but there are no instructions provided.
> > 
> > Maybe you can go ahead and send across a patch for a method you find best
> > and efficient. Others can add their way/suggestions and I am confident
> > Thomas would be happy to accept a documentation improvement patch.
> 
> Echoing this. A section in the contributor guideline just like the one I 
> followed when I
> pushed my first patch would be very helpful indeed.


Now that you have all the informations, and that you are interested in,
you are welcome to update the contributors guide and send a patch :)

Re: [dpdk-users] DPDK MLX5 Probe error

[Thomas Monjalon]

Hi,

03/04/2019 20:32, Mora Gamboa, Luis Eduardo:
> I'm not able to use the mlx5 pmd driver with some Mellanox NICs I have 
> installed on my server. The error I'm receiving during EAL initialization is:
> 
> et_mlx5: no Verbs device matches PCI device :03:00.0, are kernel drivers 
> loaded?
> 
> The DPDK version I'm currently using is: DPDK-STABLE-18.11
> 
> I have installed the OFED latest version:
> 
> mlnx-en-4.5-1.0.1.0-ubuntu16.04-x86_64
> 
> I have performed modprobe of the ib_uverbs kernel module

There is more configuration to be done when using an old distro with OFED.
You will find the information in this chapter of the doc:
http://doc.dpdk.org/guides/nics/mlx5.html#quick-start-guide-on-ofed-en

Re: [dpdk-users] devargs syntax to use MAC address for bond slaves

[Thomas Monjalon]

+Cc Chas, the maintainer of the bonding PMD

10/08/2019 18:42, Thomas Monjalon:
> Hi,
> 
> 08/08/2019 18:28, Greg O'Rawe:
> > Hi,
> > 
> > I'm using DPDK 17.11.5 to bond two interfaces via the vdev syntax, however 
> > the two interfaces are on the same NIC and share the same PCI address.
> > 
> > Is there a way to reference them using the devargs syntax by MAC address? 
> > It is discussed here https://patches.dpdk.org/patch/33808/
> > 
> > E.g.
> > class=eth,mac=00:11:22:33:44:55
> > 
> > I don't see this as an option in the bond PMD for slave interfaces even in 
> > the latest 19.05 release.
> 
> It is implemented at ethdev level as a port iterator:
>   http://git.dpdk.org/dpdk/commit/?id=8b9ea3b3ca
> 
> > I'm trying to set this using VPP e.g. normally:
> > vdev eth_bond0,mode=1,slave=PCI1,slave=PCI2
> 
> In order to use this syntax with bonding slaves,
> I think a patch is required.

Re: [dpdk-users] devargs syntax to use MAC address for bond slaves

[Thomas Monjalon]

Hi,

08/08/2019 18:28, Greg O'Rawe:
> Hi,
> 
> I'm using DPDK 17.11.5 to bond two interfaces via the vdev syntax, however 
> the two interfaces are on the same NIC and share the same PCI address.
> 
> Is there a way to reference them using the devargs syntax by MAC address? It 
> is discussed here https://patches.dpdk.org/patch/33808/
> 
> E.g.
> class=eth,mac=00:11:22:33:44:55
> 
> I don't see this as an option in the bond PMD for slave interfaces even in 
> the latest 19.05 release.

It is implemented at ethdev level as a port iterator:
http://git.dpdk.org/dpdk/commit/?id=8b9ea3b3ca

> I'm trying to set this using VPP e.g. normally:
> vdev eth_bond0,mode=1,slave=PCI1,slave=PCI2

In order to use this syntax with bonding slaves,
I think a patch is required.

Re: [dpdk-users] devargs syntax to use MAC address for bond slaves

[Thomas Monjalon]

19/08/2019 17:56, Greg O'Rawe:
> Hi,
> 
> Thanks for the reply. Is such a patch to the bonding PMD feasible?

Probably yes.
You can try to do it, and work with Chas for help.

Re: [dpdk-users] [dpdk-dev] Issue with DCB with X710 Quad 10Gb NIC

[Thomas Monjalon]

18/09/2019 09:02, Zhang, Xiao:
> 
> There is some hardware limitation and need to enable RSS to distribute 
> packets for X710.

Is this limitation documented?

Re: [dpdk-users] General Questions

[Thomas Monjalon]

09/04/2020 17:32, Stephen Hemminger:
> On Thu, 9 Apr 2020 20:57:19 +0530
> Shyam Shrivastav  wrote:
> 
> > From my experience as dpdk user
> > 
> > On Thu, Apr 9, 2020 at 11:41 AM Cristofer Martins <
> > cristofermart...@hotmail.com> wrote:  
> > 
> > > Well the reason i thought about using dpdk(together with a user space tcp
> > > stack) is because my tcp code spend so much time with syscalls that
> > > removing that would allow better throughput and latency. Is this a valid
> > > reason? My software runs in single core(and most of time in cheap vps) so 
> > > i
> > > want to extract the best i can from them.
> > >  
> > 
> > Yes using dpdk instead of getting packets from kernel stack increases
> > performance
> > 
> > 
> > 
> > > The other question is, can dpdk runs alongside with the linux network
> > > stack? I want to use dpdk in my special app but i still want to have ssh
> > > and apps working as expected without any modification.
> > >  
> > Interface used by dpdk is not available, at least another interface
> > required for management/access &  other network apps
> > 
> > 
> > 
> > >
> > > Thanks in advance.
> > >  
> 
> This might be a good use case for AF_XDP with or without DPDK

AF_XDP helps to use a device both with Linux stack and userland application.
This capability is what we call the bifurcated model.
The Mellanox drivers are also using a bifurcated model:
the same device can send some packet flows to the kernel interface,
and other (configured) packet flows to the DPDK interface.

Re: [dpdk-users] [mlx5 + DPDK 19.11] Flow insertion rate less than 4K per sec

[Thomas Monjalon]

Hi,

10/04/2020 20:11, Yan Lei:
> I am doing some study that requires inserting more than 1 million flow
> rules per second to the NIC. And I runs DPDK 19.11 on a ConnectX-5 NIC.
> 
> But I only managed to create around 3.3K rules per second.
> Below is the code I used to measure the insertion rate:

Please could you review this new application designed for such measure?
https://patches.dpdk.org/patch/68058/

Any feedback about the above patch is welcome. Feel free to try and review it.

Re: [dpdk-users] [mlx5 + DPDK 19.11] Flow insertion rate less than 4K per sec

[Thomas Monjalon]

+Cc Wisam

16/04/2020 17:32, Yan Lei:
> Hi Thomas,
> 
> 
> I tried the patch (68057 + 68058) on DPDK 19.11/20.02 + ofed 4.7.3.
> 
> 
> TL;DR
> 
> 
> 1. I was only able to generate 3K rules per second.
> 
> 2. The maximum number of distinct rules the NIC can support seems to be 65536.
> 
> 
> How can I increase the insertion rate? Any firmware/driver config I need to 
> tune? Also, is 65536 distinct flows truly a limit of the NIC? The patch 
> defaults to generate 4 million distinct flows though...
> 
> 
> Thanks in advance!
> 
> 
> 
> Initially, running
> 
> 
> ```
> 
> sudo ./flow_perf -l 3-7 -n 4 -w 02:00.0,dv_flow_en=1 -- --ingress --ether 
> --ipv4 --udp --queue --flows-count=100
> 
> ```
> 
> 
> failed after a few seconds and it gave
> 
> 
> ```
> Flow can't be created 1 message: hardware refuses to create flow
> EAL: Error - exiting with code: 1
>   Cause: error in creating flow
> ```
> 
> 
> Then I added a small debug patch (attached) and it showed that the error 
> happens when creating the 65536th flow rule.
> 
> 
> ```
> Flow can't be created 1 message: hardware refuses to create flow
> EAL: Error - exiting with code: 1
>   Cause: error in creating flow,flows generated: 65536
> ```
> 
> 
> My guess is that the NIC can only accept 65536 concurrent rules. Once I 
> changed the outer ip mask to 0x, the above command runs fine.
> 
> 
> To see how many rules I can generate per second. I ran (with the outer ip 
> mask 0x)
> 
> 
> ```
> 
> sudo ./flow_perf -l 3-7 -n 4 -w 02:00.0,dv_flow_en=1 -- --ingress --ether 
> --ipv4 --udp --queue --flows-count=65536
> 
> ```
> 
> 
> and it gives
> 
> 
> ```
> 
> :: Total flow insertion rate -> 3.015922 K/Sec
> :: The time for creating 65536 in flows 21.730005 seconds
> :: EAGAIN counter = 0
> ```
> So 3 rules per sec. Which is close to what I observed before.
> 
> ```
> sudo ./flow_perf -l 3-7 -n 4 -w 02:00.0,dv_flow_en=1 -- --ingress --ether 
> --ipv4 --udp --queue --flows-count=10
> ```
> gives
> 
> ```
> :: Total flow insertion rate -> 0.949381 K/Sec
> :: The time for creating 10 in flows 105.331842 seconds
> :: EAGAIN counter = 0
> ```
> Have no idea why it's only 1k/sec in this case...
> 
> Thanks and cheers,
> Lei
> 
> 
> 
> From: users  on behalf of Yan Lei 
> Sent: Tuesday, April 14, 2020 1:20 PM
> To: Thomas Monjalon
> Cc: users@dpdk.org
> Subject: Re: [dpdk-users] [mlx5 + DPDK 19.11] Flow insertion rate less than 
> 4K per sec
> 
> Hi Thomas,
> 
> Thanks! I will give it a try (using DPDK 19.11 + ofed 4.7.3).
> 
> Cheers,
> Lei
> 
> From: Thomas Monjalon 
> Sent: Tuesday, April 14, 2020 12:12:28 PM
> To: Yan Lei
> Cc: users@dpdk.org
> Subject: Re: [dpdk-users] [mlx5 + DPDK 19.11] Flow insertion rate less than 
> 4K per sec
> 
> Hi,
> 
> 10/04/2020 20:11, Yan Lei:
> > I am doing some study that requires inserting more than 1 million flow
> > rules per second to the NIC. And I runs DPDK 19.11 on a ConnectX-5 NIC.
> >
> > But I only managed to create around 3.3K rules per second.
> > Below is the code I used to measure the insertion rate:
> 
> Please could you review this new application designed for such measure?
> https://patches.dpdk.org/patch/68058/
> 
> Any feedback about the above patch is welcome. Feel free to try and review it.

Re: [dpdk-users] mlx5 PMD fails to receive certain icmpv6 multicast

[Thomas Monjalon]

06/03/2020 01:45, Liwu Liu:
> Hi Team,
> 
> I am using the mlx5/100G in KVM guest. The host shows this PCI vfNIC is 
> provisioned to the guest:
>   "17:01.1 Ethernet controller: Mellanox Technologies MT27800 Family 
> [ConnectX-5 Virtual Function]"
> 
> I am using DPDK 19.11 with kind of standard configurations, and when DPDK 
> application runs I still have the kernel mlx5e net device present. I have 
> both promiscuous and all-multicast turned on.
> 
> It works fine for IPV4, but for IPV6 it fails. It can receive packets 
> destined to 33:33:00:00:00:02 (IPV6 Router solicitation), but cannot receive 
> packets destined to 33:33:ff:00:00:01 (IPV6 neighbor solicitation for some 
> address).
> 
> But if I avoid DPDK, directly use the OFED-4.6 based kernel driver, 
> everything works fine as expected.
> 
> I am thinking there is some mismatch happened for MLX5 PMD. Please give some 
> advice/hints.

Adding Mellanox engineers in Cc list for help.

Re: [dpdk-users] rte_eth_stats_get: imiss is not set when using mlx4/mlx5 driver

[Thomas Monjalon]

Hi,

Sorry for the late answer.

22/10/2019 10:38, guyifan:
> DPDK version 18.11.2，imiss is always 0.
> And I could not find any code about 'imiss' in 
> 'dpdk-stable-18.11.2/drivers/net/mlx5/' or 
> 'dpdk-stable-18.11.2/drivers/net/mlx4/'.
> Is there any way to know how many packets have been dropped by a Mellanox NIC?

It is supported in DPDK 19.02:
http://git.dpdk.org/dpdk/commit/?id=ce9494d76c4783
and DPDK 18.11.3:
http://git.dpdk.org/dpdk-stable/commit/?h=81d0621264449ecc

Re: [dpdk-users] CX4-Lx VF link status in Azure

[Thomas Monjalon]

On 3/26/2020 12:00, Benoit Ganne (bganne) wrote:
> Just removing the over-strict check in mlx5 PMD is enough for everything to 
> work fine:
> https://gerrit.fd.io/r/c/vpp/+/26152/1/build/external/patches/dpdk_20.02/0002-mlx5-azure-workaround.patch
[...]
>  2) mlx5 PMD enforce that both link speed is defined and link is up to update 
> interface state

The original commit introducing this logic is:
http://git.dpdk.org/dpdk/commit/?id=cfee94752b8f8f

I would say that the first issue is a lack of comment in this code.

Second, as Benoit said, we should relax this requirement.
If the link speed is unknown, a second request can be tried, no more.

Benoit, feel free to submit a patch showing how you think it should behave.
Otherwise, I guess a maintainer of mlx5 will try to arrange it later.
Note: a patch (even not perfect) is usually speeding up resolution.

Thanks

Re: [dpdk-users] CX4-Lx VF link status in Azure

[Thomas Monjalon]

26/03/2020 21:09, Mark Bloch:
> 
> On 3/26/2020 12:00, Benoit Ganne (bganne) wrote:
> >> Pasting back this important info:
> >> "
> >> Note that ethtool and '/sys/class/net//speed' also fails
> >> to report the link speed (but not the link status).
> >> "
> >>
> >> 26/03/2020 19:27, Benoit Ganne (bganne):
> >>> Yes everything is initialized correctly. The netdev itself is configured
> >> and usable from Linux (ping etc.). Just removing the over-strict check in
> >> mlx5 PMD is enough for everything to work fine:
> >> https://gerrit.fd.io/r/c/vpp/+/26152/1/build/external/patches/dpdk_20.02/0002-mlx5-azure-workaround.patch
> >>> The link speed is unknown but this is not issue, and link state and
> >> other link info are correctly reported.
> >>> Thomas, any input regarding this behavior in mlx5 PMD?
> >>
> >> I am not aware about the lack of link speed info.
> >> It is probably not specific to ConnectX-4 Lx.
> >> I guess it happens only with Hyper-V?
> 
> Should be fixed by those 3 commits (last 1 one is just cosmetic):
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=dc392fc56f39a00a46d6db2d150571ccafe99734
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=c268ca6087f553bfc0e16ffec412b983ffe32fd4
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=2f5438ca0ee01a1b3a9c37e3f33d47c8122afe74

Thanks for the patches Mark.

> > For me there are 2 separate issues:
> >  1) Linux kernel driver does not report link speed in Azure for CX4-Lx in 
> > Ubuntu 18.04

1 looks to be addressed with patches above.

> >  2) mlx5 PMD enforce that both link speed is defined and link is up to 
> > update interface state

Yes we can look at this issue.

> > If (1) is fixed, (2) should work, but to me (2) is too strict
> > for no good reason: we do not really care about reported link speed,

I agree that link speed is less important than link status.

> > esp in a virtual environment it usually does not mean much,

Yes the link speed is shared between all VFs.

> > but we do care about link state.

Re: [dpdk-users] DPDK TX problems

[Thomas Monjalon]

Thanks for the interesting feedback.
It seems we should test this performance use case in our labs.


18/02/2020 09:36, Hrvoje Habjanic:
> On 08. 04. 2019. 11:52, Hrvoje Habjanić wrote:
> > On 29/03/2019 08:24, Hrvoje Habjanić wrote:
> >>> Hi.
> >>>
> >>> I did write an application using dpdk 17.11 (did try also with 18.11),
> >>> and when doing some performance testing, i'm seeing very odd behavior.
> >>> To verify that this is not because of my app, i did the same test with
> >>> l2fwd example app, and i'm still confused by results.
> >>>
> >>> In short, i'm trying to push a lot of L2 packets through dpdk engine -
> >>> packet processing is minimal. When testing, i'm starting with small
> >>> number of packets-per-second, and then gradually increase it to see
> >>> where is the limit. At some point, i do reach this limit - packets start
> >>> to get dropped. And this is when stuff become weird.
> >>>
> >>> When i reach peek packet rate (at which packets start to get dropped), i
> >>> would expect that reducing packet rate will remove packet drops. But,
> >>> this is not the case. For example, let's assume that peek packet rate is
> >>> 3.5Mpps. At this point everything works ok. Increasing pps to 4.0Mpps,
> >>> makes a lot of dropped packets. When reducing pps back to 3.5Mpps, app
> >>> is still broken - packets are still dropped.
> >>>
> >>> At this point, i need to drastically reduce pps (1.4Mpps) to make
> >>> dropped packets go away. Also, app is unable to successfully forward
> >>> anything beyond this 1.4M, despite the fact that in the beginning it did
> >>> forward 3.5M! Only way to recover is to restart the app.
> >>>
> >>> Also, sometimes, the app just stops forwarding any packets - packets are
> >>> received (as seen by counters), but app is unable to send anything back.
> >>>
> >>> As i did mention, i'm seeing the same behavior with l2fwd example app. I
> >>> did test dpdk 17.11 and also dpdk 18.11 - the results are the same.
> >>>
> >>> My test environment is HP DL380G8, with 82599ES 10Gig (ixgbe) cards,
> >>> connected with Cisco nexus 9300 sw. On the other side is ixia test
> >>> appliance. Application is run in virtual machine (VM), using KVM
> >>> (openstack, with sriov enabled, and numa restrictions). I did check that
> >>> VM is using only cpu's from NUMA node on which network card is
> >>> connected, so there is no cross-numa traffic. Openstack is Queens,
> >>> Ubuntu is Bionic release. Virtual machine is also using ubuntu bionic
> >>> as OS.
> >>>
> >>> I do not know how to debug this? Does someone else have the same
> >>> observations?
> >>>
> >>> Regards,
> >>>
> >>> H.
> >> There are additional findings. It seems that when i reach peak pps
> >> rate, application is not fast enough, and i can see rx missed errors
> >> on card statistics on the host. At the same time, tx side starts to
> >> show problems (tx burst starts to show it did not send all packets).
> >> Shortly after that, tx falls apart completely and top pps rate drops.
> >>
> >> Since i did not disable pause frames, i can see on the switch "RX
> >> pause" frame counter is increasing. On the other hand, if i disable
> >> pause frames (on the nic of server), host driver (ixgbe) reports "TX
> >> unit hang" in dmesg, and issues card reset. Of course, after reset
> >> none of the dpdk apps in VM's on this host does not work.
> >>
> >> Is it possible that at time of congestion DPDK does not release mbufs
> >> back to the pool, and tx ring becomes "filled" with zombie packets
> >> (not send by card and also having ref counter as they are in use)?
> >>
> >> Is there a way to check mempool or tx ring for "left-owers"? Is is
> >> possible to somehow "flush" tx ring and/or mempool?
> >>
> >> H.
> > After few more test, things become even weirder - if i do not free mbufs
> > which are not sent, but resend them again, i can "survive" over-the-peek
> > event! But, then peek rate starts to drop gradually ...
> >
> > I would ask if someone can try this on their platform and report back? I
> > would really like to know if this is problem with my deployment, or
> > there is something wrong with dpdk?
> >
> > Test should be simple - use l2fwd or l3fwd, and determine max pps. Then
> > drive pps 30%over max, and then return back and confirm that you can
> > still get max pps.
> >
> > Thanks in advance.
> >
> > H.
> >
> 
> I did receive few mails from users facing this issue, asking how it was
> resolved.
> 
> Unfortunately, there is no real fix. It seems that this issue is related
> to card and hardware used. I'm still not sure which is more to blame,
> but the combination i had is definitely problematic.
> 
> Anyhow, in the end, i did conclude that card driver have some issues
> when it is saturated with packets. My suspicion is that driver/software
> does not properly free packets, and then DPDK mempool becomes
> fragmented, and this causes performance drops. Restarting software
> releases pools, and restores proper functionality.
> 
> 

Re: [dpdk-users] CX4-Lx VF link status in Azure

[Thomas Monjalon]

27/03/2020 11:02, Benoit Ganne (bganne):
> > Second, as Benoit said, we should relax this requirement.
> > If the link speed is unknown, a second request can be tried, no more.
> > Benoit, feel free to submit a patch showing how you think it should
> > behave.
> > Otherwise, I guess a maintainer of mlx5 will try to arrange it later.
> > Note: a patch (even not perfect) is usually speeding up resolution.
> 
> I can do that, but I am not sure I understand the logic of this test to begin 
> with: looking into other PMD (mlx4, i40e), it seems to be the only one 
> worrying about updating link state only when "ready" for some not clear (to 
> me) definition of "ready".
> I'll tend to agree with other PMD here: if the syscalls did not failed we 
> should just update with what we know.
> Why was this test introduced and what did it fixed?

Unfortunately the reason was not documented.
I suggest we go with a patch from your understanding
and we'll test it in multiple conditions to validate nothing is broken.

Re: [dpdk-users] CX4-Lx VF link status in Azure

[Thomas Monjalon]

27/03/2020 18:26, Benoit Ganne (bganne):
> > Unfortunately the reason was not documented.
> > I suggest we go with a patch from your understanding
> > and we'll test it in multiple conditions to validate nothing is broken.
> 
> Done: http://mails.dpdk.org/archives/dev/2020-March/161096.html

Very good, thanks.

Re: [dpdk-users] mlx5 PMD fails to receive certain icmpv6 multicast

[Thomas Monjalon]

26/03/2020 22:11, Thomas Monjalon:
> 06/03/2020 01:45, Liwu Liu:
> > Hi Team,
> > 
> > I am using the mlx5/100G in KVM guest. The host shows this PCI vfNIC is 
> > provisioned to the guest:
> >   "17:01.1 Ethernet controller: Mellanox Technologies MT27800 Family 
> > [ConnectX-5 Virtual Function]"
> > 
> > I am using DPDK 19.11 with kind of standard configurations, and when DPDK 
> > application runs I still have the kernel mlx5e net device present. I have 
> > both promiscuous and all-multicast turned on.
> > 
> > It works fine for IPV4, but for IPV6 it fails. It can receive packets 
> > destined to 33:33:00:00:00:02 (IPV6 Router solicitation), but cannot 
> > receive packets destined to 33:33:ff:00:00:01 (IPV6 neighbor solicitation 
> > for some address).
> > 
> > But if I avoid DPDK, directly use the OFED-4.6 based kernel driver, 
> > everything works fine as expected.
> > 
> > I am thinking there is some mismatch happened for MLX5 PMD. Please give 
> > some advice/hints.
> 
> Adding Mellanox engineers in Cc list for help.

Any update to share please?

Re: [dpdk-users] Error while building DPDK version 19.11 over latest kernel version

[Thomas Monjalon]

01/10/2020 19:04, Klei rama:
> Hei,
> 
> I am trying to build DPDK version 19.11 in my ubuntu machine (18.04) with
> the latest kernel version 5.9. It gives me an error while I try to build
> it. The error is when I try to build linux/kni module.
> The error looks something like this:
> 
> *dpdk/kernel/linux/kni/kni_dev.h:104:30:* *error: *passing argument 1 of ‘
> *get_user_pages_remote*’ from incompatible pointer type [
> *-Werror=incompatible-pointer-types*]
> 
> 
> Is there any workaround?

Yes, it will be fixed with the backport of this patch:
http://git.dpdk.org/dpdk/commit/?id=87efaea6376c8

> Do I need to disable that module or should I
> downgrade the kernel version? I wanted to debug my application and see if
> it needs kni module but I did not not how to disable kni module. I tried
> common_base and common_linux under config directory but could not find the
> line which disable this module.

You can disable KNI in the config file:
CONFIG_RTE_KNI_KMOD=n

Re: [dpdk-users] Resolved Issues per release

[Thomas Monjalon]

12/08/2020 15:32, Jim Holland (jimholla):
> The release notes at https://doc.dpdk.org/guides-20.02/rel_notes/index.html 
> don't list the resolved issues/bugs. Is there a way to find what bugs are 
> resolved for each dpdk release?

When looking at the git history,
every commit fixing a bug should contain a line starting with "Fixes:"
identifying the root cause.

Catching such commits will give you all details of bugs fixed.

Re: [dpdk-users] Trouble Building DPDK with MLNX OFED(4.4) Works with 5.0

[Thomas Monjalon]

Hi,

06/06/2020 21:43, Vineeth Thapeta:
> Hi guys,
> 
> I had to roll back OFED version from 5.0 to 4.4 because another library
> Moongen has trouble building with OFED 5.0.

How Moongen is related to OFED?
Why using OFED and not upstream packaging of rdma-core?
Which distribution are you using?

> I rolled back the version and
> Moongen builds and when I try to build dpdk 18.11, 19.11,17.11. All the
> three builds fail with the following error (https://pastebin.com/fiw7iz1Z
> Link of build error). Does anyone know why this happens.

Any DPDK version should build with any version of OFED or rdma-core.
Note: DPDK 17.11 is not supported anymore.

> I am exhausted
> with roll backs,if anyone can help me with this I would be grateful. , By
> the looks of it, it might mean it had problems with the OFED driver? I
> installed the MLNX ofed driver without any flags, i.e ./mlnxofedinstall. If
> you want any further information,please let me know. Any suggestion would
> be appreciated, even if you don't think it might work. I have been trying
> to install these two for the past week, without much success. Are the above
> mentioned DPDK version not compatible with said driver(4.4)?

Let's focus on your goal and see what is the real issue you hit.
Which version of DPDK do you need?
Why not start with the latest LTS, which is 19.11?

Re: [dpdk-users] Trouble Building DPDK with MLNX OFED(4.4) Works with 5.0

[Thomas Monjalon]

06/06/2020 22:54, Vineeth Thapeta:
> Hi Thomas,
> 
> We want to use TAS which is built on top of DPDK,

Do you have a link to TAS? I don't know it.

> I installed Mellanox OFED 5.0 and TAS and everything was good,
> until we had to install Moogen.

So your only issue is to install Moongen,
given you already have DPDK working.

> Please find my answer to your questions.
> Thanks for the prompt reply. I appreciate it.
> 
> We are using Ubuntu 18.04. I indeed had success in installing dpdk 19.11
> LTS with MLNX_OFED 5.0. To answer your questions about Moongen either
> requires 1) Complete OFED drivers or 2) libibverbs, libmlx5,
> mlnx-ofed-kernel packages,

No, Moongen requires only DPDK.
If I am wrong, please could you show where you found such info?

> These packages are installed with MLNX_ofed
> drivers if I am not mistaken.

Installing the Ubuntu package rdma-core should be enough.

> So I moved on to build Moongen, When building
> moongen with OFED 5.0 I get the error fatal error: infiniband/mlx5_hw.h: No
> such file or directory #include . I thought this
> might be related to the libmlx5 and I installed libmlx5.deb from the
> packages in the DEBS/ folder. testpmd stops working with this prompt
> ./testpmd: /usr/lib/libmlx5.so.1: no version information available
> (required by ./testpmd). So I tried rolling back the OFED version to 4.4,
> which has caused problems with building dpdk with version 19.11 LTS,18.11
> LTS and 17.11 LTS as I shared in the previous mail.

You are mixing different versions of drivers.
Choose either rdma-core packaged in Ubuntu,
or OFED from mellanox.com.


> I hope this answers your questions.
> 
> Vineeth
> 
> On Sat, Jun 6, 2020 at 3:13 PM Thomas Monjalon  wrote:
> 
> > Hi,
> >
> > 06/06/2020 21:43, Vineeth Thapeta:
> > > Hi guys,
> > >
> > > I had to roll back OFED version from 5.0 to 4.4 because another library
> > > Moongen has trouble building with OFED 5.0.
> >
> > How Moongen is related to OFED?
> > Why using OFED and not upstream packaging of rdma-core?
> > Which distribution are you using?
> >
> > > I rolled back the version and
> > > Moongen builds and when I try to build dpdk 18.11, 19.11,17.11. All the
> > > three builds fail with the following error (
> > https://pastebin.com/fiw7iz1Z
> > > Link of build error). Does anyone know why this happens.
> >
> > Any DPDK version should build with any version of OFED or rdma-core.
> > Note: DPDK 17.11 is not supported anymore.
> >
> > > I am exhausted
> > > with roll backs,if anyone can help me with this I would be grateful. , By
> > > the looks of it, it might mean it had problems with the OFED driver? I
> > > installed the MLNX ofed driver without any flags, i.e ./mlnxofedinstall.
> > If
> > > you want any further information,please let me know. Any suggestion would
> > > be appreciated, even if you don't think it might work. I have been trying
> > > to install these two for the past week, without much success. Are the
> > above
> > > mentioned DPDK version not compatible with said driver(4.4)?
> >
> > Let's focus on your goal and see what is the real issue you hit.
> > Which version of DPDK do you need?
> > Why not start with the latest LTS, which is 19.11?

Re: [dpdk-users] meson: is there a mechanism for controlling compilation configuration

[Thomas Monjalon]

29/07/2020 09:20, Chengchang Tang:
> Hi,
> DPDK with 'make' will be deprecated in a future release. I have some
> questions about using meson to build DPDK.
> 
> When using the make, we can change the macros in config/common_base to
> control the compiling macros. For example, if i want to debug the mbuf,
> i can set CONFIG_RTE_LIBRTE_MBUF_DEBUG=y in config/common_base to change
> the compiling macros.
> 
> According to my understanding. DPDK meson build dose not generate
> rte_config.h based on common_base content during compilation. Is there
> any convenient way to modify the compiling macro in meson build. If all
> the compilation macros need to be modified using environment variable,
> it is inconvenient.

Please look at the work done in this series:
https://patches.dpdk.org/project/dpdk/list/?series=10930=*

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Thomas Monjalon]

11/12/2020 17:19, Slava Ovsiienko:
> From: Thomas Monjalon 
> > 09/12/2020 17:03, Alessandro Pagani:
> > > Hi all,
> > >
> > > I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5 driver).
> > >
> > > I am specifying the tx_pp parameter to provide the packet send
> > > scheduling on mbuf timestamps, but the testpmd fails with the following
> > error:
> > [...]
> > > EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: :3b:00.0
> 
> This is ConnectX-4LX (DevID is 1015), it does not support scheduling.
> Tx scheduling is supported since ConnectX-6DX. 
> 
> > > (socket 0)
> > > mlx5_pci: No available register for Sampler.
> > > mlx5_pci: WQE index ignore feature is required for packet pacing
> > > mlx5_pci: probe of PCI device :3b:00.0 aborted after encountering
> > > an
> > > error: No such device
> > > common_mlx5: Failed to load driver = mlx5_pci.
> > >
> > > EAL: Requested device :3b:00.0 cannot be used
> > [...]
> > > The error messages suggest that "WQE index ignore feature is required
> > > for packet pacing".
> > >
> > > Anyone knows the reason of this error and how to solve it?
> > 
> > I think it means your device does not support this feature.
> > But I realize it is not documented here:
> 
> Yes, indeed. I'll provide the patch, thank you for noticing that.

I think we should also improve the error message to
something like "not supported on this device".

Re: [dpdk-users] Questions about public IP included in DPDK driver code

[Thomas Monjalon]

+Cc ark maintainers

07/12/2020 15:17, HuangLiming:
> I found that the driver contains public IP(169.254.10.240), which leads to my 
> binary
> with public IP too, which is not what I want.
> What is the function of this IP, and could the community delete it for 
> security reasons?
> 
> drivers/net/ark/ark_pktgen.c: {{"dst_ip"}, OTSTRING, .v.STR = 
> "169.254.10.240"},
> drivers/net/ark/ark_pktchkr.c:{{"dst_ip"}, OTSTRING, .v.STR = 
> "169.254.10.240"}

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Thomas Monjalon]

09/12/2020 17:03, Alessandro Pagani:
> Hi all,
> 
> I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5 driver).
> 
> I am specifying the tx_pp parameter to provide the packet send scheduling
> on mbuf timestamps, but the testpmd fails with the following error:
[...]
> EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: :3b:00.0 (socket 0)
> mlx5_pci: No available register for Sampler.
> mlx5_pci: WQE index ignore feature is required for packet pacing
> mlx5_pci: probe of PCI device :3b:00.0 aborted after encountering an
> error: No such device
> common_mlx5: Failed to load driver = mlx5_pci.
> 
> EAL: Requested device :3b:00.0 cannot be used
[...]
> The error messages suggest that "WQE index ignore feature is required for
> packet pacing".
> 
> Anyone knows the reason of this error and how to solve it?

I think it means your device does not support this feature.
But I realize it is not documented here:
http://doc.dpdk.org/guides/nics/mlx5.html#supported-hardware-offloads

Re: [dpdk-users] Changes to DPDK kmod drivers / backward compatibility within LTS rel.

[Thomas Monjalon]

24/09/2020 19:12, Alexander Kotliarov:
> Hi there!
> I would like to find out what is the policy regarding changes to the DPDK's
> kmod drivers such as igb_uio.ko within a DPDK's LTS release. Are these
> changes backward compatible?
> For example, is there a guarantee that an application built
> against 19.11.5, where igb_uio.ko received changes,would run with this
> driver built from 19.11.1 version?
> 
> Does http://doc.dpdk.org/guides/contributing/stable.html section 8.4 apply
> to kmod drivers as well?

There is no such formal guarantee, but in my opinion,
it should be the case. Do you imagine a change in kmod
which could break a DPDK version?

Re: [dpdk-users] Does vmxnet3 PMD supports LSC=1 ?

[Thomas Monjalon]

+Cc Yong Wang, maintainer of this PMD.

14/01/2021 18:30, madhukar mythri:
> Hi,
> 
> Does vmxnet3 PMD support LSC=1(i.e with interrupt mode) for link changes ?
> 
> When i enable LSC=1 the functionality works fine, but, when pumping traffic
> i'm seeing increasing in CPU load on some cores which is running
> "eal-intr-thread" epoll_wait() function for more CPU-time.
> 
> Actually, interrupt should come only when Link changes, but, we are seeing
> interrupt for each incoming Rx-packet and also a lot of spurious interrupts.
> =
> ~ # cat /proc/interrupts |grep igb
>  58:1254293  0  0  0   PCI-MSI 1572864-edge
>  igb_uio
>  59:1278105  0  0  0   PCI-MSI 5767168-edge
>  igb_uio
> ~ # cat /proc/irq/58/spurious
> count 98035
> unhandled 0
> last_unhandled 0 ms
> ~ #
> ==
> 
> Does anyone tried LSC=1 in vmxnet3 PMD based apps and faced similar issues
> ? If so, please let me know.
> 
> Tried with DPDK-18.11, DPDK-19.11 and DPDK-20.05.
> 
> Thanks,
> Madhukar.

Re: [dpdk-users] DPDK: MPLS packet processing

[Thomas Monjalon]

18/01/2021 09:46, Raslan Darawsheh:
> From: raktim bhatt
> 
> > Hi All,
> > 
> > I am trying to build a multi-RX-queue dpdk program, using RSS to split the
> > incoming traffic into RX queues on a single port. Mellanox ConnectX-5 and
> > DPDK Version 19.11 is used for this purpose. It works fine when I use IP
> > over Ethernet packets as input. However when the packet contains IP over
> > MPLS over Ethernet, RSS does not seem to work. As a result, all packets
> > belonging to various flows (with different src & dst IPs, ports over MPLS)
> > are all sent into the same RX queue.
> > 
> > 
> > My queries are
> > 
> > 1. Is there any parameter/techniques in DPDK to distribute MPLS packets to
> > multiple RX queues?
> > 
> I've tried it over my setup with testpmd:
> ./build/app/dpdk-testpmd -n 4 -w :08:00.0 -- --mbcache=512 -i 
> --nb-cores=27 --rxq=4 --txq=4 --rss-ip
> testpmd> set verbose 1
> testpmd> start
> 
> then tried to send two MPLS packets with different src IP:
> packet1 =  Ether()/MPLS()/IP(src='1.1.1.1')
> packet2 =  Ether()/MPLS()/IP(src='1.1.1.2')
> 
> and I see that both packets are being spread over the queues, see the bellow 
> testpmd dump output:
> testpmd> port 0/queue 3: received 1 packets
>   src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 - 
> nb_segs=1 - RSS hash=0x43781943 - RSS queue=0x3 - hw ptype: L2_ETHER 
> L3_IPV4_EXT_UNKNOWN L4_NONFRAG  - sw ptype: L2_ETHER  - l2_len=14 - Receive 
> queue=0x3
>   ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_GOOD 
> PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> port 0/queue 1: received 1 packets
>   src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 - 
> nb_segs=1 - RSS hash=0xb8631e05 - RSS queue=0x1 - hw ptype: L2_ETHER 
> L3_IPV4_EXT_UNKNOWN L4_NONFRAG  - sw ptype: L2_ETHER  - l2_len=14 - Receive 
> queue=0x1
>   ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_GOOD 
> PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> 
> first packet was received on queue 3 and the second one was received over 
> queue 1,
> by the way, this is with both 19.11.0 and v19.11.6 
> 
> > 2. Is there any way to strip off MPLS tags (between Eth and IP) in
> > hardware, something like hw_vlan_strip?
> > 
> For this I'm not sure we have such thing in dpdk maybe Thomas can confirm 
> this here?

Look for "POP_MPLS" in rte_flow.

Re: [dpdk-users] MLX5: Using packet send scheduling / packet pacing

[Thomas Monjalon]

30/01/2021 11:54, Carsten Andrich:
> Hi Slava,
> 
> thank you for the prompt response. I think the requirements for Packet 
> Pacing should be added to Table 34.2(?) of the MLX5 docs [1].

+1 for improving the doc

Re: [dpdk-users] Unable to setup hugepages

[Thomas Monjalon]

31/05/2021 17:35, Gabriel Danjon:
> Hello,
> 
> After successfully installed the DPDK 20.11 on my Centos 8-Stream 
> (minimal), I am trying to configure the hugepages but encounters a lot 
> of difficulties.

There's some confusing info below.
Let's forget all the details and focus on simple things:
1/ use dpdk-hugepages.py
2/ choose one page size (2M or 1G)
3/ check which node requires memory with lstopo
4/ don't be confused with warnings about unused page size



> I am trying to reserve 4 hugepages of 1GB.
> 
> 
> Here the steps I have done following the documentation 
> (https://doc.dpdk.org/guides-20.11/linux_gsg/sys_reqs.html):
> 
> Additional information about meminfo :
> 
> cat /proc/meminfo
> MemTotal:   32619404 kB
> MemFree:27331024 kB
> MemAvailable:   27415524 kB
> Buffers:4220 kB
> Cached:   328628 kB
> SwapCached:0 kB
> Active:   194828 kB
> Inactive: 210156 kB
> Active(anon):   1744 kB
> Inactive(anon):83384 kB
> Active(file): 193084 kB
> Inactive(file):   126772 kB
> Unevictable:   0 kB
> Mlocked:   0 kB
> SwapTotal:  16474108 kB
> SwapFree:   16474108 kB
> Dirty: 0 kB
> Writeback: 0 kB
> AnonPages: 72136 kB
> Mapped:84016 kB
> Shmem: 12992 kB
> KReclaimable: 211956 kB
> Slab: 372852 kB
> SReclaimable: 211956 kB
> SUnreclaim:   160896 kB
> KernelStack:9120 kB
> PageTables: 6852 kB
> NFS_Unstable:  0 kB
> Bounce:0 kB
> WritebackTmp:  0 kB
> CommitLimit:30686656 kB
> Committed_AS: 270424 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:   0 kB
> VmallocChunk:  0 kB
> Percpu:28416 kB
> HardwareCorrupted: 0 kB
> AnonHugePages: 10240 kB
> ShmemHugePages:0 kB
> ShmemPmdMapped:0 kB
> FileHugePages: 0 kB
> FilePmdMapped: 0 kB
> HugePages_Total:   0
> HugePages_Free:0
> HugePages_Rsvd:0
> HugePages_Surp:0
> Hugepagesize:1048576 kB
> Hugetlb: 4194304 kB
> DirectMap4k:  225272 kB
> DirectMap2M: 4919296 kB
> DirectMap1G:30408704 kB
> 
> 1 Step follow documentation
> 
> bash -c 'echo 2048 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages'
> 
> As we're working on a NUMA machine we do this too. (We even do the 
> previous step because without it, it provides more errors)
> 
> bash -c 'echo 2048 > 
> /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages' && \
> bash -c 'echo 2048 > 
> /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages'
> 
> mkdir /mnt/huge
> mount -t hugetlbfs pagesize=1GB /mnt/huge
> 
> bash -c 'echo nodev /mnt/huge hugetlbfs pagesize=1GB 0 0 >> /etc/fstab'
> 
> So here the result of my meminfo (cat /proc/meminfo | grep Huge) :
> 
> AnonHugePages: 10240 kB
> ShmemHugePages:0 kB
> FileHugePages: 0 kB
> HugePages_Total:   0
> HugePages_Free:0
> HugePages_Rsvd:0
> HugePages_Surp:0
> Hugepagesize:1048576 kB
> Hugetlb: 4194304 kB
> 
> It looks strange that there is no total and free hugepages.
> 
> I tried the dpdk-testpmd using the DPDK documentation : dpdk-testpmd -l 
> 0-3 -n 4 -- -i --nb-cores=2
> 
> EAL: Detected 48 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: 2048 hugepages of size 2097152 reserved, but no mounted hugetlbfs 
> found for that size
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: No available hugepages reported in hugepages-1048576kB
> EAL: FATAL: Cannot get hugepage information.
> EAL: Cannot get hugepage information.
> EAL: Error - exiting with code: 1
>Cause: Cannot init EAL: Permission denied
> 
> 
> So I checked in the /mnt/huge to look if files had been created (ls 
> /mnt/huge/ -la) : Empty folder
> 
> Then I checked if my folder was correctly mounted : mount | grep huge
> pagesize=1GB on /mnt/huge type hugetlbfs 
> (rw,relatime,seclabel,pagesize=1024M)
> 
> Then I tried the helloworld example (make clean && make && 
> ./build/helloworld):
> 
> EAL: Detected 48 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Detected shared linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: 2048 hugepages of size 2097152 reserved, but no mounted hugetlbfs 
> found for that size
> EAL: No free 1048576 kB hugepages reported on node 0
> EAL: No free 1048576 kB hugepages reported on node 1
> EAL: No available 1048576 kB hugepages reported
> EAL: FATAL: Cannot get hugepage information.
> EAL: Cannot get hugepage information.
> PANIC in main():
> Cannot init EAL
> 5: [./build/helloworld() [0x40079e]]
> 4: 

Re: [dpdk-users] Enable to install DPDK on Centos 8-Stream

[Thomas Monjalon]

27/05/2021 18:10, Gabriel Danjon:
> Hello,
> 
> I am having difficulties to compile and install DPDK from sources on the 
> latest Centos 8-Stream.

Did you compile and install DPDK successfully?
Where is it installed?

> 
> After having installed the required drivers and libraries, following the 
> documentation and the DPDK build (meson build && cd build && ninja && 
> ninja install && ldconfig), I tried to compile the helloworld example 
> without success:
> 'Makefile:12: *** "no installation of DPDK found".  Stop.'
> 
> 
> Please find attached to this mail some logs.

The log is way too long to be read.
Please copy only what is relevant.

> Could you provide help please ?

It looks to be basic issue with library installation.
Did you read the doc?
http://doc.dpdk.org/guides/linux_gsg/build_dpdk.html

Especially this note:
"
On some linux distributions, such as Fedora or Redhat, paths in /usr/local are 
not in the default paths for the loader. Therefore, on these distributions, 
/usr/local/lib and /usr/local/lib64 should be added to a file in 
/etc/ld.so.conf.d/ before running ldconfig.
"

Re: [dpdk-users] Question regarding DPDK on AWS

[Thomas Monjalon]

30/06/2021 16:40, Antonis Christodoulou:
> Hello all,

Hello,

> my name is Antonis Christodoulou, and I am a new user of DPDK. I am not sure 
> this is the right place to ask a usage question, so please feel free to 
> redirect me to someone more appropriate for such questions as needed.

Your question below looks related to the TCP stack in F-stack, not DPDK itself.
I would recommend to get in touch with the F-Stack project:
https://github.com/F-Stack/f-stack/issues


> I am working on AWS, and I have set up F-stack over DPDK, successfully 
> connecting to an address within my private VPC, using a client socket. On the 
> other side I am running a simple echo server with  ncat -l 2001 -k -c 'xargs 
> -n1 echo' -vvv. However, when I just change the address to some global IP, 
> like the one used by www.example.com, ie.  
> 93.184.216.34 (I used port 80), then I am not getting any socket connection.
> 
> Would you know why this is happening? I have not set up any veth0 interface 
> yet for the DPDK NIC, I am not sure this is needed for connectivity.
> 
> Regards,
> Antonis

Re: [dpdk-users] MLX5: Using packet send scheduling / packet pacing

[Thomas Monjalon]

+Cc Slava

29/01/2021 17:30, Carsten Andrich:
> Hello everyone,
> 
> I'm trying to use packet send scheduling [1] with DPDK 20.11 and the 
> MLX5 PMD (NIC: ConnectX-5 MCX516A-CDAT). This patch contains some 
> additional information on this feature also know as packet pacing [2].
> 
> According to MLX5's docs, packet pacing requires the "tx_pp" parameter 
> [3, CTRL+F: "tx_pp"]. However, when firing up testpmd with that 
> parameter, it fails as follows:
> 
> > # dpdk-testpmd -a 81:00.0,tx_pp=500 -- -i
> > ...
> > EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: :81:00.0 (socket 0)
> > mlx5_pci: WQE rate mode is required for packet pacing
> > mlx5_pci: probe of PCI device :81:00.0 aborted after encountering an 
> > error: No such device
> > common_mlx5: Failed to load driver = mlx5_pci.
> >
> > EAL: Requested device :81:00.0 cannot be used
> The error message originates here [4] and is caused by what to me 
> appears to be a value read from the NIC [5]. Unfortunately, that leaves 
> me clueless on how to activate the required "WQE rate mode".  According 
> to the output of ibv_devinfo, my NIC does support packet pacing:
> 
> > # ibv_devinfo -v 81:00.0
> > ...
> > packet_pacing_caps:
> > qp_rate_limit_min:  1kbps
> > qp_rate_limit_max:  1kbps
> > supported_qp:
> > SUPPORT_RAW_PACKET
> I'd be grateful for any information on how to get packet pacing up and 
> running. Am I just missing another required option (which is not given 
> in the docs) or does my NIC lack packet pacing support?
> 
> Thank you very much in advance.
> 
> Best regards,
> Carsten
> 
> [1] 
> https://doc.dpdk.org/api/rte__ethdev_8h.html#a990d8351447a710628cbb24a28d3252d
> [2] https://patches.dpdk.org/patch/73742/
> [3] https://doc.dpdk.org/guides/nics/mlx5.html#run-time-configuration
> [4] 
> http://code.dpdk.org/dpdk/v20.11/source/drivers/net/mlx5/linux/mlx5_os.c#L1278
> [5] 
> http://code.dpdk.org/dpdk/v20.11/source/drivers/common/mlx5/mlx5_devx_cmds.c#L748
> 
> 

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Thomas Monjalon]

09/03/2021 05:11, Jinkun Geng:
> Too bad. :<

Why is it too bad?
How the choice of an internal build system
can affect other projects?


From: Stephen Hemminger 
> On Tue, 9 Mar 2021 01:32:16 +
> Jinkun Geng  wrote:
> 
> > Hi, all.
> > Since bazel building system is becoming more and more popular, sometimes we 
> > need to integrate DPDK library into a bazel project. However, it seems 
> > there is no much support for bazel from DPDK community.

Why the DPDK community would support building with Bazel?
What is the benefit?
Bazel projects cannot just link with DPDK using pkg-config?


> > The only support at https://github.com/bazelment/dpdk has been outdated. 
> > Based on our experience, it can only compile successfully with dpdk-16.04 
> > (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but 
> > the bazel support fails to catch up.
> >
> > It would be great if the experts in DPDK community can provide some 
> > portable BUILD files to facilitate the integration of the newest DPDK into 
> > bazel project (just like bazelment). After all, writing the bazel files can 
> > be really challenging, especially if we do not have a very deep 
> > understanding of the whole DPDK codes.
> >
> > Jinkun
> 
> DPDK is on meson now. The core team is unlikely to change build systems again.

DPDK supports library standards for compiling, installing and linking.
What else is needed?

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Thomas Monjalon]

11/03/2021 00:42, Jinkun Geng:
> For any project using bazel, if we want to use DPDK, then we need to compile 
> DPDK stuff into bazel by ourselves. It is not a trivial thing and the 
> bazelment (https://github.com/bazelment/dpdk) guys have spent much effort 
> extracting the core files in DPDK and write the BUILD files for DPDK. But now 
> it seems they have stopped maintaining that repo since DPDK 16.04. Even in 
> that version, it has some runtime failure when we use DPDK in our bazel 
> project.

Sorry I don't know bazel.
Do you mean all components must be built inside bazel?
You are not able to call meson/ninja commands from bazel?
How do you do with other libraries?
Does bazel usually reimplement what is packaged with autotools?


> ________
From: Thomas Monjalon 
> 09/03/2021 05:11, Jinkun Geng:
> > Too bad. :<
> 
> Why is it too bad?
> How the choice of an internal build system
> can affect other projects?
> 
> 
> From: Stephen Hemminger 
> > On Tue, 9 Mar 2021 01:32:16 +
> > Jinkun Geng  wrote:
> >
> > > Hi, all.
> > > Since bazel building system is becoming more and more popular, sometimes 
> > > we need to integrate DPDK library into a bazel project. However, it seems 
> > > there is no much support for bazel from DPDK community.
> 
> Why the DPDK community would support building with Bazel?
> What is the benefit?
> Bazel projects cannot just link with DPDK using pkg-config?
> 
> 
> > > The only support at https://github.com/bazelment/dpdk has been outdated. 
> > > Based on our experience, it can only compile successfully with dpdk-16.04 
> > > (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but 
> > > the bazel support fails to catch up.
> > >
> > > It would be great if the experts in DPDK community can provide some 
> > > portable BUILD files to facilitate the integration of the newest DPDK 
> > > into bazel project (just like bazelment). After all, writing the bazel 
> > > files can be really challenging, especially if we do not have a very deep 
> > > understanding of the whole DPDK codes.
> > >
> > > Jinkun
> >
> > DPDK is on meson now. The core team is unlikely to change build systems 
> > again.
> 
> DPDK supports library standards for compiling, installing and linking.
> What else is needed?

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Thomas Monjalon]

11/03/2021 01:17, Jinkun Geng:
> Do you mean all components must be built inside bazel?
> >> Sort of. We have a project that is built on bazel. Now, we need to use the 
> >> core functions of DPDK to replace the network primitives in our project, 
> >> so that we can improve the performance of our project.

You don't need to compile DPDK inside your project.
Just install the package or compile it externally.

> You are not able to call meson/ninja commands from bazel?
> >> I have found any examples using meson inside bazel. According to my 
> >> understanding, Bazel and meson are two parallel building systems and I 
> >> haven't seen anyone could use them together.
> 
> How do you do with other libraries? Does bazel usually reimplement what is 
> packaged with autotools?
> >> That is the reason why https://github.com/bazelment exists. If some 
> >> libraries are popular and do not support bazel, then these guys will help 
> >> generate some modified version of these libraries, so that developers can 
> >> integrate the library into bazel. It is not completely reimplementation, 
> >> but it indeed needs much extraction to adapt the previous library so that 
> >> it can be used in bazel.
> Unfortunately, these guys stop supporting DPDK atop bazel now.

That's crazy trying to replace the build system of projects.
Even if you find new volunteers, it will not be always up-to-date.

You should look for how to link an existing library from bazel.


> 
From: Thomas Monjalon 
> 11/03/2021 00:42, Jinkun Geng:
> > For any project using bazel, if we want to use DPDK, then we need to 
> > compile DPDK stuff into bazel by ourselves. It is not a trivial thing and 
> > the bazelment (https://github.com/bazelment/dpdk) guys have spent much 
> > effort extracting the core files in DPDK and write the BUILD files for 
> > DPDK. But now it seems they have stopped maintaining that repo since DPDK 
> > 16.04. Even in that version, it has some runtime failure when we use DPDK 
> > in our bazel project.
> 
> Sorry I don't know bazel.
> Do you mean all components must be built inside bazel?
> You are not able to call meson/ninja commands from bazel?
> How do you do with other libraries?
> Does bazel usually reimplement what is packaged with autotools?
> 
> 
> > 
> From: Thomas Monjalon 
> > 09/03/2021 05:11, Jinkun Geng:
> > > Too bad. :<
> >
> > Why is it too bad?
> > How the choice of an internal build system
> > can affect other projects?
> >
> >
> > From: Stephen Hemminger 
> > > On Tue, 9 Mar 2021 01:32:16 +
> > > Jinkun Geng  wrote:
> > >
> > > > Hi, all.
> > > > Since bazel building system is becoming more and more popular, 
> > > > sometimes we need to integrate DPDK library into a bazel project. 
> > > > However, it seems there is no much support for bazel from DPDK 
> > > > community.
> >
> > Why the DPDK community would support building with Bazel?
> > What is the benefit?
> > Bazel projects cannot just link with DPDK using pkg-config?
> >
> >
> > > > The only support at https://github.com/bazelment/dpdk has been 
> > > > outdated. Based on our experience, it can only compile successfully 
> > > > with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed 
> > > > to DPDK 21.02, but the bazel support fails to catch up.
> > > >
> > > > It would be great if the experts in DPDK community can provide some 
> > > > portable BUILD files to facilitate the integration of the newest DPDK 
> > > > into bazel project (just like bazelment). After all, writing the bazel 
> > > > files can be really challenging, especially if we do not have a very 
> > > > deep understanding of the whole DPDK codes.
> > > >
> > > > Jinkun
> > >
> > > DPDK is on meson now. The core team is unlikely to change build systems 
> > > again.
> >
> > DPDK supports library standards for compiling, installing and linking.
> > What else is needed?
> 
> 
> 
> 
> 
> 

Re: [dpdk-users] All links down with Chelsio T6 NICs

[Thomas Monjalon]

+Cc Chelsio maintainer

09/04/2021 19:24, Danushka Menikkumbura:
> Hello,
> 
> When I run testpmd on a system with 2 two-port Chelsio T6 NICs, the link
> status is down for all four ports. I use igb_uio as the kernel driver.
> Below is my testpmd commandline and the startup log.
> 
> sudo ./build/app/dpdk-testpmd -l 0,1,2,5 -b 81:00.0 -- -i
> 
> EAL: Detected 20 lcore(s)
> EAL: Detected 4 NUMA nodes
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: No available 1048576 kB hugepages reported
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: Probe PCI driver: net_cxgbe (1425:6408) device: :05:00.4 (socket 0)
> rte_cxgbe_pmd: Maskless filter support disabled. Continuing
> EAL: Probe PCI driver: net_cxgbe (1425:6408) device: :0b:00.4 (socket 0)
> rte_cxgbe_pmd: Maskless filter support disabled. Continuing
> Interactive-mode selected
> testpmd: create a new mbuf pool : n=171456, size=2176, socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
> testpmd: create a new mbuf pool : n=171456, size=2176, socket=2
> testpmd: preferred mempool ops selected: ring_mp_mc
> Configuring Port 0 (socket 0)
> Port 0: 00:07:43:5D:4E:60
> Configuring Port 1 (socket 0)
> Port 1: 00:07:43:5D:4E:68
> Configuring Port 2 (socket 0)
> Port 2: 00:07:43:5D:51:00
> Configuring Port 3 (socket 0)
> Port 3: 00:07:43:5D:51:08
> Checking link statuses...
> Done
> testpmd>
> 
> Your help is very much appreciated.

Please run the command "show port summary all"

Re: Troubles with building of DPDK.RPM and meson

[Thomas Monjalon]

12/10/2021 08:45, Ruslan R. Laishev:
> Hello All!
> 
>   I have a small study task to make DPDK.RPM with the latest DPDK from 
> the git.

You should look at what is done in Fedora for the latest DPDK.

>   I use DPDK.SPEC from the latest .SRC.RPM kit as a template.

Not sure what is your reference.

Re: [dpdk-users] DPDK CPU selection criteria?

[Thomas Monjalon]

17/09/2021 17:23, Jared Brown:
> Hello everybody!
> 
> Is there some canonical resource or at least a recommendation on how to 
> evaluate different CPUs for suitability for use with DPDK?

There are some performance reports with details:
https://core.dpdk.org/perf-reports/

It is difficult to give any generic info
because it depends on CPU architecture, use case, and application.
Please keep in mind there are hundreds of API functions so we cannot
have clues for all combinations.

> My use case is software routers (for example DANOS, 6WIND, TNRS), so I am 
> mainly interested in forwarding performance in terms of Mpps.
> 
> What I am looking for is to develop some kind of heuristics to evaluate CPUs 
> in terms of $/Mpps without having to purchase hundreds of SKUs and running 
> tests on them.
> 
> The official DPDK documentation[0] states thus:
> 
> "7.1. Hardware and Memory Requirements
> 
> For best performance use an Intel Xeon class server system such as Ivy 
> Bridge, Haswell or newer."
> 
> This is somewhat... vague.

This is only for Intel platforms.
DPDK is supported on AMD and Arm CPUs as well.

Are you interested only in Intel CPU?
All your questions below are interesting but are very hardware-specific.
+Cc few Intel engineers for specific questions.

> I suppose one could take [1] as a baseline, which states on page 2 that an 
> Ivy Bridge Xeon E3-1230 V2 is able to forward unidirectional flows at 
> linerate using 10G NICs at all frequencies above 1.6 GHz and bidirectional 
> flows at linerate using 10G NICs at 3.3 GHz.
> 
> This however pales compared with [2] that on page 23 shows that a 3rd 
> Generation Scalable Xeon 8380 manages to very nearly saturate a 100G NIC at 
> all packet sizes.
> 
> As there is almost a magnitude in difference in forwarding performance per 
> core, you can perhaps understand that I am somewhat at a loss when trying to 
> gauge the performance of a particular CPU model.
> 
> Reading [3] one learns that several aspects of the CPU affect the forwarding 
> performance, but very little light is shed on how much each feature on its 
> own contributes. On page 172 one learns that CPU frequency has a linear 
> impact on the performance. This is borne out by [1], but does not take into 
> consideration inter-generational gaps as witnessed by [2].
> 
> This begs the question, what are those inter-generational differences made of?
> 
> - L3 cache latency (p. 54) as an upper limit on Mpps. Do newer generations 
> have decidedly lower cache latencies and is this the defining performance 
> factor?
> 
> - Direct Data I/O (p. 69)? Is DDIO combined with lower L3 cache latency a 
> major force multipler? Or is prefetch sufficient to keep caches hot? This is 
> somewhat confusing, as [3] states on page 62 that DPDK can get one core to 
> handleup to 33 mpps, on average. On one hand this is the performance [1] 
> demonstrated the better part of a decade earlier, but on the other hand [2] 
> demonstrates a magnitude larger performance per core.
> 
> - New instructions? On page 171 [3] notes that the AVX512 instruction can 
> move 64 bytes per cycle which [2] indicates has an almost 30% effect on Mpps 
> on page 22. How important is Transactional Synchronization Extensions (TSX) 
> support (see page 119 of [3]) for forwarding performance?
> 
> - Other factors are mentioned, such as memory frequency, memory size, memory 
> channels and cache sizes, but nothing is said how each of these affect 
> forwarding performance in terms of Mpps. The official documentation [0] only 
> states that: "Ensure that each memory channel has at least one memory DIMM 
> inserted, and that the memory size for each is at least 4GB. Note: this has 
> one of the most direct effects on performance."
> 
> - Turbo boost and hyperthreading? Are these supposed to be enabled or 
> disabled? I am getting conflicting information.  Results listed in [2] show 
> increased Mpps by enabling, but [1] notes that they were disabled due to them 
> introducing measurement artifacts. I recall some documentation recommending 
> disabling, since enabling increases latency and variance.
> 
> - Xeon D, W, E3, E5, E7 and Scalable. Are these different processor siblings 
> observably different from each other from the perspective of DPDK? Atoms 
> certainly are as [3] notes on page 57, because they only perform at 50% 
> compared to an equivalent Xeon core. A reson isn't given, but perhaps it is 
> due to the missing L3 cache?
> 
> - Something entirely else? Am I missing something completely obvious that 
> explains the inter-generational differences between CPUS in terms of 
> forwarding performance?
> 
> 
> So, given all this, how can I perform the mundane task of comparing for 
> example the Xeon W-1250P with the Xeon W-1350P?
> 
> The 1250 is older, but has a larger L2 cache and a higher frequency.
> 
> The 1350 is newer, uses faster memory, has a higher max memory bandwidth, 
> PCIe4.0, more PCI lanes and AVX-512.
> 
> Or 

Re: [dpdk-users] can we reserve hugepage and not release

[Thomas Monjalon]

18/09/2021 04:37, Jiany Wu:
> Hello,
> 
> I met a scenario that, need to start and stop the container many times for
> the hugepage. But after several times container start and stop, the
> hugepage is not able to reserve.
> Hugepage size is 2MB, and HW only support 2MB, can't support 1GB.
> Is there anyway to make sure the hugepage is still kept continuous? Thanks
> indeed.

Interesting question.
I think we need to address it in the DPDK documentation.

Anatoly, Stephen, Bruce, any advice please?

Re: [dpdk-users] TX/RX adapter running on the same core problem

[Thomas Monjalon]

+Cc eventdev maintainers

26/09/2021 07:21, Jaeeun Ham:
> Hi,
> 
> I hope you are well.
> 
> During the traffic test, TX adapter showed starvation due to Rx adapter 
> processing on the same dpdk-core 03 and dropped the 41412 packets after 
> 154655 tx_retry.
> So, I expect I have to assign TX/RX adapter on each dpdk-core to prevent 
> starvation and packet drop.
> 
> Are those tx_retry and tx_dropped packet due to RX adapter packet processing?
> Is it possible to separate TX/RX adapter into different dpdk-core?
> 
> [ TX adapter stats ]
> tx_retry: 154655
> tx_packets: 438771
> tx_dropped: 41412
> 
> 
> < eventdev_dump.txt >
> [ Services ]
> qdispatch_service_id: 0
> txa_service_id  : 1
> rxa_service_id  : 2
> edk_service_id  : 3
> timer_service_id: 4
> timer_adapter_service_id: 5
> 
> Service Cores Summary
> dpdk-core 01: PDCP Fastpath Plugin control function thread
> 
> dpdk-core   qu-dispat   txadapter   rxadapter edk tmr   
> tmr-adapt
>03   0   770574949   770574948   0   0 
>   0
>05   816421027   0   0   816421571   816421571   
> 816421571
>07   896307282   0   0   896307828   896307828 
>   0
>09   899213296   0   0   899213687   899213687 
>   0
>11   889323680   0   0   889323871   889323871 
>   0
>13   897300534   0   0   897300686   897300686 
>   0
>15   891124716   0   0   891124811   891124811 
>   0
>17   896336177   0   0   896336212   896336211 
>   0
>19   895461845   0   0   895461846   895461846 
>   0
> 
> [ Event dev port 0 ~ 10 xstats ]
> - packet flow: f1u -> s1u one way
>   midhaul_ker -> midhaul_edk -> backhaul_edk -> backhaul_ker
>   port 09 Rx adapter -> port 07 -> port 06 -> port 08 -> port 09 Tx adapter
> 
> port 00: dpdk-core 05, worker_core // queue-dispatcher, edk, timer
> port 01: dpdk-core 07, worker_core // queue-dispatcher, edk, timer
> port 02: dpdk-core 09, worker_core // queue-dispatcher, edk, timer
> port 03: dpdk-core 11, worker_core // queue-dispatcher, edk, timer
> port 04: dpdk-core 13, worker_core // queue-dispatcher, edk, timer
> port 05: dpdk-core 15, worker_core // queue-dispatcher, edk, timer
> port 06: dpdk-core 17, worker_core // queue-dispatcher, edk, timer
> port 07: dpdk-core 19, worker_core // queue-dispatcher, edk, timer
> port 08: TX adapter, dpdk-core 03  // packet transmission
> port 09: RX adapter, dpdk-core 03  // packet receiving
> port 10: Event timer adapter   // tmr-adapter
> 
> BR/Jaeeun
> -- next part --
> An embedded and charset-unspecified text was scrubbed...
> Name: eventdev_dump.txt
> URL: 
> 
> 

Re: Using rte_flow to distribute single flow type among multiple Rx queues using DPDK in Mellanox ConnectX-5 Ex

[Thomas Monjalon]

29/09/2021 07:26, Anna A:
> Hi,
> 
> I'm trying to use rte_flow_action_type_rss to distribute packets all of the
> same flow type among multiple Rx queues on a single port. Mellanox
> ConnectX-5 Ex and DPDK version 20.05 is used for this purpose. It doesn't
> seem to work and all the packets are sent only to a single queue.

Adding mlx5 maintainers Cc.

> My queries are :
> 1. What am I missing or doing differently?
> 2. Should I be doing any other configurations in rte_eth_conf or
> rte_eth_rxmode?

Do you see any error log?
For info, you can change log level with --log-level.
Experiment options with '--log-level help' in recent DPDK.

> My rte_flow configurations:
> 
> struct rte_flow_item pattern[MAX_RTE_FLOW_PATTERN] = {};
> struct rte_flow_action action[MAX_RTE_FLOW_ACTIONS] = {};
> struct rte_flow_attr attr;
> struct rte_flow_item_eth eth;
> struct rte_flow *flow = NULL;
> struct rte_flow_error error;
> int ret;
> int no_queues =2;
> uint16_t queues[2];
> struct rte_flow_action_rss rss;
> memset(, 0x22, sizeof(error));
> memset(, 0, sizeof(attr));
> attr.egress = 0;
> attr.ingress = 1;
> 
> memset(, 0, sizeof(pattern));
> memset(, 0, sizeof(action));
> /* setting the eth to pass all packets */
> pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH;
> pattern[0].spec = 
> pattern[1].type = RTE_FLOW_ITEM_TYPE_END;
> 
> rss.types = ETH_RSS_IP;
> rss.level = 0;
> rss.func = RTE_ETH_HASH_FUNCTION_TOEPLITZ;
> rss.key_len =0;
> rss.key = NULL;
> rss.queue_num = no_queues;
> for (int i= 0; i < no_queues; i++){
> queues[i] = i;
> }
> rss.queue = queues;
> action[0].type = RTE_FLOW_ACTION_TYPE_RSS;
> action[0].conf = 
> 
> action[1].type = RTE_FLOW_ACTION_TYPE_END;
> 
> ret = rte_flow_validate(portid, , pattern, action, );
>  if (ret < 0) {
>   printf( "Flow validation failed %s\n", error.message);
> return;
> }
> flow = rte_flow_create(portid, , pattern, action, );
> 
> if (flow == NULL)
> printf(" Cannot create Flow create");
> 
> And Rx queues configuration:
> for (int j = 0; j < no_queues; j++) {
> 
>  int ret = rte_eth_rx_queue_setup(portid, j, nb_rxd,
> rte_eth_dev_socket_id(port_id),
>NULL,mbuf_pool);
>  if (ret < 0) {
>   printf( "rte_eth_rx_queue_setup:err=%d, port=%u", ret, (unsigned)
> portid);
> exit(1);
>}
> }
> 
> Thanks
> Anna

Re: [dpdk-users] Memory allocation limits

[Thomas Monjalon]

26/09/2021 17:52, Mohammad Masumi:
> Hi
> 
> I have HP server with 768GB memory 384GB in each Numa but I can't allocate
> more than 64GB by rte_malloc by changing some parameters in rte_config.h it
> increased to 128GB
> How to increase heap size?

adding people Cc to help