Re: DPDK NIC - Firmware Comptability

[Thomas Monjalon]

28/03/2024 06:53, Pujar, Shyam:
> Hi All ,
>   I have question regarding DPDK NIC-Firmware version 
> compatibility. https://doc.dpdk.org/guides/nics/i40e.html.
> I have highlighted the DPDK version in use.   Does the following matrix 
> indicates that, "minimum version of Firmware or exact version"?

Good question, Yuying, Hailin, can we upgrade the firmware version,
while being on an old DPDK and/or old kernel module?

Would be nice to make it clear in the documentation.

Thank you

Re: when link librte_net_mlx5.so and use gdb, my program can't get environment variable

[Thomas Monjalon]

Hello,

We could dig in details what happens in constructors,
but first I don't understand why you want to LD_PRELOAD some drivers.
The DPDK drivers are already loaded dynamically with dlopen() from EAL.


04/12/2023 10:45, jiangheng (G):
> Environment:
> [root@localhost jh]# cat /etc/centos-release
> CentOS Linux release 8.5.2111
> root@localhost jh]# uname -r
> 4.18.0-348.el8.x86_64
> [root@localhost jh]# rpm -qa gcc
> gcc-8.5.0-3.el8.x86_64
> [root@localhost jh]# rpm -qa dpdk
> dpdk-21.11-3.el8.x86_64
> 
> Reproduction Procedure:
> 1. We need create test.c and init.c
> 
> [root@localhost jh]# cat init.c
> #include 
> #include 
> #include 
> 
> __attribute__ ((constructor)) void init(void)
> {
> char *enval = NULL;
> enval = getenv("LD_PRELOAD");
> printf("enval LD_PRELOAD : %s\n", enval);
> return;
> }
> 
> [root@localhost jh]# cat test.c
> #include 
> 
> int main(int argc, char **argv)
> {
> printf("Hello World\n");
> return 0;
> }
> 
> 2. Build test.c
> gcc test.c -o test
> 
> 3. Build init.c as so in three cases:
> 3.1 only link librte_eal.so
> gcc --shared -fPIC -lrte_eal   init.c -o libinit.so
> 3.2 link librte_eal.so and librte_net_i40e.so
> gcc --shared -fPIC -lrte_eal -lrte_net_i40e  init.c -o libinit_i40e.so
> 3.3 link librte_eal.so and librte_net_mlx5.so
> gcc --shared -fPIC -lrte_eal -lrte_net_mlx5  init.c -o libinit_mlx5.so
> 
> 4. run test using gdb and LD_PRELOAD 
> 4.1 : libinit.so links only librte_eal.so 
> LD_PRELOAD=./libinit.so   gdb  ./test
> (gdb) r
> enval LD_PRELOAD : ./libinit.so
> enval LD_PRELOAD : ./libinit.so   (I don't know why the constrctor function 
> is executed twice.)
> Hello World
> 
> 4.2 libinit.so links librte_eal.so and librte_net_i40e.so
> LD_PRELOAD=./libinit_i40e.sogdb  ./test
> enval LD_PRELOAD : ./libinit_i40e.so
> enval LD_PRELOAD : ./libinit_i40e.so(I don't know why the constrctor 
> function is executed twice.)
> Hello World
> 
> 4.3 4.3 libinit.so links librte_eal.so and librte_net_mlx5.so
> LD_PRELOAD=./libinit_mlx5.so gdb  ./test
> enval LD_PRELOAD : (null)  // ? ? ? ?
> enval LD_PRELOAD : ./libinit_mlx5.so
> Hello World
> 
> 
> After libinit.so is linked to librte_net_mlx5.so, the symptom when the gdb is 
> used to execute the program is different from the preceding two cases. The 
> value of LD_PRELOAD obtained is NULL for the first time.
> Have you ever had the same problem?
> Looking forward to your favourable reply.
> 
> Thanks
> 
> 
> 

[korganizer] [Bug 474144] Korganizer crashes when forwarding an event

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=474144

--- Comment #2 from Thomas Monjalon  ---
In Korganizer, I do a right-clik on an event and choose "send as iCalendar".
Then I choose the recipient, click OK,
and it crashes.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[korganizer] [Bug 474144] Korganizer crashes when forwarding an event

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=474144

--- Comment #2 from Thomas Monjalon  ---
In Korganizer, I do a right-clik on an event and choose "send as iCalendar".
Then I choose the recipient, click OK,
and it crashes.

-- 
You are receiving this mail because:
You are watching all bug changes.

[korganizer] [Bug 474144] New: Korganizer crashes when forwarding an event

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=474144

Bug ID: 474144
   Summary: Korganizer crashes when forwarding an event
Classification: Applications
   Product: korganizer
   Version: unspecified
  Platform: Archlinux
OS: Linux
Status: REPORTED
  Keywords: drkonqi
  Severity: crash
  Priority: NOR
 Component: general
  Assignee: kdepim-b...@kde.org
  Reporter: tho...@monjalon.net
  Target Milestone: ---

Application: korganizer (5.24.0 (23.08.0))

Qt Version: 5.15.10
Frameworks Version: 5.109.0
Operating System: Linux 6.4.12-arch1-1 x86_64
Windowing System: X11
Distribution: "Arch Linux"
DrKonqi: 5.27.7 [KCrashBackend]

-- Information about the crash:
Since a recent update, I am not able to forward a calendar event.

The crash can be reproduced every time.

-- Backtrace:
Application: KOrganizer (korganizer), signal: Segmentation fault
Content of s_kcrashErrorMessage: std::unique_ptr = {get() = }
[KCrash Handler]
#6  0x7f2629aef0b8 in
Akonadi::MailClient::queueMessage(MailTransport::Transport const*,
MessageComposer::Composer const*, QSharedPointer
const&, KIdentityManagement::Identity const&, Akonadi::MailClient::MessageData
const&, QSharedPointer const&) (message=..., msg=...,
identity=, incidence=..., composer=,
transport=0x561d04ed36f0, this=0x561d03ffcbf0) at
/usr/src/debug/akonadi-calendar/akonadi-calendar-23.08.0/src/mailclient_p.cpp:323
#7  operator() (__closure=0x561d04eb5680) at
/usr/src/debug/akonadi-calendar/akonadi-calendar-23.08.0/src/mailclient_p.cpp:641
#8  QtPrivate::FunctorCall, QtPrivate::List<>, void,
Akonadi::MailClient::send(const KCalendarCore::IncidenceBase::Ptr&, const
KIdentityManagement::Identity&, const MessageData&, const QString&)::
>::call (arg=, f=...) at
/usr/include/qt/QtCore/qobjectdefs_impl.h:146
#9  QtPrivate::Functor, 0>::call, void>
(arg=, f=...) at /usr/include/qt/QtCore/qobjectdefs_impl.h:256
#10 QtPrivate::QFunctorSlotObject, 0, QtPrivate::List<>,
void>::impl(int, QtPrivate::QSlotObjectBase *, QObject *, void **, bool *)
(which=, this_=0x561d04eb5670, r=, a=, ret=) at /usr/include/qt/QtCore/qobjectdefs_impl.h:443
#11 0x7f262a2d1637 in QtPrivate::QSlotObjectBase::call(QObject*, void**)
(a=0x7fff5abc71f0, r=, this=0x561d04eb5670, this=, r=, a=) at
../../include/QtCore/../../src/corelib/kernel/qobjectdefs_impl.h:398
#12 doActivate(QObject*, int, void**) (sender=0x561d04ede3f0,
signal_index=6, argv=0x7fff5abc71f0) at kernel/qobject.cpp:3925
#13 0x7f262b7c30aa in KJob::result(KJob*, KJob::QPrivateSignal)
(this=this@entry=0x561d04ede3f0, _t1=, _t1@entry=0x561d04ede3f0,
_t2=...) at
/usr/src/debug/kcoreaddons/build/src/lib/KF5CoreAddons_autogen/include/moc_kjob.cpp:633
#14 0x7f262b7c907c in KJob::finishJob(bool) (this=0x561d04ede3f0,
emitResult=) at
/usr/src/debug/kcoreaddons/kcoreaddons-5.109.0/src/lib/jobs/kjob.cpp:98
#15 0x7f262a2d1637 in QtPrivate::QSlotObjectBase::call(QObject*, void**)
(a=0x7fff5abc7320, r=, this=0x561d04f1cbd0, this=, r=, a=) at
../../include/QtCore/../../src/corelib/kernel/qobjectdefs_impl.h:398
#16 doActivate(QObject*, int, void**) (sender=0x561d04ec9270,
signal_index=6, argv=0x7fff5abc7320) at kernel/qobject.cpp:3925
#17 0x7f262b7c30aa in KJob::result(KJob*, KJob::QPrivateSignal)
(this=this@entry=0x561d04ec9270, _t1=, _t1@entry=0x561d04ec9270,
_t2=...) at
/usr/src/debug/kcoreaddons/build/src/lib/KF5CoreAddons_autogen/include/moc_kjob.cpp:633
#18 0x7f262b7c907c in KJob::finishJob(bool) (this=0x561d04ec9270,
emitResult=) at
/usr/src/debug/kcoreaddons/kcoreaddons-5.109.0/src/lib/jobs/kjob.cpp:98
#19 0x7f262b7c9cd2 in KJob::emitResult() (this=) at
/usr/src/debug/kcoreaddons/kcoreaddons-5.109.0/src/lib/jobs/kjob.cpp:338
#20 0x7f2627a3a67a in operator() (auditLogAsHtml=,
auditLogError=, signature=..., result=,
__closure=0x561d04efe8e0) at
/usr/src/debug/messagelib/messagelib-23.08.0/messagecomposer/src/job/signjob.cpp:277
#21 QtPrivate::FunctorCall,
QtPrivate::List, void, MessageComposer::SignJob::process()::
>::call (arg=, f=...) at
/usr/include/qt/QtCore/qobjectdefs_impl.h:146
#22 QtPrivate::Functor, 4>::call, void> (arg=, f=...) at /usr/include/qt/QtCore/qobjectdefs_impl.h:256
#23
QtPrivate::QFunctorSlotObject, 4, QtPrivate::List, void>::impl(int,
QtPrivate::QSlotObjectBase *, QObject *, void **, bool *) (which=, this_=0x561d04efe8d0, r=, a=,
ret=) at /usr/include/qt/QtCore/qobjectdefs_impl.h:443
#24 0x7f262a2d1637 in QtPrivate::QSlotObjectBase::call(QObject*, void**)
(a=0x7fff5abc74b0, r=, this=0x561d04efe8d0, this=, r=, a=) at
../../include/QtCore/../../src/corelib/kernel/qobjectdefs_impl.h:398
#25 doActivate(QObject*, int, void**) (sender=0x561d04ec7660,
signal_index=7, argv=0x7fff5abc74b0) at kernel/qobject.cpp:3925
#26 0x7f2626d55806 in QGpgME::SignJob::result(GpgME::SigningResult const&,
QByteArray const&, QString const&, GpgME::Error const&) (this=,
_t1=, _t2=, _t3=, _t4=) at 

[korganizer] [Bug 474144] New: Korganizer crashes when forwarding an event

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=474144

Bug ID: 474144
   Summary: Korganizer crashes when forwarding an event
Classification: Applications
   Product: korganizer
   Version: unspecified
  Platform: Archlinux
OS: Linux
Status: REPORTED
  Keywords: drkonqi
  Severity: crash
  Priority: NOR
 Component: general
  Assignee: kdepim-bugs@kde.org
  Reporter: tho...@monjalon.net
  Target Milestone: ---

Application: korganizer (5.24.0 (23.08.0))

Qt Version: 5.15.10
Frameworks Version: 5.109.0
Operating System: Linux 6.4.12-arch1-1 x86_64
Windowing System: X11
Distribution: "Arch Linux"
DrKonqi: 5.27.7 [KCrashBackend]

-- Information about the crash:
Since a recent update, I am not able to forward a calendar event.

The crash can be reproduced every time.

-- Backtrace:
Application: KOrganizer (korganizer), signal: Segmentation fault
Content of s_kcrashErrorMessage: std::unique_ptr = {get() = }
[KCrash Handler]
#6  0x7f2629aef0b8 in
Akonadi::MailClient::queueMessage(MailTransport::Transport const*,
MessageComposer::Composer const*, QSharedPointer
const&, KIdentityManagement::Identity const&, Akonadi::MailClient::MessageData
const&, QSharedPointer const&) (message=..., msg=...,
identity=, incidence=..., composer=,
transport=0x561d04ed36f0, this=0x561d03ffcbf0) at
/usr/src/debug/akonadi-calendar/akonadi-calendar-23.08.0/src/mailclient_p.cpp:323
#7  operator() (__closure=0x561d04eb5680) at
/usr/src/debug/akonadi-calendar/akonadi-calendar-23.08.0/src/mailclient_p.cpp:641
#8  QtPrivate::FunctorCall, QtPrivate::List<>, void,
Akonadi::MailClient::send(const KCalendarCore::IncidenceBase::Ptr&, const
KIdentityManagement::Identity&, const MessageData&, const QString&)::
>::call (arg=, f=...) at
/usr/include/qt/QtCore/qobjectdefs_impl.h:146
#9  QtPrivate::Functor, 0>::call, void>
(arg=, f=...) at /usr/include/qt/QtCore/qobjectdefs_impl.h:256
#10 QtPrivate::QFunctorSlotObject, 0, QtPrivate::List<>,
void>::impl(int, QtPrivate::QSlotObjectBase *, QObject *, void **, bool *)
(which=, this_=0x561d04eb5670, r=, a=, ret=) at /usr/include/qt/QtCore/qobjectdefs_impl.h:443
#11 0x7f262a2d1637 in QtPrivate::QSlotObjectBase::call(QObject*, void**)
(a=0x7fff5abc71f0, r=, this=0x561d04eb5670, this=, r=, a=) at
../../include/QtCore/../../src/corelib/kernel/qobjectdefs_impl.h:398
#12 doActivate(QObject*, int, void**) (sender=0x561d04ede3f0,
signal_index=6, argv=0x7fff5abc71f0) at kernel/qobject.cpp:3925
#13 0x7f262b7c30aa in KJob::result(KJob*, KJob::QPrivateSignal)
(this=this@entry=0x561d04ede3f0, _t1=, _t1@entry=0x561d04ede3f0,
_t2=...) at
/usr/src/debug/kcoreaddons/build/src/lib/KF5CoreAddons_autogen/include/moc_kjob.cpp:633
#14 0x7f262b7c907c in KJob::finishJob(bool) (this=0x561d04ede3f0,
emitResult=) at
/usr/src/debug/kcoreaddons/kcoreaddons-5.109.0/src/lib/jobs/kjob.cpp:98
#15 0x7f262a2d1637 in QtPrivate::QSlotObjectBase::call(QObject*, void**)
(a=0x7fff5abc7320, r=, this=0x561d04f1cbd0, this=, r=, a=) at
../../include/QtCore/../../src/corelib/kernel/qobjectdefs_impl.h:398
#16 doActivate(QObject*, int, void**) (sender=0x561d04ec9270,
signal_index=6, argv=0x7fff5abc7320) at kernel/qobject.cpp:3925
#17 0x7f262b7c30aa in KJob::result(KJob*, KJob::QPrivateSignal)
(this=this@entry=0x561d04ec9270, _t1=, _t1@entry=0x561d04ec9270,
_t2=...) at
/usr/src/debug/kcoreaddons/build/src/lib/KF5CoreAddons_autogen/include/moc_kjob.cpp:633
#18 0x7f262b7c907c in KJob::finishJob(bool) (this=0x561d04ec9270,
emitResult=) at
/usr/src/debug/kcoreaddons/kcoreaddons-5.109.0/src/lib/jobs/kjob.cpp:98
#19 0x7f262b7c9cd2 in KJob::emitResult() (this=) at
/usr/src/debug/kcoreaddons/kcoreaddons-5.109.0/src/lib/jobs/kjob.cpp:338
#20 0x7f2627a3a67a in operator() (auditLogAsHtml=,
auditLogError=, signature=..., result=,
__closure=0x561d04efe8e0) at
/usr/src/debug/messagelib/messagelib-23.08.0/messagecomposer/src/job/signjob.cpp:277
#21 QtPrivate::FunctorCall,
QtPrivate::List, void, MessageComposer::SignJob::process()::
>::call (arg=, f=...) at
/usr/include/qt/QtCore/qobjectdefs_impl.h:146
#22 QtPrivate::Functor, 4>::call, void> (arg=, f=...) at /usr/include/qt/QtCore/qobjectdefs_impl.h:256
#23
QtPrivate::QFunctorSlotObject, 4, QtPrivate::List, void>::impl(int,
QtPrivate::QSlotObjectBase *, QObject *, void **, bool *) (which=, this_=0x561d04efe8d0, r=, a=,
ret=) at /usr/include/qt/QtCore/qobjectdefs_impl.h:443
#24 0x7f262a2d1637 in QtPrivate::QSlotObjectBase::call(QObject*, void**)
(a=0x7fff5abc74b0, r=, this=0x561d04efe8d0, this=, r=, a=) at
../../include/QtCore/../../src/corelib/kernel/qobjectdefs_impl.h:398
#25 doActivate(QObject*, int, void**) (sender=0x561d04ec7660,
signal_index=7, argv=0x7fff5abc74b0) at kernel/qobject.cpp:3925
#26 0x7f2626d55806 in QGpgME::SignJob::result(GpgME::SigningResult const&,
QByteArray const&, QString const&, GpgME::Error const&) (this=,
_t1=, _t2=, _t3=, _t4=) at 

Re: Direct Mem Pool vs Indirect mem pool creation

[Thomas Monjalon]

01/09/2023 14:47, omer yamac:
> Hello,
> 
> I need clarification while creating direct/indirect buffers for mbuf. I
> couldn't find exact documentation, and I just looked over the fragmentation
> test case and saw that two pools were created. One is a
> direct pool, and the other is an indirect pool. Here are the methods to
> create pools:
> direct_pool = rte_pktmbuf_pool_create("FRAG_D_MBUF_POOL",
>   NUM_MBUFS, BURST, 0,
>   RTE_MBUF_DEFAULT_BUF_SIZE,
>   SOCKET_ID_ANY);
> indirect_pool = rte_pktmbuf_pool_create("FRAG_I_MBUF_POOL",
> NUM_MBUFS, BURST, 0,
> 0, SOCKET_ID_ANY)
> 
> I couldn't see the exact difference. Just the "data_room_size" parameter is
> different. If this parameter is 0, then is the pool indirect?

A pool is neither direct or indirect, it is just a pool of buffers
with a defined size for all buffers of a pool.
You are free to create any pool for your needs.

Now if you create a pool of buffers with size 0,
we can expect you will save some data elsewhere,
using rte_pktmbuf_attach_extbuf() for instance.

More explanations can be found in the doc:
https://doc.dpdk.org/guides/prog_guide/mbuf_lib.html#direct-and-indirect-buffers

Re: help

[Thomas Monjalon]

Thanks for the info.
Do you think it should be written in the vmxnet3 page of the DPDK documentation?
If yes, would you like to initiate a patch for review?


11/08/2023 10:43, Igor de Paula:
> Hi again,
> I got this resolved with VMWARE support so I thought to share it here.
> What I originally wanted was to use IOVA-VA on an AMD host. Which didn't
> work. I have learned that the ESXI version that supports
> virtual IOMMU in AMD hosts (which is a prerequisite to IOVA-VA) is ESXI 7.0
> U1. After updating it worked. On Intel hosts ESXI 6.7 supports it already
> as far as I know.
> 
> 
> On Tue, Jul 25, 2023 at 6:19 PM Varghese, Vipin 
> wrote:
> 
> > [AMD Official Use Only - General]
> >
> > Like I said earlier, trying with the Intel host I have on VMWARE,
> > specifically  Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz
> > With IOMMU enabled, VMXNET3 works with VA as well as PA.
> >
> > [VV] since ` enable_unsafe_iommu: not enabled` on Intel platform, could
> > it be possible the specific version EXSI hypervisor supports the HW IOMMU
> > specific to the platform. My suspicion will be in case on AMD platform
> >  changes for required to enable HW iommu might not be available to specific
> > EXSI (hypervior OS) used.
> >
> >
> >
> > I am not an expert on virtio_user PMD, but I can check if it will work
> > with PA with deferred setting for vmx_net3 PMD are ok?
> >
> >
> >
> > *From:* Igor de Paula 
> > *Sent:* Tuesday, July 25, 2023 8:42 PM
> > *To:* Varghese, Vipin 
> > *Cc:* Yigit, Ferruh ; Jochen Behrens <
> > jbehr...@vmware.com>; Thomas Monjalon ;
> > users@dpdk.org; Gupta, Nipun ; Agarwal, Nikhil <
> > nikhil.agar...@amd.com>; Ronak Doshi ; Immanni, Venkat
> > ; Chenbo Xia 
> > *Subject:* Re: help
> >
> >
> >
> > *Caution:* This message originated from an External Source. Use proper
> > caution when opening attachments, clicking links, or responding.
> >
> >
> >
> > Well,
> > Like I said earlier, trying with the Intel host I have on VMWARE,
> > specifically  Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz
> > With IOMMU enabled, VMXNET3 works with VA as well as PA.
> > Meaning, PA works regardless if IOMMU is enabled or not. From my
> > experience anyway.
> > That's why I thought that:
> > virtio_user needs VA to work.
> > For some reason VMXNET3 does not work with VA (only on AMD host).
> >
> >
> >
> >
> >
> > On Tue, Jul 25, 2023 at 4:04 PM Varghese, Vipin 
> > wrote:
> >
> > [AMD Official Use Only - General]
> >
> >
> >
> > Thanks Igor,
> >
> >
> >
> > As suspected the vmx_net3 works with
> >
> >
> >
> >1. Iommu: disabled
> >2. enable_unsafe_iommu: enabled
> >3. dpdk eal iova mode: PA
> >
> >
> >
> > as pointed by you in logs, the virtio_user fails as it expects VA too.
> >
> >
> >
> > Will check and get back.
> >
> >
> >
> > *From:* Igor de Paula 
> > *Sent:* Tuesday, July 25, 2023 8:16 PM
> > *To:* Yigit, Ferruh 
> > *Cc:* Jochen Behrens ; Thomas Monjalon <
> > tho...@monjalon.net>; users@dpdk.org; Gupta, Nipun ;
> > Agarwal, Nikhil ; Ronak Doshi ;
> > Immanni, Venkat ; Varghese, Vipin <
> > vipin.vargh...@amd.com>; Chenbo Xia 
> > *Subject:* Re: help
> >
> >
> >
> > *Caution:* This message originated from an External Source. Use proper
> > caution when opening attachments, clicking links, or responding.
> >
> >
> >
> > Hi,
> > Attaching the logs of EAL when trying to run a configuration with
> > virtio_user port when IOMMU is
> > disabled and enable_unsafe_iommu is enabled. As you can see it forces IOVA
> > as PA but the viritui_user needs IOVA as VA.
> > I am also attaching the output of dmesg. I am not sure which kernel logs
> > you wanted... if there is anything else please let me know..
> > Regarding the ESXI logs, they are HUGE so I will send to you on a separate
> > email.
> >
> >
> >
> > On Fri, Jul 21, 2023 at 1:14 PM Ferruh Yigit  wrote:
> >
> > On 7/21/2023 12:39 PM, Igor de Paula wrote:
> > > I am trying to use virtio_user for an interface with the
> > > kernel:
> > https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html <
> > https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html>
> > > I think this requires IOVA as va.
> > >
> >
> > I am not sure if virtio-user has IOVA as VA 

Re: Supporting RSS with DPDK in a VM

[Thomas Monjalon]

Hello,

You can have packets distributed to multiple queues without RSS.

If you really wants to enable the RSS algorithms,
it seems not supported for now with vhost_user.
It can be enabled with vhost running in the kernel:
https://qemu.readthedocs.io/en/latest/devel/ebpf_rss.html


26/07/2023 00:58, Matheus Stolet:
> Hello,
> 
> I am trying to run a DPDK application with RSS enabled so that I can 
> have multiple rx queues. This application is running inside a VM. This 
> VM is hosted by QEMU using KVM acceleration and OvS with DPDK and 
> vhost-user are used in the backend. So to clarify things there are two 
> DPDK portions to this. The first is the DPDK portion used by OvS that 
> bypasses the host operating system. This is working fine. The other is a 
> DPDK application inside the virtual machine that will bypass the guest 
> operating system. This is where I am having trouble with.
> 
> When I set rte_eth_conf.rxmode.mq_mode = ETH_MQ_RX_RSS in my application 
> I get the following errors:
>Warning: NIC does not support all requested RSS hash functions.
>virtio_dev_configure(): RSS support requested but not supported by the 
> device
>Port0 dev_configure = -95
> 
> I setup my VM in QEMU to have mq=on and queues=10. I also set the number 
> of rx_queues when creating the vhost port using ovs to 10. Before 
> binding the interface to DPDK, I used ethtool to verify if the network 
> interface was actually setup to have multiple queues.
> 
> Running the 'ethtool -l enps02' command yields the following output:
>Pre-set maximums:
>RX: 0
>TX: 0
>Other:  0
>Combined:   10
>Current hardware settings:
>RX: 0
>TX: 0
>Other:  0
>Combined:   10
> 
>  From my understanding the combined values indicate that the interface 
> was properly setup to have multiple queues, so why am I getting the 
> unsupported RSS error? Are there other configuration steps that I have 
> to take to get this to work? Is RSS with DPDK not supported at all 
> inside a VM at the moment? Perhaps the "Port0 dev_configure() = -95" 
> error means something else? Without the receive side scaling turned on 
> my application is not able to achieve the desired throughput and won't 
> scale when I assign more cores to the application.
> 
> Versions:
> VM
> DPDK: 21.11.4
> Kernel: 5.4.0-148-generic
> Distribution: Ubuntu 20.04
> 
> Host
> DPDK: 21.11.4
> QEMU: 8.0.90
> OvS: 3.0.5
> Kernel: 5.15.111.1.amd64-smp
> Distribution: Debian 11
> 

Re: Enable RSS for virtio application ( dpdk version 21.11)

[Thomas Monjalon]

You may need vhost running in the Linux kernel with some BPF code.
There is a documentation about eBPF RSS:
https://qemu.readthedocs.io/en/latest/devel/ebpf_rss.html


26/07/2023 09:32, shiv chittora:
> Thanks Bing for quick response.
> 
> The virtio driver version 1.0.0 is included in the Linux kernel version 4.9
> that powers VM.
> 
> ethtool -i eth1
> driver: virtio_net
> version: 1.0.0
> firmware-version:
> expansion-rom-version:
> bus-info: :00:04.0
> 
> Nutanix document stats that "Ensure the AHV UVM is running the latest
> Nutanix VirtIO driver package. Nutanix VirtIO 1.1.6 or higher is required
> for RSS support. " Linux kernel version: 5.4 and later will have Virtio
> 1.1.6.
> 
> Since the programme is built on the dpdk, the PMD driver will use the eth
> interface rather than the one that the kernel provides. I apologise if I'm
> mistaken. RSS is supported by the dpdk PMD version in use.
> 
> Because of the client-centric nature of this application, upgrading the
> kernel will be challenging.
> 
> Do you believe that the only option is to upgrade the vm kernel version?
> 
> Thanks ,
> Shiv
> 
> On Wed, Jul 26, 2023 at 12:33 PM Bing Zhao  wrote:
> 
> > IIRC, the “VIRTIO_NET_F_RSS” is some capability reported and decided
> > during the driver setup/communication stage. It is mostly like that your
> > libs/drivers running on the host for the VM does not support this feature.
> >
> > Have you tried to update the versions of VM or the package/lib of VirtIO
> > for this VM?
> >
> >
> >
> > *From:* shiv chittora 
> > *Sent:* Wednesday, July 26, 2023 1:05 PM
> > *To:* users@dpdk.org
> > *Subject:* Enable RSS for virtio application ( dpdk version 21.11)
> >
> >
> >
> > *External email: Use caution opening links or attachments*
> >
> >
> >
> > I'm using a Nutanix virtual machine to run a DPDK(Version 21.11)-based
> > application.
> > Application is failing during rte_eth_dev_configure . For our application,
> > RSS support is required.
> >
> > eth_config.rxmode.mq_mode = ETH_MQ_RX_RSS;
> > static uint8_t hashKey[] = {
> > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
> > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
> > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
> > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
> > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
> > };
> >
> > eth_config.rx_adv_conf.rss_conf.rss_key = hashKey;
> > eth_config.rx_adv_conf.rss_conf.rss_key_len = sizeof(hashKey);
> > eth_config.rx_adv_conf.rss_conf.rss_hf = 260
> >
> >
> >
> > With the aforementioned RSS configuration, the application is not coming
> > up. The same application runs without any issues on a VMware virtual
> > machine.
> >
> > When I set
> >
> > eth_config.rxmode.mq_mode = ETH_MQ_RX_NONE
> > eth_config.rx_adv_conf.rss_conf.rss_hf = 0
> >
> > Application starts working fine. Since we need RSS support for our
> > application I cannot set eth_config.rxmode.mq_mode = ETH_MQ_RX_NONE.
> >
> > I looked at the DPDK 21.11 release notes, and it mentions that virtio_net
> > supports RSS support.
> >
> >
> > In this application traffic is tapped to capture port. I have also created
> > two queues using ACLI comments.
> >
> >  vm.nic_create nutms1-ms type=kNetworkFunctionNic
> > network_function_nic_type=kTap queues=2
> >
> >  vm.nic_get testvm
> > xx:xx:xx:xx:xx:xx {
> >   mac_addr: "xx:xx:xx:xx:xx:xx"
> >   network_function_nic_type: "kTap"
> >   network_type: "kNativeNetwork"
> >   queues: 2
> >   type: "kNetworkFunctionNic"
> >   uuid: "9c26c704-bcb3-4483-bdaf-4b64bb9233ef"
> > }
> >
> >
> > Additionally, I've turned on dpdk logging. PFB the dpdk log's output.
> >
> > EAL: PCI device :00:05.0 on NUMA socket 0
> > EAL:   probe driver: 1af4:1000 net_virtio
> > EAL: Probe PCI driver: net_virtio (1af4:1000) device: :00:05.0 (socket
> > 0)
> > EAL:   PCI memory mapped at 0x94000
> > EAL:   PCI memory mapped at 0x940001000
> > virtio_read_caps(): [98] skipping non VNDR cap id: 11
> > virtio_read_caps(): [84] cfg type: 5, bar: 0, offset: , len: 0
> > virtio_read_caps(): [70] cfg type: 2, bar: 4, offset: 3000, len: 4096
> > virtio_read_caps(): [60] cfg type: 4, bar: 4, offset: 2000, len: 4096
> > virtio_read_caps(): [50] cfg type: 3, bar: 4, offset: 1000, len: 4096
> > virtio_read_caps(): [40] cfg type: 1, bar: 4, offset: , len: 4096
> > virtio_read_caps(): found modern virtio pci device.
> > virtio_read_caps(): common cfg mapped at: 0x940001000
> > virtio_read_caps(): device cfg mapped at: 0x940003000
> > virtio_read_caps(): isr cfg mapped at: 0x940002000
> > virtio_read_caps(): notify base: 0x940004000, notify off multiplier: 4
> > vtpci_init(): modern virtio pci detected.
> > virtio_ethdev_negotiate_features(): guest_features before negotiate =
> > 805f10ef8028
> > virtio_ethdev_negotiate_features(): host_features before negotiate =
> > 130a7
> > virtio_ethdev_negotiate_features(): features after 

Re: help

[Thomas Monjalon]

+Cc some AMD maintainers, they can have an idea about IOMMU settings.


20/07/2023 14:44, Igor de Paula:
> I have enabled it in the host and in the BIOS for AMD...
> In the Bios I changed to amd_iommu=on and in the host it's the same for
> either.
> 
> On Thu, Jul 20, 2023 at 1:31 PM Thomas Monjalon  wrote:
> 
> > 20/07/2023 11:35, Igor de Paula:
> > > The weird thing is that it only happens when I am using a host with an
> > AMD
> > > processor. It doesn't happen when I use a host with an Intel processor.
> >
> > So it's probably a matter of BIOS settings for the IOMMU?
> >
> >
> > > On Thu, Jul 20, 2023 at 10:32 AM Thomas Monjalon 
> > > wrote:
> > >
> > > > +Cc the vmxnet3 maintainer.
> > > >
> > > > Please Jochen, do you have an idea what's wrong below?
> > > >
> > > >
> > > > 20/07/2023 11:25, Igor de Paula:
> > > > > This is because it can't negotiate the IOMMU type with any port.
> > > > >
> > > > > On Thu, Jul 20, 2023 at 5:08 AM Thomas Monjalon  > >
> > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > The first error is "Cause: Error: number of ports must be even"
> > > > > >
> > > > > >
> > > > > > 03/05/2023 18:13, Igor de Paula:
> > > > > > > I am running a VM inside a VMWARE server (vSphere).
> > > > > > > My goal it to set up DPDK with two HW ports, and set up a
> > > > virtio_user to
> > > > > > > interact with the kernel stack.
> > > > > > > In another app I have it working but instead of virtio_user I am
> > > > running
> > > > > > > KNI, it works in IOVA-PA mode.
> > > > > > > I am looking to replace the KNI.
> > > > > > >
> > > > > > > When I try to set up virtio_user port as in the doc:
> > > > > > >
> > > > > >
> > > >
> > https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path
> > > > > > > I get a error it can't run in PA mode.
> > > > > > >
> > > > > > >
> > > > > > > When I try to run as VA mode from a parameter, I get the
> > following
> > > > > > errors:
> > > > > > > EAL: lib.eal log level changed from info to debug
> > > > > > > EAL: Detected lcore 0 as core 0 on socket 0
> > > > > > > EAL: Detected lcore 1 as core 0 on socket 0
> > > > > > > EAL: Support maximum 128 logical core(s) by configuration.
> > > > > > > EAL: Detected 2 lcore(s)
> > > > > > > EAL: Detected 1 NUMA nodes
> > > > > > > EAL: Checking presence of .so 'librte_eal.so.21.3'
> > > > > > > EAL: Checking presence of .so 'librte_eal.so.21'
> > > > > > > EAL: Checking presence of .so 'librte_eal.so'
> > > > > > > EAL: Detected static linkage of DPDK
> > > > > > > EAL: Ask a virtual area of 0x7000 bytes
> > > > > > > EAL: Virtual area found at 0x1 (size = 0x7000)
> > > > > > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > > > > > > EAL: DPAA Bus not present. Skipping.
> > > > > > > EAL: VFIO PCI modules not loaded
> > > > > > > EAL: Selected IOVA mode 'VA'
> > > > > > > EAL: Probing VFIO support...
> > > > > > > EAL: IOMMU type 1 (Type 1) is supported
> > > > > > > EAL: IOMMU type 7 (sPAPR) is not supported
> > > > > > > EAL: IOMMU type 8 (No-IOMMU) is supported
> > > > > > > EAL: VFIO support initialized
> > > > > > > EAL: Ask a virtual area of 0x5b000 bytes
> > > > > > > EAL: Virtual area found at 0x17000 (size = 0x5b000)
> > > > > > > EAL: Setting up physically contiguous memory...
> > > > > > > EAL: Setting maximum number of open files to 1048576
> > > > > > > EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> > > > > > > EAL: Creating 2 segment lists: n_segs:128 socket_id:0
> > > > > > hugepage_sz:1073741824
> > > > > > > EAL: Ask a virtual area of 0x2000 bytes
> > > > > > > EAL: V

Re: help

[Thomas Monjalon]

20/07/2023 11:35, Igor de Paula:
> The weird thing is that it only happens when I am using a host with an AMD
> processor. It doesn't happen when I use a host with an Intel processor.

So it's probably a matter of BIOS settings for the IOMMU?


> On Thu, Jul 20, 2023 at 10:32 AM Thomas Monjalon 
> wrote:
> 
> > +Cc the vmxnet3 maintainer.
> >
> > Please Jochen, do you have an idea what's wrong below?
> >
> >
> > 20/07/2023 11:25, Igor de Paula:
> > > This is because it can't negotiate the IOMMU type with any port.
> > >
> > > On Thu, Jul 20, 2023 at 5:08 AM Thomas Monjalon 
> > wrote:
> > >
> > > > Hello,
> > > >
> > > > The first error is "Cause: Error: number of ports must be even"
> > > >
> > > >
> > > > 03/05/2023 18:13, Igor de Paula:
> > > > > I am running a VM inside a VMWARE server (vSphere).
> > > > > My goal it to set up DPDK with two HW ports, and set up a
> > virtio_user to
> > > > > interact with the kernel stack.
> > > > > In another app I have it working but instead of virtio_user I am
> > running
> > > > > KNI, it works in IOVA-PA mode.
> > > > > I am looking to replace the KNI.
> > > > >
> > > > > When I try to set up virtio_user port as in the doc:
> > > > >
> > > >
> > https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path
> > > > > I get a error it can't run in PA mode.
> > > > >
> > > > >
> > > > > When I try to run as VA mode from a parameter, I get the following
> > > > errors:
> > > > > EAL: lib.eal log level changed from info to debug
> > > > > EAL: Detected lcore 0 as core 0 on socket 0
> > > > > EAL: Detected lcore 1 as core 0 on socket 0
> > > > > EAL: Support maximum 128 logical core(s) by configuration.
> > > > > EAL: Detected 2 lcore(s)
> > > > > EAL: Detected 1 NUMA nodes
> > > > > EAL: Checking presence of .so 'librte_eal.so.21.3'
> > > > > EAL: Checking presence of .so 'librte_eal.so.21'
> > > > > EAL: Checking presence of .so 'librte_eal.so'
> > > > > EAL: Detected static linkage of DPDK
> > > > > EAL: Ask a virtual area of 0x7000 bytes
> > > > > EAL: Virtual area found at 0x1 (size = 0x7000)
> > > > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > > > > EAL: DPAA Bus not present. Skipping.
> > > > > EAL: VFIO PCI modules not loaded
> > > > > EAL: Selected IOVA mode 'VA'
> > > > > EAL: Probing VFIO support...
> > > > > EAL: IOMMU type 1 (Type 1) is supported
> > > > > EAL: IOMMU type 7 (sPAPR) is not supported
> > > > > EAL: IOMMU type 8 (No-IOMMU) is supported
> > > > > EAL: VFIO support initialized
> > > > > EAL: Ask a virtual area of 0x5b000 bytes
> > > > > EAL: Virtual area found at 0x17000 (size = 0x5b000)
> > > > > EAL: Setting up physically contiguous memory...
> > > > > EAL: Setting maximum number of open files to 1048576
> > > > > EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> > > > > EAL: Creating 2 segment lists: n_segs:128 socket_id:0
> > > > hugepage_sz:1073741824
> > > > > EAL: Ask a virtual area of 0x2000 bytes
> > > > > EAL: Virtual area found at 0x100062000 (size = 0x2000)
> > > > > EAL: Memseg list allocated at socket 0, page size 0x10kB
> > > > > EAL: Ask a virtual area of 0x20 bytes
> > > > > EAL: Virtual area found at 0x14000 (size = 0x20)
> > > > > EAL: VA reserved for memseg list at 0x14000, size 20
> > > > > EAL: Ask a virtual area of 0x2000 bytes
> > > > > EAL: Virtual area found at 0x214000 (size = 0x2000)
> > > > > EAL: Memseg list allocated at socket 0, page size 0x10kB
> > > > > EAL: Ask a virtual area of 0x20 bytes
> > > > > EAL: Virtual area found at 0x218000 (size = 0x20)
> > > > > EAL: VA reserved for memseg list at 0x218000, size 20
> > > > > EAL: TSC frequency is ~235 KHz
> > > > > EAL: Main lcore 0 is ready (tid=7f8ad790ec00;cpuset=[0])
> > > > > EAL: lcore 1 is ready (tid=7f8ad6907400;cpuset=[1])
> > > > > EAL:

Re: help

[Thomas Monjalon]

+Cc the vmxnet3 maintainer.

Please Jochen, do you have an idea what's wrong below?


20/07/2023 11:25, Igor de Paula:
> This is because it can't negotiate the IOMMU type with any port.
> 
> On Thu, Jul 20, 2023 at 5:08 AM Thomas Monjalon  wrote:
> 
> > Hello,
> >
> > The first error is "Cause: Error: number of ports must be even"
> >
> >
> > 03/05/2023 18:13, Igor de Paula:
> > > I am running a VM inside a VMWARE server (vSphere).
> > > My goal it to set up DPDK with two HW ports, and set up a virtio_user to
> > > interact with the kernel stack.
> > > In another app I have it working but instead of virtio_user I am running
> > > KNI, it works in IOVA-PA mode.
> > > I am looking to replace the KNI.
> > >
> > > When I try to set up virtio_user port as in the doc:
> > >
> > https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path
> > > I get a error it can't run in PA mode.
> > >
> > >
> > > When I try to run as VA mode from a parameter, I get the following
> > errors:
> > > EAL: lib.eal log level changed from info to debug
> > > EAL: Detected lcore 0 as core 0 on socket 0
> > > EAL: Detected lcore 1 as core 0 on socket 0
> > > EAL: Support maximum 128 logical core(s) by configuration.
> > > EAL: Detected 2 lcore(s)
> > > EAL: Detected 1 NUMA nodes
> > > EAL: Checking presence of .so 'librte_eal.so.21.3'
> > > EAL: Checking presence of .so 'librte_eal.so.21'
> > > EAL: Checking presence of .so 'librte_eal.so'
> > > EAL: Detected static linkage of DPDK
> > > EAL: Ask a virtual area of 0x7000 bytes
> > > EAL: Virtual area found at 0x1 (size = 0x7000)
> > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > > EAL: DPAA Bus not present. Skipping.
> > > EAL: VFIO PCI modules not loaded
> > > EAL: Selected IOVA mode 'VA'
> > > EAL: Probing VFIO support...
> > > EAL: IOMMU type 1 (Type 1) is supported
> > > EAL: IOMMU type 7 (sPAPR) is not supported
> > > EAL: IOMMU type 8 (No-IOMMU) is supported
> > > EAL: VFIO support initialized
> > > EAL: Ask a virtual area of 0x5b000 bytes
> > > EAL: Virtual area found at 0x17000 (size = 0x5b000)
> > > EAL: Setting up physically contiguous memory...
> > > EAL: Setting maximum number of open files to 1048576
> > > EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> > > EAL: Creating 2 segment lists: n_segs:128 socket_id:0
> > hugepage_sz:1073741824
> > > EAL: Ask a virtual area of 0x2000 bytes
> > > EAL: Virtual area found at 0x100062000 (size = 0x2000)
> > > EAL: Memseg list allocated at socket 0, page size 0x10kB
> > > EAL: Ask a virtual area of 0x20 bytes
> > > EAL: Virtual area found at 0x14000 (size = 0x20)
> > > EAL: VA reserved for memseg list at 0x14000, size 20
> > > EAL: Ask a virtual area of 0x2000 bytes
> > > EAL: Virtual area found at 0x214000 (size = 0x2000)
> > > EAL: Memseg list allocated at socket 0, page size 0x10kB
> > > EAL: Ask a virtual area of 0x20 bytes
> > > EAL: Virtual area found at 0x218000 (size = 0x20)
> > > EAL: VA reserved for memseg list at 0x218000, size 20
> > > EAL: TSC frequency is ~235 KHz
> > > EAL: Main lcore 0 is ready (tid=7f8ad790ec00;cpuset=[0])
> > > EAL: lcore 1 is ready (tid=7f8ad6907400;cpuset=[1])
> > > EAL: Trying to obtain current memory policy.
> > > EAL: Setting policy MPOL_PREFERRED for socket 0
> > > EAL: Restoring previous memory policy: 0
> > > EAL: request: mp_malloc_sync
> > > EAL: Heap on socket 0 was expanded by 1024MB
> > > EAL: PCI device :0b:00.0 on NUMA socket -1
> > > EAL:   probe driver: 15ad:7b0 net_vmxnet3
> > > EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not
> > initializing
> > > EAL: Requested device :0b:00.0 cannot be used
> > > EAL: PCI device :13:00.0 on NUMA socket -1
> > > EAL:   probe driver: 15ad:7b0 net_vmxnet3
> > > EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not
> > initializing
> > > EAL: Requested device :13:00.0 cannot be used
> > > EAL: Bus (pci) probe failed.
> > > EAL: lib.telemetry log level changed from disabled to warning
> > > EAL: Error - exiting with code: 1
> > >   Cause: Error: number of ports must be even
> > > EAL: request: mp_malloc_sync
> > > 

Re: help

[Thomas Monjalon]

Hello,

The first error is "Cause: Error: number of ports must be even"


03/05/2023 18:13, Igor de Paula:
> I am running a VM inside a VMWARE server (vSphere).
> My goal it to set up DPDK with two HW ports, and set up a virtio_user to
> interact with the kernel stack.
> In another app I have it working but instead of virtio_user I am running
> KNI, it works in IOVA-PA mode.
> I am looking to replace the KNI.
> 
> When I try to set up virtio_user port as in the doc:
> https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path
> I get a error it can't run in PA mode.
> 
> 
> When I try to run as VA mode from a parameter, I get the following errors:
> EAL: lib.eal log level changed from info to debug
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 0 on socket 0
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 2 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Checking presence of .so 'librte_eal.so.21.3'
> EAL: Checking presence of .so 'librte_eal.so.21'
> EAL: Checking presence of .so 'librte_eal.so'
> EAL: Detected static linkage of DPDK
> EAL: Ask a virtual area of 0x7000 bytes
> EAL: Virtual area found at 0x1 (size = 0x7000)
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: DPAA Bus not present. Skipping.
> EAL: VFIO PCI modules not loaded
> EAL: Selected IOVA mode 'VA'
> EAL: Probing VFIO support...
> EAL: IOMMU type 1 (Type 1) is supported
> EAL: IOMMU type 7 (sPAPR) is not supported
> EAL: IOMMU type 8 (No-IOMMU) is supported
> EAL: VFIO support initialized
> EAL: Ask a virtual area of 0x5b000 bytes
> EAL: Virtual area found at 0x17000 (size = 0x5b000)
> EAL: Setting up physically contiguous memory...
> EAL: Setting maximum number of open files to 1048576
> EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> EAL: Creating 2 segment lists: n_segs:128 socket_id:0 hugepage_sz:1073741824
> EAL: Ask a virtual area of 0x2000 bytes
> EAL: Virtual area found at 0x100062000 (size = 0x2000)
> EAL: Memseg list allocated at socket 0, page size 0x10kB
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x14000 (size = 0x20)
> EAL: VA reserved for memseg list at 0x14000, size 20
> EAL: Ask a virtual area of 0x2000 bytes
> EAL: Virtual area found at 0x214000 (size = 0x2000)
> EAL: Memseg list allocated at socket 0, page size 0x10kB
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x218000 (size = 0x20)
> EAL: VA reserved for memseg list at 0x218000, size 20
> EAL: TSC frequency is ~235 KHz
> EAL: Main lcore 0 is ready (tid=7f8ad790ec00;cpuset=[0])
> EAL: lcore 1 is ready (tid=7f8ad6907400;cpuset=[1])
> EAL: Trying to obtain current memory policy.
> EAL: Setting policy MPOL_PREFERRED for socket 0
> EAL: Restoring previous memory policy: 0
> EAL: request: mp_malloc_sync
> EAL: Heap on socket 0 was expanded by 1024MB
> EAL: PCI device :0b:00.0 on NUMA socket -1
> EAL:   probe driver: 15ad:7b0 net_vmxnet3
> EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not initializing
> EAL: Requested device :0b:00.0 cannot be used
> EAL: PCI device :13:00.0 on NUMA socket -1
> EAL:   probe driver: 15ad:7b0 net_vmxnet3
> EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not initializing
> EAL: Requested device :13:00.0 cannot be used
> EAL: Bus (pci) probe failed.
> EAL: lib.telemetry log level changed from disabled to warning
> EAL: Error - exiting with code: 1
>   Cause: Error: number of ports must be even
> EAL: request: mp_malloc_sync
> EAL: Heap on socket 0 was shrunk by 1024MB
> 
> 
> 
> For some reason the HW ports won't setup. From what I understand
> net_vmxnet3 should work with VA mode.
> I enabled I/OMUU for the VM.
> The weird thing even when enabled, I still have the
> enable_unsafe_noiommu_mode flag on.
> And because it's on the this:
> 
> dev_iova_mode = pci_device_iova_mode(dr, dev);
> 
> return PA mode, and it fails.
> 
> When I disable it by modifying
> /sys/module/vfio/parameters/enable_unsafe_noiommu_mode, I get another error.
> The error is that it doesn't find a suitable IOMMU type:
> Just putting the relevant message:
> 
> 
> EAL: Heap on socket 0 was expanded by 1024MB
> EAL: PCI device :0b:00.0 on NUMA socket -1
> EAL:   probe driver: 15ad:7b0 net_vmxnet3
> EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device)
> EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device)
> EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such device)
> EAL: :0b:00.0 failed to select IOMMU type
> EAL: Requested device :0b:00.0 cannot be used
> EAL: PCI device :13:00.0 on NUMA socket -1
> EAL:   probe driver: 15ad:7b0 net_vmxnet3
> EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device)
> EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device)
> EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No 

Re: Generic flow string parser

[Thomas Monjalon]

This thread is an API suggestion, it should be discussed in
the developer mailing list (did the Cc here).

29/04/2023 16:23, Cliff Burdick:
> > Would rather the flow parser was rewritten as well. Doing open coded
> > parser is much more error prone and hard to extend versus writing the
> > parser in yacc/lex (ie bison/flex).
> 
> I agree, and that's kind of why the original suggestion of using testpmd
> came from. Writing a new parser is obviously the better choice and would
> have been great if testpmd started that way, but a significant amount of
> time was invested in that method. Since it works and is tested, it didn't
> seem like a bad request to build off that and bring that code into an rte_
> API. I'd imagine building a proper parser would not just require the parser
> piece, but also making sure all the tests now use that, and also the legacy
> testpmd was converted. It seemed unlikely all of this could be done in a
> reasonable amount of time and a lot of input from many people. Given the
> amount of debugging I (and others) have spent on figuring on why a flow
> spec didn't work properly, this could be a huge timesaver for new projects
> like Tom mentioned.
> 
> On Fri, Apr 28, 2023 at 5:04 PM Stephen Hemminger <
> step...@networkplumber.org> wrote:
> 
> > On Fri, 28 Apr 2023 12:13:26 -0700
> > Cliff Burdick  wrote:
> >
> > > Hi Stephen, it would definitely not be worthwhile to repeat everything
> > > that's already tested with testpmd. I was thinking that given that there
> > > already is a "flow_parse" function that does almost everything needed,
> > > something like that could be exposed. If we think of the testpmd flow
> > > string as a sort of "IR" for string flow specification, that would allow
> > > others to implement higher-level transform of a schema like JSON or YAML
> > > into the testpmd language. Due to the complexity of testpmd and how it's
> > > the source of true for testing flows, I think it's too great of an ask to
> > > have testpmd support a new type of parsing. My only suggestion would be
> > > to take what already exists and expose it in a public API that is included
> > > in a DPDK install.

So the only things we need are 2 functions, if I understand well:

int rte_flow_to_text(const struct rte_flow*);
struct rte_flow *rte_flow_from_text(const char *);

Here I assume the output of rte_flow_from_text() would be a created flow,
meaning it calls rte_flow_create() under the hood.
Is it what you wish?
Or should it fill port ID, attributes, patterns and actions?

Re: Generic flow string parser

[Thomas Monjalon]

26/04/2023 07:47, David Marchand:
> On Wed, Apr 26, 2023 at 6:47 AM Cliff Burdick  wrote:
> >
> > Does anyone know if a generic parser for flow strings exists anywhere? The 
> > one inside of testpmd is ideal, but unfortunately it's self-contained and 
> > not distributed as part of a normal DPDK install. This seems like something 
> > that is likely reinvented over and over and it would be useful if there was 
> > a single API to take in strings and generate flows.
> 
> I heard this same question in the past, but I don't remember the answer.
> Copying Thomas and Ori who might know.

I'm not sure how the testpmd code could help another application.
And in general, if your application has a CLI,
you need to integrate the flow commands in a broader context.

Re: Running testpmd as non-root user with uio_pci_generic

[Thomas Monjalon]

14/01/2023 23:17, Isaac Boukris:
> Hi,
> 
> I tried to run testpmd as a non-root user with uio_pci_generic (i.e.
> not vfio-pci) on a vmxnet3 interface by setting the
> 'cap_ipc_lock,cap_sys_admin' capabilities as according to the doc at:
> https://doc.dpdk.org/guides-21.11/linux_gsg/enable_func.html
> 
> But that didn't work and I was still getting the documented error:
> EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Permission denied
> 
> I dug a little and found that I had to add the 'cap_dac_override' as
> well and then it worked, the hint was at (which also includes a small
> demo program): https://bugs.centos.org/view.php?id=17176
> 
> I thought it was worth sharing as I have seen it being asked a couple of 
> times.

Thank you for reporting.

The preferred solution is to use the capability DAC_READ_SEARCH.

The DPDK doc is updated:
https://git.dpdk.org/dpdk/commit/?id=50b567c66da268bcc
(so you become a DPDK contributor :)

Re: USB device support in DPDK

[Thomas Monjalon]

02/12/2022 14:01, John Bize:
> Will DPDK support USB-Ethernet devices?

If there is a need and an advantage for USB devices
to be supported in DPDK, why not?

In general, DPDK supports what is contributed and maintained.

Re: [ovs-dev] [RFC PATCH v2] dpdk: Update to use v22.11.

[Thomas Monjalon]

30/11/2022 16:30, Stokes, Ian:
> > Hi Ian,
> > 
> > Cc: Thomas and John
> > 
> > On Wed, Nov 23, 2022 at 12:52 PM Ian Stokes  wrote:
> > > diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> > > index 23c8bbb7a..0aa90e55e 100755
> > > --- a/.ci/linux-build.sh
> > > +++ b/.ci/linux-build.sh
> > > @@ -142,7 +142,7 @@ function install_dpdk()
> > >  fi
> > >  # No cache or version mismatch.
> > >  rm -rf dpdk-dir
> > > -wget https://fast.dpdk.org/rel/dpdk-$1.tar.xz
> > > +wget https://git.dpdk.org/dpdk/snapshot/dpdk-$1.tar.xz
> > >  tar xvf dpdk-$1.tar.xz > /dev/null
> > >  DIR_NAME=$(tar -tf dpdk-$1.tar.xz | head -1 | cut -f1 -d"/")
> > >  mv ${DIR_NAME} dpdk-dir
> > 
> > dpdk.org server experienced a quite heavy load this morning, with many
> > clients requesting git related operations through http.
> > I am not sure what or who triggered this, but I just remembered this patch 
> > here.
> > 
> > Downloading a rc4 tarball through dpdk.org cgit is to be avoided.
> > dpdk.org server does not cache this kind of operations, it is heavy on
> > cpu, memory and bandwidth.
> > 
> > 
> > I think OVS should switch to the github mirror, this would work for
> > both rc and final release tarballs.
> > IOW: wget https://github.com/DPDK/dpdk/archive/refs/tags/v$1.tar.gz
> > 
> > One drawback is that github does not seem to provide xz compressed
> > tarballs, so you would need to update the script further.
> > 
> 
> Thanks for the input David, I've posted a v3 but didn’t include this change 
> as I wanted to discuss a bit further. 
> 
> So we use DPDK.org in our ci but we also make reference to its use throughout 
> the OVS documentation when users wish to download DPDK, should this also be 
> replaced with github?
> 
> I guess to my mind if github is the preferred method to retrieve the repo 
> then does it raise the question of dpdk org in general providing the 
> releases? Is there a plan to transition completely to github for DPDK in the 
> future?
> 
> If the preferred method is to go ahead with github then thats OK on my side 
> but I think it would be better to keep that change in a separate patch along 
> with an explanation as to why github should be used rather than dpdk.org as 
> we've been using the current approach for a few years without issue.

fast.dpdk.org is fine and preferred.
git.dpdk.org should not be used for tarball download in general.
github.com/DPDK is OK for tarball download, including release candidates.



___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[kmailtransport] [Bug 392138] First SMTP authentication fails for almost every emails

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=392138

Thomas Monjalon  changed:

   What|Removed |Added

 Resolution|WAITINGFORINFO  |UNMAINTAINED
 Status|NEEDSINFO   |RESOLVED

--- Comment #3 from Thomas Monjalon  ---
I use STARTTLS, so cannot say if the bug is still there without it.
As nobody tried to reproduce after 4 years, closing it.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmailtransport] [Bug 392138] First SMTP authentication fails for almost every emails

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=392138

Thomas Monjalon  changed:

   What|Removed |Added

 Resolution|WAITINGFORINFO  |UNMAINTAINED
 Status|NEEDSINFO   |RESOLVED

--- Comment #3 from Thomas Monjalon  ---
I use STARTTLS, so cannot say if the bug is still there without it.
As nobody tried to reproduce after 4 years, closing it.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kmail2] [Bug 384255] Idea to suggest latest contacts in recipient picker

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=384255

Thomas Monjalon  changed:

   What|Removed |Added

 Status|NEEDSINFO   |RESOLVED
 Resolution|WAITINGFORINFO  |FIXED

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmail2] [Bug 384255] Idea to suggest latest contacts in recipient picker

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=384255

Thomas Monjalon  changed:

   What|Removed |Added

 Status|NEEDSINFO   |RESOLVED
 Resolution|WAITINGFORINFO  |FIXED

-- 
You are receiving this mail because:
You are watching all bug changes.

Re: OpenWRT Related Question

[Thomas Monjalon]

21/07/2022 03:08, R T:
> On Wednesday, July 20, 2022 at 06:03:32 AM EDT, Thomas Monjalon 
>  wrote:  
> 
> > We have this howto page about OpenWRT:
> > https://doc.dpdk.org/guides/howto/openwrt.html
> > I am not sure how much it is up to date,
> > please do not hesitate to give feedback if anything can be improved.
> 
> One of the items that I'm not seeing, where the regular mason build env on 
> Ubuntu was pretty clear on, is building any of the example for testing on 
> OpenWRT.
> How would I go about building and installing one or more of the DPDK examples 
> on the OpenWRT Image?

It should not be different of any other OS.
You can compile examples either by adding -Dexamples=all in meson command,
or by calling make in the example directory.

Re: OpenWRT Related Question

[Thomas Monjalon]

Hello,

20/07/2022 06:15, R T:
> Hello Folks,I've been writing a small application on Ubuntu 20.04 that uses 
> DPDK.  I started with some of the simpler examples such as l2fwd and 
> packet_ordering and added some other function I wanted to experiment with.  
> I'm getting good packet throughput and the library is working well.  Overall, 
> DPDK is feature rich, easy to compile, and working as advertised.

Great to read, thanks.

> As next steps, I would like to move my application to run on top of a very 
> small Linux image that can be loaded off of a very small flash or even over 
> the network.OpenWRT checks all the boxes for being small, having a lot of 
> network utilities, and supporting DPDK.
> Looking at the documents, the OpenWRT docs is pretty small and I don't see 
> mmay uch discussions about it on the archives of this mailing list.
> I'm asking if people could provide some feedback on writing a DPDK app for 
> use on OpenWRT.
> Is the advertised integration working well?How easy is it to cross compile an 
> app to run on OpenWRT (x86 in m y case)?
> I would welcome any comment on this topic

We have this howto page about OpenWRT:
https://doc.dpdk.org/guides/howto/openwrt.html
I am not sure how much it is up to date,
please do not hesitate to give feedback if anything can be improved.

Re: Fwd: QOS sample example.

[Thomas Monjalon]

+Cc QoS scheduler maintainers (see file MAINTAINERS)

31/03/2022 18:59, satish amara:
> Hi,
> I am trying to understand the QOS sample scheduler application code.
> Trying to understand what is tc_period in the config.
> 30. QoS Scheduler Sample Application — Data Plane Development Kit 21.05.0
> documentation (dpdk.org)
>  Is
> tc_period same as  tb_period
> tb_period Bytes Time period that should elapse since the last credit update
> in order for the bucket to be awarded tb_credits_per_period worth or
> credits.
> Regards,
> Satish Amara
> 

Re: [ovs-dev] OVS DPDK DSA Architecture/Design Discussion

[Thomas Monjalon]

15/03/2022 12:14, Stokes, Ian:
> > 15/03/2022 11:27, Stokes, Ian:
> > > Hi All,
> > >
> > > We'd like to put a public meeting in place for the stakeholders
> > > of DPDK and OVS to discuss the next steps and design of
> > > the DSA library along with its integration in OVS.
> > 
> > There is no DSA library.
> > Do you mean the IOAT driver in DPDK for the DMA library?
> 
> Apologies Thomas,
> 
> DMA-Dev library is actually what's intended to be discussed to clarify.
> Will re-send the invite with the correct title.

If you want to have a good number of attendees,
you should postpone to a later date, so we can plan in advance.
You also need to describe the problem to discuss.
That's really better to start with emails, and to have a summary
after the meeting.

> > If it is just for discussing the Intel driver, no way I attend.
> > And even if it is for discussing the DMA library integration,
> > I don't understand why the DPDK community is required.
> > 
> > If there is an issue with the library API, please tell it.
> > We need email discussions, not meeting request sent few hours in advance.
> > Ian, I know you are not a beginner in the community, what happens?

So if I understand well, there is no urgent problem to discuss?


___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] OVS DPDK DSA Architecture/Design Discussion

[Thomas Monjalon]

15/03/2022 11:27, Stokes, Ian:
> Hi All,
> 
> We'd like to put a public meeting in place for the stakeholders
> of DPDK and OVS to discuss the next steps and design of
> the DSA library along with its integration in OVS.

There is no DSA library.
Do you mean the IOAT driver in DPDK for the DMA library?

If it is just for discussing the Intel driver, no way I attend.
And even if it is for discussing the DMA library integration,
I don't understand why the DPDK community is required.

If there is an issue with the library API, please tell it.
We need email discussions, not meeting request sent few hours in advance.
Ian, I know you are not a beginner in the community, what happens?


___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: ConnectX5 Setup with DPDK

[Thomas Monjalon]

25/02/2022 19:29, Aaron Lee:
> Hi Thomas,
> 
> I was doing some more testing and wanted to increase the RX queues for the
> CX5 but was wondering how I could do that. I see in the usage example in
> the docs, I could pass in --rxq=2 --txq=2 to set the queues to 2 each but I
> don't see that in my output when I run the command. Below is the output
> from running the command in
> https://doc.dpdk.org/guides/nics/mlx5.html#usage-example. Does this mean
> that the MCX515A-CCAT I have can't support more than 1 queue or am I
> supposed to configure another setting?

I see nothing about the number of queues in your output.
You should try the command "show config rxtx".


> EAL: Detected 80 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: :af:00.0 (socket 1)
> mlx5_pci: Size 0x is not power of 2, will be aligned to 0x1.
> EAL: No legacy callbacks, legacy socket not created
> Interactive-mode selected
> testpmd: create a new mbuf pool : n=203456, size=2176, socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
> testpmd: create a new mbuf pool : n=203456, size=2176, socket=1
> testpmd: preferred mempool ops selected: ring_mp_mc
> 
> Warning! port-topology=paired and odd forward ports number, the last port
> will pair with itself.
> 
> Configuring Port 0 (socket 1)
> mlx5_pci: Failed to init cache list FDB_ingress_0_matcher_cache entry (nil).
> mlx5_pci: Failed to init cache list FDB_ingress_0_matcher_cache entry (nil).
> mlx5_pci: Failed to init cache list FDB_ingress_0_matcher_cache entry (nil).
> mlx5_pci: Failed to init cache list FDB_ingress_0_matcher_cache entry (nil).
> Port 0: EC:0D:9A:68:21:A8
> Checking link statuses...
> Done
> mlx5_pci: Failed to init cache list FDB_ingress_0_matcher_cache entry (nil).
> 
> Best,
> Aaron

Re: ConnectX5 Setup with DPDK

[Thomas Monjalon]

21/02/2022 21:10, Aaron Lee:
> Hi Thomas,
> 
> Actually I remembered in my previous setup I had run dpdk-devbind.py to
> bind the mlx5 NIC to igb_uio. I read somewhere that you don't need to do
> this and just wanted to confirm that this is correct.

Indeed, mlx5 PMD runs on top of mlx5 kernel driver.
We don't need UIO or VFIO drivers.
The kernel modules must remain loaded and can be used in the same time.
When DPDK is working, the traffic goes to the userspace PMD by default,
but it is possible to configure some flows to go directly to the kernel driver.
This behaviour is called "bifurcated model".


> On Mon, Feb 21, 2022 at 11:45 AM Aaron Lee  wrote:
> 
> > Hi Thomas,
> >
> > I tried installing things from scratch two days ago and have gotten
> > things working! I think part of the problem was figuring out the correct
> > hugepage allocation for my system. If I recall correctly, I tried setting
> > up my system with default page size 1G but perhaps didn't have enough pages
> > allocated at the time. Currently have the following which gives me the
> > output you've shown previously.
> >
> > root@yeti-04:~/dpdk-21.11# usertools/dpdk-hugepages.py -s
> > Node Pages Size Total
> > 0161Gb16Gb
> > 1161Gb16Gb
> >
> > root@yeti-04:~/dpdk-21.11# echo show port summary all |
> > build/app/dpdk-testpmd --in-memory -- -i
> > EAL: Detected CPU lcores: 80
> > EAL: Detected NUMA nodes: 2
> > EAL: Detected static linkage of DPDK
> > EAL: Selected IOVA mode 'PA'
> > EAL: No free 2048 kB hugepages reported on node 0
> > EAL: No free 2048 kB hugepages reported on node 1
> > EAL: No available 2048 kB hugepages reported
> > EAL: VFIO support initialized
> > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: :af:00.0 (socket 1)
> > TELEMETRY: No legacy callbacks, legacy socket not created
> > Interactive-mode selected
> > testpmd: create a new mbuf pool : n=779456, size=2176, socket=0
> > testpmd: preferred mempool ops selected: ring_mp_mc
> > testpmd: create a new mbuf pool : n=779456, size=2176, socket=1
> > testpmd: preferred mempool ops selected: ring_mp_mc
> >
> > Warning! port-topology=paired and odd forward ports number, the last port
> > will pair with itself.
> >
> > Configuring Port 0 (socket 1)
> > Port 0: EC:0D:9A:68:21:A8
> > Checking link statuses...
> > Done
> > testpmd> show port summary all
> > Number of available ports: 1
> > Port MAC Address   Name Driver Status   Link
> > 0EC:0D:9A:68:21:A8 :af:00.0 mlx5_pci   up   100 Gbps
> >
> > Best,
> > Aaron
> >
> > On Mon, Feb 21, 2022 at 11:03 AM Thomas Monjalon 
> > wrote:
> >
> >> 21/02/2022 19:52, Thomas Monjalon:
> >> > 18/02/2022 22:12, Aaron Lee:
> >> > > Hello,
> >> > >
> >> > > I'm trying to get my ConnectX5 NIC working with DPDK v21.11 but I'm
> >> > > wondering if the card I have simply isn't compatible. I first noticed
> >> that
> >> > > the model I was given is MCX515A-CCA_Ax_Bx. Below are some of the
> >> error
> >> > > logs when running dpdk-pdump.
> >> >
> >> > When testing a NIC, it is more convenient to use dpdk-testpmd.
> >> >
> >> > > EAL: Detected CPU lcores: 80
> >> > > EAL: Detected NUMA nodes: 2
> >> > > EAL: Detected static linkage of DPDK
> >> > > EAL: Multi-process socket
> >> /var/run/dpdk/rte/mp_socket_383403_1ac7441297c92
> >> > > EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such
> >> file or
> >> > > directory
> >> > > EAL: Fail to send request /var/run/dpdk/rte/mp_socket:bus_vdev_mp
> >> > > vdev_scan(): Failed to request vdev from primary
> >> > > EAL: Selected IOVA mode 'PA'
> >> > > EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such
> >> file or
> >> > > directory
> >> > > EAL: Fail to send request /var/run/dpdk/rte/mp_socket:eal_vfio_mp_sync
> >> > > EAL: Cannot request default VFIO container fd
> >> > > EAL: VFIO support could not be initialized
> >> > > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: :af:00.0
> >> (socket 1)
> >> > > EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such
> >> file or
> >> > > directory
> >> > > EAL: Fail to send request /var/run/dpdk/rte/mp_socket:common_mlx5_mp
> >

Re: ConnectX5 Setup with DPDK

[Thomas Monjalon]

21/02/2022 19:52, Thomas Monjalon:
> 18/02/2022 22:12, Aaron Lee:
> > Hello,
> > 
> > I'm trying to get my ConnectX5 NIC working with DPDK v21.11 but I'm
> > wondering if the card I have simply isn't compatible. I first noticed that
> > the model I was given is MCX515A-CCA_Ax_Bx. Below are some of the error
> > logs when running dpdk-pdump.
> 
> When testing a NIC, it is more convenient to use dpdk-testpmd.
> 
> > EAL: Detected CPU lcores: 80
> > EAL: Detected NUMA nodes: 2
> > EAL: Detected static linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket_383403_1ac7441297c92
> > EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> > directory
> > EAL: Fail to send request /var/run/dpdk/rte/mp_socket:bus_vdev_mp
> > vdev_scan(): Failed to request vdev from primary
> > EAL: Selected IOVA mode 'PA'
> > EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> > directory
> > EAL: Fail to send request /var/run/dpdk/rte/mp_socket:eal_vfio_mp_sync
> > EAL: Cannot request default VFIO container fd
> > EAL: VFIO support could not be initialized
> > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: :af:00.0 (socket 1)
> > EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> > directory
> > EAL: Fail to send request /var/run/dpdk/rte/mp_socket:common_mlx5_mp
> > mlx5_common: port 0 request to primary process failed
> > mlx5_net: probe of PCI device :af:00.0 aborted after encountering an
> > error: No such file or directory
> > mlx5_common: Failed to load driver mlx5_eth
> > EAL: Requested device :af:00.0 cannot be used
> > EAL: Error - exiting with code: 1
> >   Cause: No Ethernet ports - bye
> 
> From this log, we miss the previous steps before running the application.
> 
> Please check these simple steps:
> - install rdma-core
> - build dpdk (meson build && ninja -C build)
> - reserve hugepages (usertools/dpdk-hugepages.py -r 1G)
> - run testpmd (echo show port summary all | build/app/dpdk-testpmd 
> --in-memory -- -i)
> 
> EAL: Detected CPU lcores: 10
> EAL: Detected NUMA nodes: 1
> EAL: Detected static linkage of DPDK
> EAL: Selected IOVA mode 'PA'
> EAL: Probe PCI driver: mlx5_pci (15b3:101f) device: :08:00.0 (socket 0)
> Interactive-mode selected
> testpmd: create a new mbuf pool : n=219456, size=2176, socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
> Configuring Port 0 (socket 0)
> Port 0: 0C:42:A1:D6:E0:00
> Checking link statuses...
> Done
> testpmd> show port summary all
> Number of available ports: 1
> Port MAC Address   Name Driver Status   Link
> 00C:42:A1:D6:E0:00 08:00.0  mlx5_pci   up   25 Gbps
> 
> > I noticed that the pci id of the card I was given is 15b3:1017 as below.
> > This sort of indicates to me that the PMD driver isn't supported on this
> > card.
> 
> This card is well supported and even officially tested with DPDK 21.11,
> as you can see in the release notes:
> https://doc.dpdk.org/guides/rel_notes/release_21_11.html#tested-platforms
> 
> > af:00.0 Ethernet controller [0200]: Mellanox Technologies MT27800 Family
> > [ConnectX-5] [15b3:1017]
> > 
> > I'd appreciate it if someone has gotten this card to work with DPDK to
> > point me in the right direction or if my suspicions were correct that this
> > card doesn't work with the PMD.

If you want to check which hardware is supported by a PMD,
you can use this command:

usertools/dpdk-pmdinfo.py build/drivers/librte_net_mlx5.so  
PMD NAME: mlx5_eth
PMD KMOD DEPENDENCIES: * ib_uverbs & mlx5_core & mlx5_ib
PMD HW SUPPORT:
 Mellanox Technologies (15b3) : MT27700 Family [ConnectX-4] (1013) (All 
Subdevices)
 Mellanox Technologies (15b3) : MT27700 Family [ConnectX-4 Virtual Function] 
(1014) (All Subdevices)
 Mellanox Technologies (15b3) : MT27710 Family [ConnectX-4 Lx] (1015) (All 
Subdevices)
 Mellanox Technologies (15b3) : MT27710 Family [ConnectX-4 Lx Virtual Function] 
(1016) (All Subdevices)
 Mellanox Technologies (15b3) : MT27800 Family [ConnectX-5] (1017) (All 
Subdevices)
 Mellanox Technologies (15b3) : MT27800 Family [ConnectX-5 Virtual Function] 
(1018) (All Subdevices)
 Mellanox Technologies (15b3) : MT28800 Family [ConnectX-5 Ex] (1019) (All 
Subdevices)
 Mellanox Technologies (15b3) : MT28800 Family [ConnectX-5 Ex Virtual Function] 
(101a) (All Subdevices)
 Mellanox Technologies (15b3) : MT416842 BlueField integrated ConnectX-5 
network controller (a2d2) (All Subdevices)
 Mellanox Technologies (15b3) : MT416842 BlueField multicore SoC family VF 
(a2d3) (All Subdevices)
 Mellanox Technologies (15b3) : MT28908 Fami

Re: ConnectX5 Setup with DPDK

[Thomas Monjalon]

18/02/2022 22:12, Aaron Lee:
> Hello,
> 
> I'm trying to get my ConnectX5 NIC working with DPDK v21.11 but I'm
> wondering if the card I have simply isn't compatible. I first noticed that
> the model I was given is MCX515A-CCA_Ax_Bx. Below are some of the error
> logs when running dpdk-pdump.

When testing a NIC, it is more convenient to use dpdk-testpmd.

> EAL: Detected CPU lcores: 80
> EAL: Detected NUMA nodes: 2
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket_383403_1ac7441297c92
> EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> directory
> EAL: Fail to send request /var/run/dpdk/rte/mp_socket:bus_vdev_mp
> vdev_scan(): Failed to request vdev from primary
> EAL: Selected IOVA mode 'PA'
> EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> directory
> EAL: Fail to send request /var/run/dpdk/rte/mp_socket:eal_vfio_mp_sync
> EAL: Cannot request default VFIO container fd
> EAL: VFIO support could not be initialized
> EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: :af:00.0 (socket 1)
> EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> directory
> EAL: Fail to send request /var/run/dpdk/rte/mp_socket:common_mlx5_mp
> mlx5_common: port 0 request to primary process failed
> mlx5_net: probe of PCI device :af:00.0 aborted after encountering an
> error: No such file or directory
> mlx5_common: Failed to load driver mlx5_eth
> EAL: Requested device :af:00.0 cannot be used
> EAL: Error - exiting with code: 1
>   Cause: No Ethernet ports - bye

>From this log, we miss the previous steps before running the application.

Please check these simple steps:
- install rdma-core
- build dpdk (meson build && ninja -C build)
- reserve hugepages (usertools/dpdk-hugepages.py -r 1G)
- run testpmd (echo show port summary all | build/app/dpdk-testpmd --in-memory 
-- -i)

EAL: Detected CPU lcores: 10
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Selected IOVA mode 'PA'
EAL: Probe PCI driver: mlx5_pci (15b3:101f) device: :08:00.0 (socket 0)
Interactive-mode selected
testpmd: create a new mbuf pool : n=219456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 0 (socket 0)
Port 0: 0C:42:A1:D6:E0:00
Checking link statuses...
Done
testpmd> show port summary all
Number of available ports: 1
Port MAC Address   Name Driver Status   Link
00C:42:A1:D6:E0:00 08:00.0  mlx5_pci   up   25 Gbps

> I noticed that the pci id of the card I was given is 15b3:1017 as below.
> This sort of indicates to me that the PMD driver isn't supported on this
> card.

This card is well supported and even officially tested with DPDK 21.11,
as you can see in the release notes:
https://doc.dpdk.org/guides/rel_notes/release_21_11.html#tested-platforms

> af:00.0 Ethernet controller [0200]: Mellanox Technologies MT27800 Family
> [ConnectX-5] [15b3:1017]
> 
> I'd appreciate it if someone has gotten this card to work with DPDK to
> point me in the right direction or if my suspicions were correct that this
> card doesn't work with the PMD.

Please tell me what drove you into the wrong direction,
because I really would like to improve the documentation & tools.

Re: Initial Setup and best practices

[Thomas Monjalon]

13/02/2022 18:37, Simon Brown:
> Hello,

Hello

> I'm new to DPDK and I'm trying to setup a simple project to count packets. 
> I'm 
> using MoonGen to generate the traffic on one machine and can receive the data 
> using traditional sockets on another machine. So I know that part works.
> 
> I've built dpdk 21.11 and I've tried to modify the example rxtx callbacks 
> application to count packets, but it doesn't see any traffic. So I presume 
> there's something wrong with my environment.

You should explain more what is your setup.

> Can you advise on how to verify that my environment is correct and what is 
> the 
> recommended setup for new projects? Should I be using the virtualisation 
> interface vfio-pci or the other interfaces? I have mlx5, i40e and ice NICs 
> available for test.

There is no best setup I think.
You should check your environment with dpdk-testpmd.

> For mlx5 dpdk-devbind suggests that vfio-pci is compatible

That's interesting, we should take mlx4 and mlx5 as exceptions
in this script because they are bifurcated, i.e. no need of UIO or VFIO.
We should use the accurate info given by dpdk-pmdinfo.py

> whereas mlx5_core 
> is a kernel driver, but trying to run with vfio-pci leads to:
> 
> mlx5_common: No Verbs device matches PCI device :01:00.0, are kernel 
> drivers loaded?
> 
> mlx5 seems to work correctly with MoonGen.

mlx5 is working with its own kernel driver.
The DPDK PMD is negotiating with the kernel driver to get the traffic.
The main benefit is that we can choose some flows to go directly in kernel,
and the rest being managed directly by DPDK, bypassing the kernel.

Re: net_mlx5: unable to recognize master/representors on the multiple IB devices

[Thomas Monjalon]

+Cc mlx5 experts


14/01/2022 11:10, Rocio Dominguez:
> Hi,
> 
> I'm doing a setup with Mellanox ConnectX-4 (MCX416A-CCA) NICs.
> 
> I'm using:
> 
> OS SLES 15 SP2
> DPDK 19.11.4 (the official supported version for SLES 15 SP2)
> MLNX_OFED_LINUX-5.5-1.0.3.2-sles15sp2-x86_64 (the latest one)
> Mellanox adapters firmware 12.28.2006 (corresponding to this MLNX_OFED 
> version)
> kernel 5.3.18-24.34-default
> 
> 
> This is my SRIOV configuration for DPDK capable PCI slots:
> 
> {
> "resourceName": "mlnx_sriov_netdevice",
> "resourcePrefix": "mellanox.com",
> "isRdma": true,
> "selectors": {
> "vendors": ["15b3"],
> "devices": ["1014"],
> "drivers": ["mlx5_core"],
> "pciAddresses": [":d8:00.2", ":d8:00.3", 
> ":d8:00.4", ":d8:00.5"],
> "isRdma": true
> }
> 
> The sriov device plugin starts without problems, the devices are correctly 
> allocated:
> 
> {
>   "cpu": "92",
>   "ephemeral-storage": "419533922385",
>   "hugepages-1Gi": "8Gi",
>   "hugepages-2Mi": "4Gi",
>   "intel.com/intel_sriov_dpdk": "0",
>   "intel.com/sriov_cre": "3",
>   "mellanox.com/mlnx_sriov_netdevice": "4",
>   "mellanox.com/sriov_dp": "0",
>   "memory": "183870336Ki",
>   "pods": "110"
> }
> 
> The Mellanox NICs are binded to the kernel driver mlx5_core:
> 
> pcgwpod009-c04:~ # dpdk-devbind --status
> 
> Network devices using kernel driver
> ===
> :18:00.0 'Ethernet Controller 10G X550T 1563' if=em1 drv=ixgbe 
> unused=vfio-pci
> :18:00.1 'Ethernet Controller 10G X550T 1563' if=em2 drv=ixgbe 
> unused=vfio-pci
> :19:00.0 'Ethernet Controller 10G X550T 1563' if=em3 drv=ixgbe 
> unused=vfio-pci
> :19:00.1 'Ethernet Controller 10G X550T 1563' if=em4 drv=ixgbe 
> unused=vfio-pci
> :3b:00.0 'MT27700 Family [ConnectX-4] 1013' if=enp59s0f0 drv=mlx5_core 
> unused=vfio-pci
> :3b:00.1 'MT27700 Family [ConnectX-4] 1013' if=enp59s0f1 drv=mlx5_core 
> unused=vfio-pci
> :5e:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=p3p1 
> drv=ixgbe unused=vfio-pci
> :5e:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=p3p2 
> drv=ixgbe unused=vfio-pci
> :5e:10.0 '82599 Ethernet Controller Virtual Function 10ed' if= 
> drv=ixgbevf unused=vfio-pci
> :5e:10.2 '82599 Ethernet Controller Virtual Function 10ed' if=p3p1_1 
> drv=ixgbevf unused=vfio-pci
> :5e:10.4 '82599 Ethernet Controller Virtual Function 10ed' if= 
> drv=ixgbevf unused=vfio-pci
> :5e:10.6 '82599 Ethernet Controller Virtual Function 10ed' if=p3p1_3 
> drv=ixgbevf unused=vfio-pci
> :af:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=p4p1 
> drv=ixgbe unused=vfio-pci
> :af:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=p4p2 
> drv=ixgbe unused=vfio-pci
> :d8:00.0 'MT27700 Family [ConnectX-4] 1013' if=enp216s0f0 drv=mlx5_core 
> unused=vfio-pci
> :d8:00.1 'MT27700 Family [ConnectX-4] 1013' if=enp216s0f1 drv=mlx5_core 
> unused=vfio-pci
> :d8:00.2 'MT27700 Family [ConnectX-4 Virtual Function] 1014' 
> if=enp216s0f2 drv=mlx5_core unused=vfio-pci
> :d8:00.3 'MT27700 Family [ConnectX-4 Virtual Function] 1014' 
> if=enp216s0f3 drv=mlx5_core unused=vfio-pci
> :d8:00.4 'MT27700 Family [ConnectX-4 Virtual Function] 1014' 
> if=enp216s0f4 drv=mlx5_core unused=vfio-pci
> :d8:00.5 'MT27700 Family [ConnectX-4 Virtual Function] 1014' 
> if=enp216s0f5 drv=mlx5_core unused=vfio-pci
> 
> The interfaces are up:
> 
> pcgwpod009-c04:~ # ibdev2netdev -v
> :3b:00.0 mlx5_0 (MT4115 - MT1646K01301) CX416A - ConnectX-4 QSFP28 fw 
> 12.28.2006 port 1 (ACTIVE) ==> enp59s0f0 (Up)
> :3b:00.1 mlx5_1 (MT4115 - MT1646K01301) CX416A - ConnectX-4 QSFP28 fw 
> 12.28.2006 port 1 (ACTIVE) ==> enp59s0f1 (Up)
> :d8:00.0 mlx5_2 (MT4115 - MT1646K00538) CX416A - ConnectX-4 QSFP28 fw 
> 12.28.2006 port 1 (ACTIVE) ==> enp216s0f0 (Up)
> :d8:00.1 mlx5_3 (MT4115 - MT1646K00538) CX416A - ConnectX-4 QSFP28 fw 
> 12.28.2006 port 1 (ACTIVE) ==> enp216s0f1 (Up)
> :d8:00.2 mlx5_4 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTIVE) ==> 
> enp216s0f2 (Up)
> :d8:00.3 mlx5_5 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTIVE) ==> 
> enp216s0f3 (Up)
> :d8:00.4 mlx5_6 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTIVE) ==> 
> enp216s0f4 (Up)
> :d8:00.5 mlx5_7 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTIVE) ==> 
> enp216s0f5 (Up)
> pcgwpod009-c04:~ #
> 
> 
> But when I run my application the Mellanox adapters are probed and I obtain 
> the following error:
> 
> {"proc_id":"6"},"message":"[pio] EAL: Probe PCI driver: mlx5_pci (15b3:1014) 
> device: :d8:00.4 (socket 1)"}
> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.826+00:00","severity":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id":"6"},"message":"[pio]
>  net_mlx5: unable to recognize 

release schedule change proposal

[Thomas Monjalon]

For the last 5 years, DPDK was doing 4 releases per year,
in February, May, August and November (the LTS one):
.02   .05   .08   .11 (LTS)

This schedule has multiple issues:
- clash with China's Spring Festival
- too many rushes, impacting maintainers & testers
- not much buffer, impacting proposal period

I propose to switch to a new schedule with 3 releases per year:
.03  .07  .11 (LTS)

New LTS branch would start at the same time of the year as before.
There would be one less intermediate release during spring/summer:
.05 and .08 intermediate releases would become a single .07.
I think it has almost no impact for the users.
This change could be done starting next year.

In details, this is how we could extend some milestones:

ideal schedule so far (in 13 weeks):
proposal deadline: 4
rc1 - API freeze: 5
rc2 - PMD features freeze: 2
rc3 - app features freeze: 1
rc4 - last chance to fix: 1
release: 0

proposed schedule (in 17 weeks):
proposal deadline: 4
rc1 - API freeze: 7
rc2 - PMD features freeze: 3
rc3 - app features freeze: 1
rc4 - more fixes: 1
rc5 - last chance buffer: 1
release: 0

Opinions?

Re: Enable RX interrupts

[Thomas Monjalon]

01/11/2021 09:56, Jack Humphries:
> Hi Dmitry,
> 
> Thanks for the quick response! Yes, I was calling rte_eth_dev_rx_intr_enable 
> — I just tried re-arming interrupts before each call to epoll and my initial 
> experiment shows that fixes the issue.
> 
> You are right that l3fwd-power does the re-arming as well. If possible, it 
> would be helpful to have a comment in that example code about the re-arming 
> since the impression I got from quickly looking at the l3fwd-power code is 
> that interrupts are turned off because the app wants to poll for a threshold 
> of time before “giving up” and turning interrupts back on again. I’m happy to 
> open a pull request to add this comment, too, if you prefer.

The best is to submit a patch on the mailing list, Cc'in the relevant 
maintainers.
Thanks

Re: [dpdk-dev] Doubt regarding DPDK hash Library implementation

[Thomas Monjalon]

04/11/2021 17:39, Medvedkin, Vladimir:
> >> 01/11/2021 11:55, Syam Prasad N Pearson:
> >>> /** Number of items per bucket. */
> >>> *#define RTE_HASH_BUCKET_ENTRIES 8*
> >>>
> >>> defined inside:
> >>> dpdk-20.11.3/dpdk-stable-20.11.3/lib/librte_hash /rte_cuckoo_hash.h
> >>>
> >>> Why does the library take this value as *8*, is there any particular
> >>> reason for this? what if it is 16,32... etc.
> 
> Yes, RTE_HASH_BUCKET_ENTRIES can be any power of 2.
> The reason for choosing 8 is a tradeoff between performance and memory. 
> When it is equal to 8, the sizeof(struct rte_hash_bucket) equal to 
> RTE_CACHE_LINE_SIZE, thus, there are no gaps in memory between the hash 
> buckets due to their alignment.

That's a good comment to add in the code.

Re: [dpdk-dev] Doubt regarding DPDK hash Library implementation

[Thomas Monjalon]

+Cc hash lib maintainers

01/11/2021 11:55, Syam Prasad N Pearson:
> Dear Sir/Madam,
> I am a developer trying to get familiar with the DPDK hash library. I tried
> to make and use a hash table successfully.
> During the development I came across a variable
> 
> /** Number of items per bucket. */
> *#define RTE_HASH_BUCKET_ENTRIES 8*
> 
> defined inside:
> dpdk-20.11.3/dpdk-stable-20.11.3/lib/librte_hash /rte_cuckoo_hash.h
> 
> Why does the library take this value as *8*, is there any particular reason
> for this? what if it is 16,32... etc.
> 
> I am using DPDK 20.11.3 LTS.
> 
> Please help.

Re: [ovs-dev] [PATCH] RFC: netdev-dpdk: Add Windows support.

[Thomas Monjalon]

06/10/2021 19:00, William Tu:
> The patch adds OVS-DPDK supports on Windows.
[...]
> Implementation on Windows
> -
> It's harder than I thought due to my Linux only background.
> Sergey and I first need to add meson build support to OVS, in order
> to make compiling and linking to Windows DPDK library easier[2].
> In this patch, we use clang compiler on Windows to compile DPDK and
> OVS, and OVS links to DPDK static library (Dynamic lib still not
> supported yet in DPDK Windows).
> 
> Windows DPDK library hasn't finished all its library/driver porting
> to Windows. So a couple of DPDK library code used by OVS doesn't
> compile. For examples:
>   1) ovs-numa.c: used to detect lcores used by DPDK.
>   2) open_memstream, fopencookies: used to redirect DPDK logs.
>   3) vhostuser doesn't support: used to connect VM/container.
>   4) dpdk-socket-mem not support: configuration
> I simply remove them in this patch.
> In the future, I will probably refactor or #ifdef.

Yes please, removing code is clearly not the right direction :)

[...]
> Future Work
> ---
> In priority ordering
> a. Add CI or Dockerfile for building the code
> b. Performance numbers, compared with OVS kernel datapath
> c. Try rte_flow HW offload on Windows
> d. Resolve all the compiler warnings
> e. Refactor the vhostuser code
> 
> Some thoughts:
> f. With clang support on Windows, can we remove MSVC?
>so OVS will have less compiler-specific code, ex:
>include\openvswitch\compiler.h
> g. DPDK starts to implement pthread in Windows, can
>OVS uses pthread library from DPDK, instead of pthreads4w?

The thread API is not ready yet in DPDK.
What would be the benefit for OVS to use the same pthread layer as DPDK?
OVS needs to be able to compile without DPDK.

> h. Mingw/msys is pretty slow, by switching to meson+clang,
>what's the improvement of build process?
> f. How does OVS-DPDK connect to VM/Container in Windows?
>There is no vhostuser, we should look at netvsc/vmbus.
> 
> I got lots of help from many people in DPDK community, thanks!


___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: Troubles with building of DPDK.RPM and meson

[Thomas Monjalon]

12/10/2021 08:45, Ruslan R. Laishev:
> Hello All!
> 
>   I have a small study task to make DPDK.RPM with the latest DPDK from 
> the git.

You should look at what is done in Fedora for the latest DPDK.

>   I use DPDK.SPEC from the latest .SRC.RPM kit as a template.

Not sure what is your reference.

Re: I need DPDK MLX5 Probe error support

[Thomas Monjalon]

I don't even know which Linux distribution you are using.
Please send the Dockerfile.
If it compiles in Docker, it should run.


06/10/2021 14:27, Jaeeun Ham:
> Hi Thomas,
> 
> The cause is that I fail to load mlx5 driver using pci address on the docker 
> container.
> So, I tried to add rdma-core library to solve dependency issue you mentioned 
> as below.
> Docker image is built with these Dockerfiles.
> This docker image is built with DPDK20.11.
> How should I add rdma-core library?
> 
> I don't find any rdma related so files in the docker container.
> b273016e5be8:/usr/local/lib # ls *mlx*
> librte_common_mlx5.so  librte_common_mlx5.so.21  librte_common_mlx5.so.21.0  
> librte_net_mlx5.so  librte_net_mlx5.so.21  librte_net_mlx5.so.21.0
> b273016e5be8:/usr/local/lib # ls *rdma*
> ls: cannot access '*rdma*': No such file or directory
> 
> dpdk-20.11/doc/guides/rel_notes/release_20_11.rst
> 911:  * rdma-core:
> 913:* rdma-core-31.0-1 and above
> 
> 
> 
> < error log >
> f1d23550a947:/ # cat /tmp/logs/epp.log
> MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1 
> MIDHAUL_IP_ADDR:10.255.20.125, BACKHAUL_IP_ADDR:10.255.20.124
> mlx5_pci: unable to recognize master/representors on the multiple IB devices
> common_mlx5: Failed to load driver = mlx5_pci.
> 
> EAL: Requested device :12:01.0 cannot be used
> mlx5_pci: unable to recognize master/representors on the multiple IB devices
> common_mlx5: Failed to load driver = mlx5_pci.
> 
> EAL: Requested device :12:01.1 cannot be used
> EAL: Bus (pci) probe failed.
> FATAL: epp_init.c::copy_mac_addr:130: Call to 
> rte_eth_dev_get_port_by_name(src_dpdk_dev_name, _id) failed: -19 
> (Unknown error -19), rte_errno=0 (not set)
> 
> Caught signal 6
> Obtained 7 stack frames, tid=1377.
> tid=1377, /usr/local/bin/ericsson-packet-processor() [0x40a3c4] tid=1377, 
> /lib64/libpthread.so.0(+0x13f80) [0x7f56c4786f80] tid=1377, 
> /lib64/libc.so.6(gsignal+0x10b) [0x7f56c229018b] tid=1377, 
> /lib64/libc.so.6(abort+0x175) [0x7f56c2291585] tid=1377, 
> /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] tid=1377, 
> /lib64/libc.so.6(__libc_start_main+0xed) [0x7f56c227b34d] tid=1377, 
> /usr/local/bin/ericsson-packet-processor(_start+0x2a) [0x4090ea]
> 
> 
> 
> BR/Jaeeun
> 
> -Original Message-
> From: Thomas Monjalon 
> Sent: Wednesday, October 6, 2021 7:59 PM
> To: Jaeeun Ham 
> Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com
> Subject: Re: I need DPDK MLX5 Probe error support
> 
> Installing dependencies is not an issue.
> I don't understand which support you need.
> 
> 
> 06/10/2021 11:57, Jaeeun Ham:
> > Hi Thomas,
> >
> > Could you take a look at the attached file?
> > My engineer managed to compile DPDK 20.11 to support MLX5. Please find the 
> > output from dpdk-testpmd command in attached file. As you can see testpmd 
> > was able to probe mlx5_pci drivers and get MAC addresses.
> > The key issue in his case for enabling MLX5 support was to export rdma-core 
> > lib path to shared libs for meson/ninja commands as new build system 
> > automatically enables MLX5 support if needed dependencies are available.
> >
> > BR/Jaeeun
> >
> > -Original Message-
> > From: Thomas Monjalon mailto:tho...@monjalon.net>>
> > Sent: Sunday, October 3, 2021 4:51 PM
> > To: Jaeeun Ham mailto:jaeeun@ericsson.com>>
> > Cc: users@dpdk.org<mailto:users@dpdk.org>; 
> > alia...@nvidia.com<mailto:alia...@nvidia.com>; 
> > rasl...@nvidia.com<mailto:rasl...@nvidia.com>;
> > as...@nvidia.com<mailto:as...@nvidia.com>
> > Subject: Re: I need DPDK MLX5 Probe error support
> >
> > Hi,
> >
> > I think you need to read the documentation.
> > For DPDK install on Linux:
> > https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d8a
> > 30ca42b-d871f122b4a0a61a=1=88eca0f4-aa71-4ba8-a332-179f08406da3=
> > https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fbuild_dpdk.html%23co
> > mpiling-and-installing-dpdk-system-wide
> > For mlx5 specific dependencies, install rdma-core package:
> > https://protect2.fireeye.com/v1/url?k=9bce4984-c45570e5-9bce091f-86d8a
> > 30ca42b-25bd3d467b5f290d=1=88eca0f4-aa71-4ba8-a332-179f08406da3=
> > https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23linux-prerequ
> > isites
> >
> >
> > 02/10/2021 12:57, Jaeeun Ham:
> > > Hi,
> > >
> > > Could you teach me how to install dpdk-testpmd?
> > > I have to run the application on the host server, not a development 
> > > se

Re: I need DPDK MLX5 Probe error support

[Thomas Monjalon]

Installing dependencies is not an issue.
I don't understand which support you need.


06/10/2021 11:57, Jaeeun Ham:
> Hi Thomas,
> 
> Could you take a look at the attached file?
> My engineer managed to compile DPDK 20.11 to support MLX5. Please find the 
> output from dpdk-testpmd command in attached file. As you can see testpmd was 
> able to probe mlx5_pci drivers and get MAC addresses.
> The key issue in his case for enabling MLX5 support was to export rdma-core 
> lib path to shared libs for meson/ninja commands as new build system 
> automatically enables MLX5 support if needed dependencies are available.
> 
> BR/Jaeeun
> 
> -----Original Message-
> From: Thomas Monjalon  
> Sent: Sunday, October 3, 2021 4:51 PM
> To: Jaeeun Ham 
> Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com
> Subject: Re: I need DPDK MLX5 Probe error support
> 
> Hi,
> 
> I think you need to read the documentation.
> For DPDK install on Linux:
> https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d8a30ca42b-d871f122b4a0a61a=1=88eca0f4-aa71-4ba8-a332-179f08406da3=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fbuild_dpdk.html%23compiling-and-installing-dpdk-system-wide
> For mlx5 specific dependencies, install rdma-core package:
> https://protect2.fireeye.com/v1/url?k=9bce4984-c45570e5-9bce091f-86d8a30ca42b-25bd3d467b5f290d=1=88eca0f4-aa71-4ba8-a332-179f08406da3=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23linux-prerequisites
> 
> 
> 02/10/2021 12:57, Jaeeun Ham:
> > Hi,
> > 
> > Could you teach me how to install dpdk-testpmd?
> > I have to run the application on the host server, not a development server.
> > So, I don't know how to get dpdk-testpmd.
> > 
> > By the way, testpmd run result is as below.
> > root@seroics05590:~/ejaeham# testpmd
> > EAL: Detected 64 lcore(s)
> > EAL: libmlx4.so.1: cannot open shared object file: No such file or 
> > directory
> > EAL: FATAL: Cannot init plugins
> > 
> > EAL: Cannot init plugins
> > 
> > PANIC in main():
> > Cannot init EAL
> > 5: [testpmd(_start+0x2a) [0x55d301d98e1a]]
> > 4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) 
> > [0x7f5e044a4bf7]]
> > 3: [testpmd(main+0x907) [0x55d301d98d07]]
> > 2: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(__rte_panic+0xbd) 
> > [0x7f5e04ca3cfd]]
> > 1: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(rte_dump_stack+0x2e) 
> > [0x7f5e04cac19e]] Aborted
> > 
> > 
> > I added option below when the process is starting in the docker.
> >  dv_flow_en=0 \
> >  --log-level=pmd,8 \
> > < MLX5 log >
> > 415a695ba348:/tmp/logs # cat epp.log
> > MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1 
> > MIDHAUL_IP_ADDR:10.255.21.177, BACKHAUL_IP_ADDR:10.255.21.178
> > mlx5_pci: unable to recognize master/representors on the multiple IB 
> > devices
> > common_mlx5: Failed to load driver = mlx5_pci.
> > 
> > EAL: Requested device :12:01.0 cannot be used
> > mlx5_pci: unable to recognize master/representors on the multiple IB 
> > devices
> > common_mlx5: Failed to load driver = mlx5_pci.
> > 
> > EAL: Requested device :12:01.1 cannot be used
> > EAL: Bus (pci) probe failed.
> > EAL: Trying to obtain current memory policy.
> > EAL: Setting policy MPOL_PREFERRED for socket 1 Caught signal 15
> > EAL: Restoring previous memory policy: 0
> > EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)'
> > EAL: request: mp_malloc_sync
> > EAL: Heap on socket 1 was expanded by 5120MB
> > FATAL: epp_init.c::copy_mac_addr:130: Call to 
> > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, _id) failed: -19 
> > (Unknown error -19), rte_errno=0 (not set)
> > 
> > Caught signal 6
> > Obtained 7 stack frames, tid=713.
> > tid=713, /usr/local/bin/ericsson-packet-processor() [0x40a4a4] 
> > tid=713, /lib64/libpthread.so.0(+0x13f80) [0x7f7e1eae8f80] tid=713, 
> > /lib64/libc.so.6(gsignal+0x10b) [0x7f7e1c5f818b] tid=713, 
> > /lib64/libc.so.6(abort+0x175) [0x7f7e1c5f9585] tid=713, 
> > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] 
> > tid=713, /lib64/libc.so.6(__libc_start_main+0xed) [0x7f7e1c5e334d] 
> > tid=713, /usr/local/bin/ericsson-packet-processor(_start+0x2a) 
> > [0x4091ca]
> > 
> > < i40e log >
> > cat epp.log
> > MIDHAUL_PCI_ADDR::3b:0d.5, BACKHAUL_PCI_ADDR::3b:0d.4 
> > MIDHAUL_IP_ADDR:10.51.21.112, BACKHAUL_IP_ADDR:10.51.21.113
> > EAL: Trying to obtain current memory policy.
> > EAL: Setting policy MPOL_PREFERRED for socket 1
&

Re: I need DPDK MLX5 Probe error support

[Thomas Monjalon]

05/10/2021 03:17, Jaeeun Ham:
> Hi Thomas,
> 
> I attached the testpmd result which is gathered on the host sever.
> Could you please take a look at the mlx5_core PCI issue?

I see no real issue in the log.
For doing more tests, I recommend using the latest DPDK version.


> Thank you in advance.
> 
> BR/Jaeeun
> 
> -Original Message-----
> From: Thomas Monjalon  
> Sent: Sunday, October 3, 2021 4:51 PM
> To: Jaeeun Ham 
> Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com
> Subject: Re: I need DPDK MLX5 Probe error support
> 
> Hi,
> 
> I think you need to read the documentation.
> For DPDK install on Linux:
> https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d8a30ca42b-d871f122b4a0a61a=1=88eca0f4-aa71-4ba8-a332-179f08406da3=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fbuild_dpdk.html%23compiling-and-installing-dpdk-system-wide
> For mlx5 specific dependencies, install rdma-core package:
> https://protect2.fireeye.com/v1/url?k=9bce4984-c45570e5-9bce091f-86d8a30ca42b-25bd3d467b5f290d=1=88eca0f4-aa71-4ba8-a332-179f08406da3=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23linux-prerequisites
> 
> 
> 02/10/2021 12:57, Jaeeun Ham:
> > Hi,
> > 
> > Could you teach me how to install dpdk-testpmd?
> > I have to run the application on the host server, not a development server.
> > So, I don't know how to get dpdk-testpmd.
> > 
> > By the way, testpmd run result is as below.
> > root@seroics05590:~/ejaeham# testpmd
> > EAL: Detected 64 lcore(s)
> > EAL: libmlx4.so.1: cannot open shared object file: No such file or 
> > directory
> > EAL: FATAL: Cannot init plugins
> > 
> > EAL: Cannot init plugins
> > 
> > PANIC in main():
> > Cannot init EAL
> > 5: [testpmd(_start+0x2a) [0x55d301d98e1a]]
> > 4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) 
> > [0x7f5e044a4bf7]]
> > 3: [testpmd(main+0x907) [0x55d301d98d07]]
> > 2: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(__rte_panic+0xbd) 
> > [0x7f5e04ca3cfd]]
> > 1: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(rte_dump_stack+0x2e) 
> > [0x7f5e04cac19e]] Aborted
> > 
> > 
> > I added option below when the process is starting in the docker.
> >  dv_flow_en=0 \
> >  --log-level=pmd,8 \
> > < MLX5 log >
> > 415a695ba348:/tmp/logs # cat epp.log
> > MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1 
> > MIDHAUL_IP_ADDR:10.255.21.177, BACKHAUL_IP_ADDR:10.255.21.178
> > mlx5_pci: unable to recognize master/representors on the multiple IB 
> > devices
> > common_mlx5: Failed to load driver = mlx5_pci.
> > 
> > EAL: Requested device :12:01.0 cannot be used
> > mlx5_pci: unable to recognize master/representors on the multiple IB 
> > devices
> > common_mlx5: Failed to load driver = mlx5_pci.
> > 
> > EAL: Requested device :12:01.1 cannot be used
> > EAL: Bus (pci) probe failed.
> > EAL: Trying to obtain current memory policy.
> > EAL: Setting policy MPOL_PREFERRED for socket 1 Caught signal 15
> > EAL: Restoring previous memory policy: 0
> > EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)'
> > EAL: request: mp_malloc_sync
> > EAL: Heap on socket 1 was expanded by 5120MB
> > FATAL: epp_init.c::copy_mac_addr:130: Call to 
> > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, _id) failed: -19 
> > (Unknown error -19), rte_errno=0 (not set)
> > 
> > Caught signal 6
> > Obtained 7 stack frames, tid=713.
> > tid=713, /usr/local/bin/ericsson-packet-processor() [0x40a4a4] 
> > tid=713, /lib64/libpthread.so.0(+0x13f80) [0x7f7e1eae8f80] tid=713, 
> > /lib64/libc.so.6(gsignal+0x10b) [0x7f7e1c5f818b] tid=713, 
> > /lib64/libc.so.6(abort+0x175) [0x7f7e1c5f9585] tid=713, 
> > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] 
> > tid=713, /lib64/libc.so.6(__libc_start_main+0xed) [0x7f7e1c5e334d] 
> > tid=713, /usr/local/bin/ericsson-packet-processor(_start+0x2a) 
> > [0x4091ca]
> > 
> > < i40e log >
> > cat epp.log
> > MIDHAUL_PCI_ADDR::3b:0d.5, BACKHAUL_PCI_ADDR::3b:0d.4 
> > MIDHAUL_IP_ADDR:10.51.21.112, BACKHAUL_IP_ADDR:10.51.21.113
> > EAL: Trying to obtain current memory policy.
> > EAL: Setting policy MPOL_PREFERRED for socket 1
> > EAL: Restoring previous memory policy: 0
> > EAL: Calling mem event callback 'vfio_mem_event_clb:(nil)'
> > EAL: request: mp_malloc_sync
> > EAL: Heap on socket 1 was expanded by 5120MB
> > i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> > i40evf_handle_aq_msg(): adminq response is 

Re: I need DPDK MLX5 Probe error support

[Thomas Monjalon]

03/10/2021 10:10, Jaeeun Ham:
> Hi Thomas,
> 
> Thank you so much for your sincere support.
> I will follow your suggestion and do my best to solve this issue.
> 
> By the way, is it okay to use mlx5_core driver by different applications 
> which have different DPDK versions?
> :12:01.0 (DPDK 20.11 - mlx5_pci: unable to recognize master/representors 
> on the multiple IB)
> :12:01.1 (DPDK 20.11 - mlx5_pci: unable to recognize master/representors 
> on the multiple IB)
> :12:01.2 (DPDK 18.11 - currently used)

I think it should be OK but it is not well tested.

Re: I need DPDK MLX5 Probe error support

[Thomas Monjalon]

 is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> Caught signal 10
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> 
> 
> process start option which is triggered by shell script is as below.
> 
> < start-epp.sh >
> exec /usr/local/bin/ericsson-packet-processor \
>   $(get_dpdk_core_list_parameter) \
>   $(get_dpdk_mem_parameter) \
>   $(get_dpdk_hugepage_parameters) \
>  -d /usr/local/lib/librte_mempool_ring.so \
>  -d /usr/local/lib/librte_mempool_stack.so \
>  -d /usr/local/lib/librte_net_pcap.so \
>  -d /usr/local/lib/librte_net_i40e.so \
>  -d /usr/local/lib/librte_net_mlx5.so \
>  -d /usr/local/lib/librte_event_dsw.so \
>  $DPDK_PCI_OPTIONS \
>  --vdev=event_dsw0 \
>  --vdev=eth_pcap0,iface=midhaul_edk \
>  --vdev=eth_pcap1,iface=backhaul_edk \
>  --file-prefix=container \
>  --log-level lib.eal:debug \
>  dv_flow_en=0 \
>  --log-level=pmd,8 \
>  -- \
>   $(get_epp_mempool_parameter) \
>  
> "--neighbor-discovery-interface=midhaul_ker,${MIDHAUL_IP_ADDR},mac_addr_dev=${MIDHAUL_MAC_ADDR_DEV},vr_id=0"
>  \
>  
> "--neighbor-discovery-interface=backhaul_ker,${BACKHAUL_IP_ADDR},mac_addr_dev=${BACKHAUL_MAC_ADDR_DEV},vr_id=1"
> 
> BR/Jaeeun
> 
> -Original Message-
> From: Thomas Monjalon  
> Sent: Wednesday, September 29, 2021 8:16 PM
> To: Jaeeun Ham 
> Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com
> Subject: Re: I need DPDK MLX5 Probe error support
> 
> 27/09/2021 02:18, Jaeeun Ham:
> > Hi,
> > 
> > I hope you are well.
> > My name is Jaeeun Ham and I have been working for the Ericsson.
> > 
> > I am suffering from enabling MLX5 NIC, so could you take a look at how to 
> > run it?
> > There are two pci address for the SRIOV(vfio) mlx5 nic support but it 
> > doesn't run correctly. (12:01.0, 12:01.1)
> > 
> > I started one process which is running inside the docker process that is on 
> > the MLX5 NIC support host server.
> > The process started to run with following option.
> > -d /usr/local/lib/librte_net_mlx5.so And the docker process has 
> > mlx5 libraries as below.
> 
> Did you try on the host outside of any container?
> 
> Please could you try following commands (variables to be replaced)?
> 
> dpdk-hugepages.py --reserve 1G
> ip link set $netdev netns $container
> docker run --cap-add SYS_NICE --cap-add IPC_LOCK --cap-add NET_ADMIN \
>--device /dev/infiniband/ $image
> echo show port summary all | dpdk-testpmd --in-memory -- -i
> 
> 
> 
> > 706a37a35d29:/usr/local/lib # ls -1 | grep mlx librte_common_mlx5.so
> > librte_common_mlx5.so.21
> > librte_common_mlx5.so.21.0
> > librte_net_mlx5.so
> > librte_net_mlx5.so.21
> > librte_net_mlx5.so.21.0
> > 
> > But I failed to run the process with following error. 
> > (MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1)
> > 
> > ---
> > 
> > mlx5_pci: unable to recognize master/representors on the multiple IB 
> > devices
> > common_mlx5: Failed to load driver = mlx5_pci.
> > EAL: Requested device :12:01.0 cannot be used
> > mlx5_pci: unable to recognize master/representors on the multiple IB 
> > devices
> > common_mlx5: Failed to load driver = mlx5_pci.
> > EAL: Requested device :12:01.1 cannot be used
> > EAL: Bus (pci) probe failed.
> > 
> > ---
> > 
> > For the success case of pci address 12:01.2, it showed following messages.
> > 
> > ---
> > 
> > EAL: Detected 64 lcore(s)
> > EAL: Detected 2 NUMA nodes
> > EAL: Multi-process socket /var/run/dpdk/nah2/mp_socket
> > EAL: Probing VFIO support...
> > EAL: VFIO support initialized
> > EAL: PCI device :12:01.2 on NUMA socket 0
> > EAL:   probe driver: 15b3:1016 net_mlx5
> > net_mlx5: MPLS over GRE/UDP tunnel offloading disabled due to old 
> > OFED/rdma-core version or firmware configuration
> > net_mlx5: port 0 the requested maximum Rx packet size (2056) is larger 
> > than a single mbuf (2048) and scattered mode has not been requested
> > USER1: rte_ip_frag_table_create: allocated of 6291584 bytes at socket 
> > 0
> > 
> > ---
> > 
> > BR/Jaeeun
> 

Re: [dpdk-users] Memory allocation limits

[Thomas Monjalon]

29/09/2021 14:43, Burakov, Anatoly:
> From: Thomas Monjalon 
> > 29/09/2021 12:14, Burakov, Anatoly:
> > > From: Thomas Monjalon 
> > > > 26/09/2021 17:52, Mohammad Masumi:
> > > > > I have HP server with 768GB memory 384GB in each Numa but I can't
> > > > > allocate more than 64GB by rte_malloc by changing some parameters
> > > > > in rte_config.h it increased to 128GB How to increase heap size?
> > >
> > > This is intentional. In order to increase the amount of contiguous 
> > > allocation
> > possible to perform in DPDK, you need to adjust the following values in
> > rte_config.h:
> > >
> > > #define RTE_MAX_MEMSEG_PER_LIST 8192
> > > #define RTE_MAX_MEM_MB_PER_LIST 32768
> > > #define RTE_MAX_MEMSEG_PER_TYPE 32768
> > > #define RTE_MAX_MEM_MB_PER_TYPE 65536
> > >
> > > I do not recommend arbitrarily changing them as this is untested, but
> > increasing them proportionally (e.g. multiply all of them by 2 or 4) should 
> > not
> > break anything.
> > 
> > It looks to be something to add in docs, right?
> 
> [[AB]] 
> Yes, which is why I already have 
> 
> http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation
> 
> There's a section on "Maximum amount of memory" there.

It says "Normally, these options do not need to be changed."
Would it be meaningful to add how we can increase (by multiplying all of them)?

Re: I need DPDK MLX5 Probe error support

[Thomas Monjalon]

27/09/2021 02:18, Jaeeun Ham:
> Hi,
> 
> I hope you are well.
> My name is Jaeeun Ham and I have been working for the Ericsson.
> 
> I am suffering from enabling MLX5 NIC, so could you take a look at how to run 
> it?
> There are two pci address for the SRIOV(vfio) mlx5 nic support but it doesn't 
> run correctly. (12:01.0, 12:01.1)
> 
> I started one process which is running inside the docker process that is on 
> the MLX5 NIC support host server.
> The process started to run with following option.
> -d /usr/local/lib/librte_net_mlx5.so
> And the docker process has mlx5 libraries as below.

Did you try on the host outside of any container?

Please could you try following commands (variables to be replaced)?

dpdk-hugepages.py --reserve 1G
ip link set $netdev netns $container
docker run --cap-add SYS_NICE --cap-add IPC_LOCK --cap-add NET_ADMIN \
   --device /dev/infiniband/ $image
echo show port summary all | dpdk-testpmd --in-memory -- -i



> 706a37a35d29:/usr/local/lib # ls -1 | grep mlx
> librte_common_mlx5.so
> librte_common_mlx5.so.21
> librte_common_mlx5.so.21.0
> librte_net_mlx5.so
> librte_net_mlx5.so.21
> librte_net_mlx5.so.21.0
> 
> But I failed to run the process with following error. 
> (MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1)
> 
> ---
> 
> mlx5_pci: unable to recognize master/representors on the multiple IB devices
> common_mlx5: Failed to load driver = mlx5_pci.
> EAL: Requested device :12:01.0 cannot be used
> mlx5_pci: unable to recognize master/representors on the multiple IB devices
> common_mlx5: Failed to load driver = mlx5_pci.
> EAL: Requested device :12:01.1 cannot be used
> EAL: Bus (pci) probe failed.
> 
> ---
> 
> For the success case of pci address 12:01.2, it showed following messages.
> 
> ---
> 
> EAL: Detected 64 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/nah2/mp_socket
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: PCI device :12:01.2 on NUMA socket 0
> EAL:   probe driver: 15b3:1016 net_mlx5
> net_mlx5: MPLS over GRE/UDP tunnel offloading disabled due to old 
> OFED/rdma-core version or firmware configuration
> net_mlx5: port 0 the requested maximum Rx packet size (2056) is larger than a 
> single mbuf (2048) and scattered mode has not been requested
> USER1: rte_ip_frag_table_create: allocated of 6291584 bytes at socket 0
> 
> ---
> 
> BR/Jaeeun

Re: [dpdk-users] Memory allocation limits

[Thomas Monjalon]

29/09/2021 12:14, Burakov, Anatoly:
> From: Thomas Monjalon 
> > 26/09/2021 17:52, Mohammad Masumi:
> > > Hi
> > >
> > > I have HP server with 768GB memory 384GB in each Numa but I can't
> > > allocate more than 64GB by rte_malloc by changing some parameters in
> > > rte_config.h it increased to 128GB How to increase heap size?
> > 
> > adding people Cc to help
> > 
> 
> Hi,
> 
> This is intentional. In order to increase the amount of contiguous allocation 
> possible to perform in DPDK, you need to adjust the following values in 
> rte_config.h:
> 
> #define RTE_MAX_MEMSEG_PER_LIST 8192
> #define RTE_MAX_MEM_MB_PER_LIST 32768
> #define RTE_MAX_MEMSEG_PER_TYPE 32768
> #define RTE_MAX_MEM_MB_PER_TYPE 65536
> 
> I do not recommend arbitrarily changing them as this is untested, but 
> increasing them proportionally (e.g. multiply all of them by 2 or 4) should 
> not break anything.

It looks to be something to add in docs, right?

Re: [dpdk-users] can we reserve hugepage and not release

[Thomas Monjalon]

29/09/2021 12:16, Burakov, Anatoly:
> From: Thomas Monjalon 
> > 18/09/2021 04:37, Jiany Wu:
> > > Hello,
> > >
> > > I met a scenario that, need to start and stop the container many times
> > > for the hugepage. But after several times container start and stop,
> > > the hugepage is not able to reserve.
> > > Hugepage size is 2MB, and HW only support 2MB, can't support 1GB.
> > > Is there anyway to make sure the hugepage is still kept continuous?
> > > Thanks indeed.
> > 
> > Interesting question.
> > I think we need to address it in the DPDK documentation.
> > 
> > Anatoly, Stephen, Bruce, any advice please?
> > 
> 
> Hi,
> 
> From description, I don't quite understand what's the issue here. Is the 
> problem about "contiguousness of memory", or is it about inability to reserve 
> more hugepages?

I think the issue is that sometimes some pages are not properly released,
so we cannot reserve them again.
That's something I experienced myself.
Any trick to reset hugepages state?

> How are hugepages assigned to your container?
> Have you tried using --in-memory mode?

Re: Issues Cross compiling DPDK

[Thomas Monjalon]

27/09/2021 17:45, Ginés García Avilés:
> Hi,
> I'm trying to cross-compile DPDK using the information provided
> here (56. Installing DPDK Using the meson build system — Data Plane
> Development Kit 21.11.0-rc0 documentation
> ) but, after
> running "meson build --cross-file config/defconfig_x86_64-native-linux-icc"
> I'm getting the following error:
> 
>   - configparser.MissingSectionHeaderError: File contains no section
> headers.
>   - dpdk/config/defconfig_x86_64-native-linux-icc', line: 6
> 'CONFIG_RTE_MACHINE="native"\n'
> 
> 1st workaround: Compiling directly at the destination machine with "make
> install T=x86_64-native-linux-icc" works.
> 
> Any ideas about how to solve it?

Looks like you are not using a recent DPDK version.

Re: [dpdk-users] what's the cache size of rte_mempool_create()?

[Thomas Monjalon]

+Cc mempool maintainers

08/09/2021 11:18, topperxin:
> HI list
>  A question about the value of cache size of rte_mempool_crate() 
> function, the defination of this function like below:
> 
> 
> struct rte_mempool *
> 
> rte_mempool_create(const char *name, unsigned n, unsigned elt_size,
> 
>unsigned cache_size, unsigned private_data_size,
> 
>rte_mempool_ctor_t *mp_init, void *mp_init_arg,
> 
>rte_mempool_obj_cb_t *obj_init, void *obj_init_arg,
> 
>int socket_id, unsigned flags);
> 
> 
> 
> 
> 
>  My question is : what's cache_size value means ? what's difference 
> between if I set cache_size = 0 and cache_size = 512 ?
>  I get some information from the the dpdk 20.11 it said that, if we set 
> cache size to 0 , it can be useful to avoid losing objects in cache , I can't 
> understand this point, does it mean
>  that if we set the cache size to non zero, it will suffer the risk that 
> some packages will lost ? right ?
> 
> 
>  Thanks for your tips.
> 
> 
>  BR.

Re: [dpdk-users] pktgen-dpdk failed to load LUA script

[Thomas Monjalon]

+Cc the maintainer

10/09/2021 09:34, Kevin Chen (陳奕儒) : 8553:
> Dear DPDK Community,
> 
> I'm trying to execute the LUA script with the following command.
> # usr/local/bin/pktgen -l 0-2 -- -T -P -m 1.0,2.1 -f test/hello-world.lua
> 
> But the LUA script is not executed with this message.
> >>> User State for CLI not set for Lua
> 
> Is there anything I left to set?
> 
> 
> 
> -- Pktgen 21.03.1 (DPDK 21.11.0-rc0)  Powered by DPDK  (pid:17619) 
> 
> 

> 
> 
> ** Version: DPDK 21.11.0-rc0, Command Line Interface without timers
> Pktgen:/>
> Executing 'test/hello-world.lua'
> >>> User State for CLI not set for Lua
> 
> 
> Regards,
> Kevin
> 

Re: [dpdk-users] pktgen not showing any capture.

[Thomas Monjalon]

+Cc dpdk-pktgen maintainer

10/09/2021 12:02, Filip Janiszewski:
> Hi,
> 
> While attempting to capture with pktgen, I see the counter
> rx_steer_missed_packets increasing in ethtool and nothing being captured.
> 
> in pktgen 'page stats' is always empty and 'page xstats' shows something
> is received but i guess nothing is delivered to the queues.
> 
> How should pktgen be configured to steer packets properly?
> 
> Thanks

Re: [dpdk-users] MLX ConnectX-4 Discarding packets

[Thomas Monjalon]

Great, thanks for the update!


12/09/2021 11:32, Filip Janiszewski:
> Alright, nailed it down to a wrong preferred PCIe device in the BIOS
> configuration, it has not been changed after the NIC have been moved to
> another PCIe slot.
> 
> Now the EPYC is going really great, getting 100Gbps rate easily.
> 
> Thank
> 
> Il 9/11/21 4:34 PM, Filip Janiszewski ha scritto:
> > I wanted just to add, while running the same exact testpmd on the other
> > machine I won't get a single miss with the same patter traffic:
> > 
> > .
> > testpmd> stop
> > Telling cores to stop...
> > Waiting for lcores to finish...
> > 
> >   --- Forward Stats for RX Port= 0/Queue= 0 -> TX Port= 0/Queue= 0
> > ---
> >   RX-packets: 61711939   TX-packets: 0  TX-dropped: 0
> > 
> > 
> >   --- Forward Stats for RX Port= 0/Queue= 1 -> TX Port= 0/Queue= 1
> > ---
> >   RX-packets: 62889424   TX-packets: 0  TX-dropped: 0
> > 
> > 
> >   --- Forward Stats for RX Port= 0/Queue= 2 -> TX Port= 0/Queue= 2
> > ---
> >   RX-packets: 61914199   TX-packets: 0  TX-dropped: 0
> > 
> > 
> >   --- Forward Stats for RX Port= 0/Queue= 3 -> TX Port= 0/Queue= 3
> > ---
> >   RX-packets: 63484438   TX-packets: 0  TX-dropped: 0
> > 
> > 
> >   -- Forward statistics for port 0
> > --
> >   RX-packets: 25000  RX-dropped: 0 RX-total: 25000
> >   TX-packets: 0  TX-dropped: 0 TX-total: 0
> > 
> > 
> > 
> >   +++ Accumulated forward statistics for all
> > ports+++
> >   RX-packets: 25000  RX-dropped: 0 RX-total: 25000
> >   TX-packets: 0  TX-dropped: 0 TX-total: 0
> > 
> > 
> > .
> > 
> > In the lab I've the EPYC connected directly to the Xeon using a 100GbE
> > link, both same RHL8.4 and same DPDK 21.02, running:
> > 
> > .
> > ./dpdk-testpmd -l 21-31 -n 8 -w 81:00.1  -- -i --rxq=4 --txq=4
> > --burst=64 --forward-mode=rxonly --rss-ip --total-num-mbufs=4194304
> > --nb-cores=4
> > .
> > 
> > and sending from the other end with pktgen, the EPYC loss tons of
> > packets (see my previous email), the Xeon don't loss anything.
> > 
> > *Confusion!*
> > 
> > Il 9/11/21 4:19 PM, Filip Janiszewski ha scritto:
> >> Thanks,
> >>
> >> I knew that document and we've implemented many of those settings/rules,
> >> but perhaps there's one crucial I've forgot? Wonder which one.
> >>
> >> Anyway, increasing the amount of queues impinge the performance, while
> >> sending 250M packets over a 100GbE link to an Intel 810-cqda2 NIC
> >> mounted on the EPYC Milan server, i see:
> >>
> >> .
> >> 1 queue, 30Gbps, ~45Mpps, 64B frame = imiss: 54,590,111
> >> 2 queue, 30Gbps, ~45Mpps, 64B frame = imiss: 79,394,138
> >> 4 queue, 30Gbps, ~45Mpps, 64B frame = imiss: 87,414,030
> >> .
> >>
> >> With DPDK 21.02 on RHL8.4. I can't observe this situation while
> >> capturing from my Intel server where increasing the queues leads to
> >> better performance (while with the test input set I drop with one queue,
> >> I do not drop anymore with 2 on the Intel server.)
> >>
> >> A customer with a brand new EPYC Milan server in his lab observed as
> >> well this scenario which is a bit of a worry, but again it might be some
> >> config/compilation issue we need do deal with?
> >>
> >> BTW, the same issue can be reproduced with testpmd, using 4 queues and
> >> the same input data set (250M of 64bytes frame at 30Gbps):
> >>
> >> .
> >> testpmd> stop
> >> Telling cores to stop...
> >> Waiting for lcores to finish...
> >>
> >>   --- Forward Stats for RX Port= 0/Queue= 0 -> TX Port= 0/Queue= 0
> >> ---
> >>   RX-packets: 41762999   TX-packets: 0  TX-dropped: 0
> >>
> >>
> >>   --- Forward Stats for RX Port= 0/Queue= 1 -> TX Port= 0/Queue= 1
> >> ---
> >>   RX-packets: 40152306   TX-packets: 0  TX-dropped: 0
> >>
> >>
> >>   --- Forward Stats for RX Port= 0/Queue= 2 -> TX Port= 0/Queue= 2
> >> ---
> >>   RX-packets: 41153402   TX-packets: 0  TX-dropped: 0
> >>
> >>
> >>   --- Forward Stats for RX Port= 0/Queue= 3 -> TX Port= 0/Queue= 3
> >> ---
> >>   RX-packets: 38341370   TX-packets: 0  TX-dropped: 0
> >>
> >>
> >>   -- Forward statistics for port 0
> >> --
> >>   RX-packets: 161410077  RX-dropped: 88589923  RX-total: 25000
> >>   TX-packets: 0  TX-dropped: 0 TX-total: 0
> >>
> >> 
> >> .
> >>
> >> .
> >> testpmd> show port xstats 0
> >> ## NIC extended statistics for port 0
> >> rx_good_packets: 161410081
> >> tx_good_packets: 0
> >> rx_good_bytes: 9684605284
> 

Re: [dpdk-users] SW Turbo Poll Mode Driver

[Thomas Monjalon]

+Cc maintainer

14/09/2021 12:58, Ginés García Avilés:
> Hi all,
> After following the steps listed here (3. SW Turbo Poll Mode Driver — Data
> Plane Development Kit 21.08.0 documentation (dpdk.org)
> ), using the specific
> versions of
> DPDK and FlexRAN, I'm facing an error while trying to run one of the bbdev
> tests:
>   - command:
> > python2 test-bbdev.py
> -e="--vdev=baseband_turbo_sw,socket_id=0,max_nb_queues=8" -c validation -v
> turbo_dec_default.data
>   - Error:
> > "Device 0 (baseband_turbo_sw) does not support specified capabilities"
> 
> which I think is due to an incorrect linkage of DPDK and FlexRAN. I have
> checked all the environmental variables pointing to the different
> components  and everything seems to be correct.
> 
> Any suggestions about how to solve this problem?
> 
> Thanks a lot for your help,
> Ginés.

Re: [dpdk-users] EAL: Failed to attach device on primary process error in dpdk 21.08

[Thomas Monjalon]

15/09/2021 16:08, animesh tripathi:
> Hi Team,
> 
> I am migrating from dpdk 19.11 to dpdk 21.08 for my application and I am
> getting the following error while executing the binary:
> 
> *EAL: Failed to attach device on primary process*
> 
> Can anyone please help me out here in understanding the real reason for
> this error and how to resolve this issue.

We need more info: command line, other logs, etc.
Do you reproduce with testpmd application?

Re: [dpdk-users] [Broadcom BNXT] Link event not working

[Thomas Monjalon]

17/09/2021 14:39, Antoine POLLENUS:
> Hi,
> 
> I'm experiencing some issues with a Broadcom P225P and DPDK in 19.11.3 
> version.
> 
> We register a handler to take care of the link status as specified in the 
> DPDK documentation.
> 
> But with this specific board the event is never triggered. I tested by 
> plugging and unplugging the SFP28 cable.
> 
> The issue is kind of problematic because our code relies on this event.
> 
> I've tested with an intel xxv710-da2 and with that board no problem.
> 
> Is it normal that this link status event is not working on Broadcom ?
> 
> Is it fixed in an higher version ?

Possible, please try testpmd with a recent DPDK.
If it doesn't work on DPDK 21.08, then you can open a bug in bugs.dpdk.org.

Thanks

Re: [dpdk-users] DPDK CPU selection criteria?

[Thomas Monjalon]

17/09/2021 17:23, Jared Brown:
> Hello everybody!
> 
> Is there some canonical resource or at least a recommendation on how to 
> evaluate different CPUs for suitability for use with DPDK?

There are some performance reports with details:
https://core.dpdk.org/perf-reports/

It is difficult to give any generic info
because it depends on CPU architecture, use case, and application.
Please keep in mind there are hundreds of API functions so we cannot
have clues for all combinations.

> My use case is software routers (for example DANOS, 6WIND, TNRS), so I am 
> mainly interested in forwarding performance in terms of Mpps.
> 
> What I am looking for is to develop some kind of heuristics to evaluate CPUs 
> in terms of $/Mpps without having to purchase hundreds of SKUs and running 
> tests on them.
> 
> The official DPDK documentation[0] states thus:
> 
> "7.1. Hardware and Memory Requirements
> 
> For best performance use an Intel Xeon class server system such as Ivy 
> Bridge, Haswell or newer."
> 
> This is somewhat... vague.

This is only for Intel platforms.
DPDK is supported on AMD and Arm CPUs as well.

Are you interested only in Intel CPU?
All your questions below are interesting but are very hardware-specific.
+Cc few Intel engineers for specific questions.

> I suppose one could take [1] as a baseline, which states on page 2 that an 
> Ivy Bridge Xeon E3-1230 V2 is able to forward unidirectional flows at 
> linerate using 10G NICs at all frequencies above 1.6 GHz and bidirectional 
> flows at linerate using 10G NICs at 3.3 GHz.
> 
> This however pales compared with [2] that on page 23 shows that a 3rd 
> Generation Scalable Xeon 8380 manages to very nearly saturate a 100G NIC at 
> all packet sizes.
> 
> As there is almost a magnitude in difference in forwarding performance per 
> core, you can perhaps understand that I am somewhat at a loss when trying to 
> gauge the performance of a particular CPU model.
> 
> Reading [3] one learns that several aspects of the CPU affect the forwarding 
> performance, but very little light is shed on how much each feature on its 
> own contributes. On page 172 one learns that CPU frequency has a linear 
> impact on the performance. This is borne out by [1], but does not take into 
> consideration inter-generational gaps as witnessed by [2].
> 
> This begs the question, what are those inter-generational differences made of?
> 
> - L3 cache latency (p. 54) as an upper limit on Mpps. Do newer generations 
> have decidedly lower cache latencies and is this the defining performance 
> factor?
> 
> - Direct Data I/O (p. 69)? Is DDIO combined with lower L3 cache latency a 
> major force multipler? Or is prefetch sufficient to keep caches hot? This is 
> somewhat confusing, as [3] states on page 62 that DPDK can get one core to 
> handleup to 33 mpps, on average. On one hand this is the performance [1] 
> demonstrated the better part of a decade earlier, but on the other hand [2] 
> demonstrates a magnitude larger performance per core.
> 
> - New instructions? On page 171 [3] notes that the AVX512 instruction can 
> move 64 bytes per cycle which [2] indicates has an almost 30% effect on Mpps 
> on page 22. How important is Transactional Synchronization Extensions (TSX) 
> support (see page 119 of [3]) for forwarding performance?
> 
> - Other factors are mentioned, such as memory frequency, memory size, memory 
> channels and cache sizes, but nothing is said how each of these affect 
> forwarding performance in terms of Mpps. The official documentation [0] only 
> states that: "Ensure that each memory channel has at least one memory DIMM 
> inserted, and that the memory size for each is at least 4GB. Note: this has 
> one of the most direct effects on performance."
> 
> - Turbo boost and hyperthreading? Are these supposed to be enabled or 
> disabled? I am getting conflicting information.  Results listed in [2] show 
> increased Mpps by enabling, but [1] notes that they were disabled due to them 
> introducing measurement artifacts. I recall some documentation recommending 
> disabling, since enabling increases latency and variance.
> 
> - Xeon D, W, E3, E5, E7 and Scalable. Are these different processor siblings 
> observably different from each other from the perspective of DPDK? Atoms 
> certainly are as [3] notes on page 57, because they only perform at 50% 
> compared to an equivalent Xeon core. A reson isn't given, but perhaps it is 
> due to the missing L3 cache?
> 
> - Something entirely else? Am I missing something completely obvious that 
> explains the inter-generational differences between CPUS in terms of 
> forwarding performance?
> 
> 
> So, given all this, how can I perform the mundane task of comparing for 
> example the Xeon W-1250P with the Xeon W-1350P?
> 
> The 1250 is older, but has a larger L2 cache and a higher frequency.
> 
> The 1350 is newer, uses faster memory, has a higher max memory bandwidth, 
> PCIe4.0, more PCI lanes and AVX-512.
> 
> Or 

Re: [dpdk-users] can we reserve hugepage and not release

[Thomas Monjalon]

18/09/2021 04:37, Jiany Wu:
> Hello,
> 
> I met a scenario that, need to start and stop the container many times for
> the hugepage. But after several times container start and stop, the
> hugepage is not able to reserve.
> Hugepage size is 2MB, and HW only support 2MB, can't support 1GB.
> Is there anyway to make sure the hugepage is still kept continuous? Thanks
> indeed.

Interesting question.
I think we need to address it in the DPDK documentation.

Anatoly, Stephen, Bruce, any advice please?

Re: [dpdk-users] TX/RX adapter running on the same core problem

[Thomas Monjalon]

+Cc eventdev maintainers

26/09/2021 07:21, Jaeeun Ham:
> Hi,
> 
> I hope you are well.
> 
> During the traffic test, TX adapter showed starvation due to Rx adapter 
> processing on the same dpdk-core 03 and dropped the 41412 packets after 
> 154655 tx_retry.
> So, I expect I have to assign TX/RX adapter on each dpdk-core to prevent 
> starvation and packet drop.
> 
> Are those tx_retry and tx_dropped packet due to RX adapter packet processing?
> Is it possible to separate TX/RX adapter into different dpdk-core?
> 
> [ TX adapter stats ]
> tx_retry: 154655
> tx_packets: 438771
> tx_dropped: 41412
> 
> 
> < eventdev_dump.txt >
> [ Services ]
> qdispatch_service_id: 0
> txa_service_id  : 1
> rxa_service_id  : 2
> edk_service_id  : 3
> timer_service_id: 4
> timer_adapter_service_id: 5
> 
> Service Cores Summary
> dpdk-core 01: PDCP Fastpath Plugin control function thread
> 
> dpdk-core   qu-dispat   txadapter   rxadapter edk tmr   
> tmr-adapt
>03   0   770574949   770574948   0   0 
>   0
>05   816421027   0   0   816421571   816421571   
> 816421571
>07   896307282   0   0   896307828   896307828 
>   0
>09   899213296   0   0   899213687   899213687 
>   0
>11   889323680   0   0   889323871   889323871 
>   0
>13   897300534   0   0   897300686   897300686 
>   0
>15   891124716   0   0   891124811   891124811 
>   0
>17   896336177   0   0   896336212   896336211 
>   0
>19   895461845   0   0   895461846   895461846 
>   0
> 
> [ Event dev port 0 ~ 10 xstats ]
> - packet flow: f1u -> s1u one way
>   midhaul_ker -> midhaul_edk -> backhaul_edk -> backhaul_ker
>   port 09 Rx adapter -> port 07 -> port 06 -> port 08 -> port 09 Tx adapter
> 
> port 00: dpdk-core 05, worker_core // queue-dispatcher, edk, timer
> port 01: dpdk-core 07, worker_core // queue-dispatcher, edk, timer
> port 02: dpdk-core 09, worker_core // queue-dispatcher, edk, timer
> port 03: dpdk-core 11, worker_core // queue-dispatcher, edk, timer
> port 04: dpdk-core 13, worker_core // queue-dispatcher, edk, timer
> port 05: dpdk-core 15, worker_core // queue-dispatcher, edk, timer
> port 06: dpdk-core 17, worker_core // queue-dispatcher, edk, timer
> port 07: dpdk-core 19, worker_core // queue-dispatcher, edk, timer
> port 08: TX adapter, dpdk-core 03  // packet transmission
> port 09: RX adapter, dpdk-core 03  // packet receiving
> port 10: Event timer adapter   // tmr-adapter
> 
> BR/Jaeeun
> -- next part --
> An embedded and charset-unspecified text was scrubbed...
> Name: eventdev_dump.txt
> URL: 
> 
> 

Re: [dpdk-users] Memory allocation limits

[Thomas Monjalon]

26/09/2021 17:52, Mohammad Masumi:
> Hi
> 
> I have HP server with 768GB memory 384GB in each Numa but I can't allocate
> more than 64GB by rte_malloc by changing some parameters in rte_config.h it
> increased to 128GB
> How to increase heap size?

adding people Cc to help

Re: Using rte_flow to distribute single flow type among multiple Rx queues using DPDK in Mellanox ConnectX-5 Ex

[Thomas Monjalon]

29/09/2021 07:26, Anna A:
> Hi,
> 
> I'm trying to use rte_flow_action_type_rss to distribute packets all of the
> same flow type among multiple Rx queues on a single port. Mellanox
> ConnectX-5 Ex and DPDK version 20.05 is used for this purpose. It doesn't
> seem to work and all the packets are sent only to a single queue.

Adding mlx5 maintainers Cc.

> My queries are :
> 1. What am I missing or doing differently?
> 2. Should I be doing any other configurations in rte_eth_conf or
> rte_eth_rxmode?

Do you see any error log?
For info, you can change log level with --log-level.
Experiment options with '--log-level help' in recent DPDK.

> My rte_flow configurations:
> 
> struct rte_flow_item pattern[MAX_RTE_FLOW_PATTERN] = {};
> struct rte_flow_action action[MAX_RTE_FLOW_ACTIONS] = {};
> struct rte_flow_attr attr;
> struct rte_flow_item_eth eth;
> struct rte_flow *flow = NULL;
> struct rte_flow_error error;
> int ret;
> int no_queues =2;
> uint16_t queues[2];
> struct rte_flow_action_rss rss;
> memset(, 0x22, sizeof(error));
> memset(, 0, sizeof(attr));
> attr.egress = 0;
> attr.ingress = 1;
> 
> memset(, 0, sizeof(pattern));
> memset(, 0, sizeof(action));
> /* setting the eth to pass all packets */
> pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH;
> pattern[0].spec = 
> pattern[1].type = RTE_FLOW_ITEM_TYPE_END;
> 
> rss.types = ETH_RSS_IP;
> rss.level = 0;
> rss.func = RTE_ETH_HASH_FUNCTION_TOEPLITZ;
> rss.key_len =0;
> rss.key = NULL;
> rss.queue_num = no_queues;
> for (int i= 0; i < no_queues; i++){
> queues[i] = i;
> }
> rss.queue = queues;
> action[0].type = RTE_FLOW_ACTION_TYPE_RSS;
> action[0].conf = 
> 
> action[1].type = RTE_FLOW_ACTION_TYPE_END;
> 
> ret = rte_flow_validate(portid, , pattern, action, );
>  if (ret < 0) {
>   printf( "Flow validation failed %s\n", error.message);
> return;
> }
> flow = rte_flow_create(portid, , pattern, action, );
> 
> if (flow == NULL)
> printf(" Cannot create Flow create");
> 
> And Rx queues configuration:
> for (int j = 0; j < no_queues; j++) {
> 
>  int ret = rte_eth_rx_queue_setup(portid, j, nb_rxd,
> rte_eth_dev_socket_id(port_id),
>NULL,mbuf_pool);
>  if (ret < 0) {
>   printf( "rte_eth_rx_queue_setup:err=%d, port=%u", ret, (unsigned)
> portid);
> exit(1);
>}
> }
> 
> Thanks
> Anna

[Akonadi] [Bug 386985] akonadi CalDav resource not synching with certain servers

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=386985

--- Comment #56 from Thomas Monjalon  ---
With recent versions, it works fine for me: syncing with fastmail caldav.

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 386985] akonadi CalDav resource not synching with certain servers

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=386985

--- Comment #56 from Thomas Monjalon  ---
With recent versions, it works fine for me: syncing with fastmail caldav.

-- 
You are receiving this mail because:
You are the assignee for the bug.

Re: [dpdk-users] [DISCUSSION] code snippet documentation

[Thomas Monjalon]

15/07/2021 09:01, Asaf Penso:
> Hello DPDK community,
> 
> I would like to bring up a discussion about a way to have code snippets as an 
> example for proper usage.
> The DPDK tree is filled with great pieces of code that are well documented 
> and maintained in high quality.
> I feel we are a bit behind when we talk about usage examples.
> 
> One way, whenever we implement a new feature, is to extend one of the test-* 
> under the "app" folder.
> This, however, provides means to test but doesn't provide a good usage 
> example.
> 
> Another way is to check the content of the "example" folder and whenever we 
> have a BIG new feature it seems like a good place.
> This, however, doesn't provide a good option when we talk about small 
> features.
> If, for example, we extend rte_flow with an extra action then providing a 
> full-blown example application is somewhat an entry barrier.
> 
> A third option could be to document it in one of the .rst files we have.
> Obviously, this requires high maintenance and no option to assure it still 
> compiles.
> 
> I'd like to propose another approach that will address the main two issues: 
> remove the entry barrier and assure compilation.
> In this approach, inside the "examples" folder we'll create another folder 
> for "snippets".
> Inside "snippets" we'll have several files per category, for example, 
> rte_flow_snippets.c
> Each .c file will include a main function that calls the different use cases 
> we want to give as an example.
> The purpose is not to generate traffic nor read rx/tx packets from the DPDK 
> ports. 
> The purpose is to have a good example that compiles properly.
> 
> Taking the rte_flow_snippets.c as an example its main function would look 
> like this:
> 
> int
> main(int argc, char **argv)
> {
>   rte_flow_snippet_match_5tuple_and_drop();
>   rte_flow_snippet_match_geneve_ope_and_rss();
>   ...
>   Return 0;
> }

I think we need to have a policy or justification about which snippet
is worth to have.
My thought is to avoid creating snippets which have no other value
than showing a function call.
I think there is a value if the context is not simple.

Please could you provide a more complex example?

[dpdk-users] new IRC channel

[Thomas Monjalon]

As agreed in a Technical Board meeting, the preferred IRC channel
for the DPDK project moved from freenode to Libera.Chat:
https://mails.dpdk.org/archives/dev/2021-July/214662.html

The website is updated:
https://core.dpdk.org/contribute/

There is a good guide to start with Libera.Chat:
https://libera.chat/guides/connect

Let's meet on Libera.Chat channel #DPDK
for quick questions, synchronization, or just to say hello!

Re: [dpdk-users] Question regarding DPDK on AWS

[Thomas Monjalon]

30/06/2021 16:40, Antonis Christodoulou:
> Hello all,

Hello,

> my name is Antonis Christodoulou, and I am a new user of DPDK. I am not sure 
> this is the right place to ask a usage question, so please feel free to 
> redirect me to someone more appropriate for such questions as needed.

Your question below looks related to the TCP stack in F-stack, not DPDK itself.
I would recommend to get in touch with the F-Stack project:
https://github.com/F-Stack/f-stack/issues


> I am working on AWS, and I have set up F-stack over DPDK, successfully 
> connecting to an address within my private VPC, using a client socket. On the 
> other side I am running a simple echo server with  ncat -l 2001 -k -c 'xargs 
> -n1 echo' -vvv. However, when I just change the address to some global IP, 
> like the one used by www.example.com, ie.  
> 93.184.216.34 (I used port 80), then I am not getting any socket connection.
> 
> Would you know why this is happening? I have not set up any veth0 interface 
> yet for the DPDK NIC, I am not sure this is needed for connectivity.
> 
> Regards,
> Antonis

Re: [dpdk-users] Unable to setup hugepages

[Thomas Monjalon]

31/05/2021 17:35, Gabriel Danjon:
> Hello,
> 
> After successfully installed the DPDK 20.11 on my Centos 8-Stream 
> (minimal), I am trying to configure the hugepages but encounters a lot 
> of difficulties.

There's some confusing info below.
Let's forget all the details and focus on simple things:
1/ use dpdk-hugepages.py
2/ choose one page size (2M or 1G)
3/ check which node requires memory with lstopo
4/ don't be confused with warnings about unused page size



> I am trying to reserve 4 hugepages of 1GB.
> 
> 
> Here the steps I have done following the documentation 
> (https://doc.dpdk.org/guides-20.11/linux_gsg/sys_reqs.html):
> 
> Additional information about meminfo :
> 
> cat /proc/meminfo
> MemTotal:   32619404 kB
> MemFree:27331024 kB
> MemAvailable:   27415524 kB
> Buffers:4220 kB
> Cached:   328628 kB
> SwapCached:0 kB
> Active:   194828 kB
> Inactive: 210156 kB
> Active(anon):   1744 kB
> Inactive(anon):83384 kB
> Active(file): 193084 kB
> Inactive(file):   126772 kB
> Unevictable:   0 kB
> Mlocked:   0 kB
> SwapTotal:  16474108 kB
> SwapFree:   16474108 kB
> Dirty: 0 kB
> Writeback: 0 kB
> AnonPages: 72136 kB
> Mapped:84016 kB
> Shmem: 12992 kB
> KReclaimable: 211956 kB
> Slab: 372852 kB
> SReclaimable: 211956 kB
> SUnreclaim:   160896 kB
> KernelStack:9120 kB
> PageTables: 6852 kB
> NFS_Unstable:  0 kB
> Bounce:0 kB
> WritebackTmp:  0 kB
> CommitLimit:30686656 kB
> Committed_AS: 270424 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:   0 kB
> VmallocChunk:  0 kB
> Percpu:28416 kB
> HardwareCorrupted: 0 kB
> AnonHugePages: 10240 kB
> ShmemHugePages:0 kB
> ShmemPmdMapped:0 kB
> FileHugePages: 0 kB
> FilePmdMapped: 0 kB
> HugePages_Total:   0
> HugePages_Free:0
> HugePages_Rsvd:0
> HugePages_Surp:0
> Hugepagesize:1048576 kB
> Hugetlb: 4194304 kB
> DirectMap4k:  225272 kB
> DirectMap2M: 4919296 kB
> DirectMap1G:30408704 kB
> 
> 1 Step follow documentation
> 
> bash -c 'echo 2048 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages'
> 
> As we're working on a NUMA machine we do this too. (We even do the 
> previous step because without it, it provides more errors)
> 
> bash -c 'echo 2048 > 
> /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages' && \
> bash -c 'echo 2048 > 
> /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages'
> 
> mkdir /mnt/huge
> mount -t hugetlbfs pagesize=1GB /mnt/huge
> 
> bash -c 'echo nodev /mnt/huge hugetlbfs pagesize=1GB 0 0 >> /etc/fstab'
> 
> So here the result of my meminfo (cat /proc/meminfo | grep Huge) :
> 
> AnonHugePages: 10240 kB
> ShmemHugePages:0 kB
> FileHugePages: 0 kB
> HugePages_Total:   0
> HugePages_Free:0
> HugePages_Rsvd:0
> HugePages_Surp:0
> Hugepagesize:1048576 kB
> Hugetlb: 4194304 kB
> 
> It looks strange that there is no total and free hugepages.
> 
> I tried the dpdk-testpmd using the DPDK documentation : dpdk-testpmd -l 
> 0-3 -n 4 -- -i --nb-cores=2
> 
> EAL: Detected 48 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: 2048 hugepages of size 2097152 reserved, but no mounted hugetlbfs 
> found for that size
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: No available hugepages reported in hugepages-1048576kB
> EAL: FATAL: Cannot get hugepage information.
> EAL: Cannot get hugepage information.
> EAL: Error - exiting with code: 1
>Cause: Cannot init EAL: Permission denied
> 
> 
> So I checked in the /mnt/huge to look if files had been created (ls 
> /mnt/huge/ -la) : Empty folder
> 
> Then I checked if my folder was correctly mounted : mount | grep huge
> pagesize=1GB on /mnt/huge type hugetlbfs 
> (rw,relatime,seclabel,pagesize=1024M)
> 
> Then I tried the helloworld example (make clean && make && 
> ./build/helloworld):
> 
> EAL: Detected 48 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Detected shared linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: 2048 hugepages of size 2097152 reserved, but no mounted hugetlbfs 
> found for that size
> EAL: No free 1048576 kB hugepages reported on node 0
> EAL: No free 1048576 kB hugepages reported on node 1
> EAL: No available 1048576 kB hugepages reported
> EAL: FATAL: Cannot get hugepage information.
> EAL: Cannot get hugepage information.
> PANIC in main():
> Cannot init EAL
> 5: [./build/helloworld() [0x40079e]]
> 4: 

Re: [dpdk-users] Enable to install DPDK on Centos 8-Stream

[Thomas Monjalon]

27/05/2021 18:10, Gabriel Danjon:
> Hello,
> 
> I am having difficulties to compile and install DPDK from sources on the 
> latest Centos 8-Stream.

Did you compile and install DPDK successfully?
Where is it installed?

> 
> After having installed the required drivers and libraries, following the 
> documentation and the DPDK build (meson build && cd build && ninja && 
> ninja install && ldconfig), I tried to compile the helloworld example 
> without success:
> 'Makefile:12: *** "no installation of DPDK found".  Stop.'
> 
> 
> Please find attached to this mail some logs.

The log is way too long to be read.
Please copy only what is relevant.

> Could you provide help please ?

It looks to be basic issue with library installation.
Did you read the doc?
http://doc.dpdk.org/guides/linux_gsg/build_dpdk.html

Especially this note:
"
On some linux distributions, such as Fedora or Redhat, paths in /usr/local are 
not in the default paths for the loader. Therefore, on these distributions, 
/usr/local/lib and /usr/local/lib64 should be added to a file in 
/etc/ld.so.conf.d/ before running ldconfig.
"

Re: [dpdk-users] All links down with Chelsio T6 NICs

[Thomas Monjalon]

+Cc Chelsio maintainer

09/04/2021 19:24, Danushka Menikkumbura:
> Hello,
> 
> When I run testpmd on a system with 2 two-port Chelsio T6 NICs, the link
> status is down for all four ports. I use igb_uio as the kernel driver.
> Below is my testpmd commandline and the startup log.
> 
> sudo ./build/app/dpdk-testpmd -l 0,1,2,5 -b 81:00.0 -- -i
> 
> EAL: Detected 20 lcore(s)
> EAL: Detected 4 NUMA nodes
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: No available 1048576 kB hugepages reported
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: Probe PCI driver: net_cxgbe (1425:6408) device: :05:00.4 (socket 0)
> rte_cxgbe_pmd: Maskless filter support disabled. Continuing
> EAL: Probe PCI driver: net_cxgbe (1425:6408) device: :0b:00.4 (socket 0)
> rte_cxgbe_pmd: Maskless filter support disabled. Continuing
> Interactive-mode selected
> testpmd: create a new mbuf pool : n=171456, size=2176, socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
> testpmd: create a new mbuf pool : n=171456, size=2176, socket=2
> testpmd: preferred mempool ops selected: ring_mp_mc
> Configuring Port 0 (socket 0)
> Port 0: 00:07:43:5D:4E:60
> Configuring Port 1 (socket 0)
> Port 1: 00:07:43:5D:4E:68
> Configuring Port 2 (socket 0)
> Port 2: 00:07:43:5D:51:00
> Configuring Port 3 (socket 0)
> Port 3: 00:07:43:5D:51:08
> Checking link statuses...
> Done
> testpmd>
> 
> Your help is very much appreciated.

Please run the command "show port summary all"

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Thomas Monjalon]

11/03/2021 01:17, Jinkun Geng:
> Do you mean all components must be built inside bazel?
> >> Sort of. We have a project that is built on bazel. Now, we need to use the 
> >> core functions of DPDK to replace the network primitives in our project, 
> >> so that we can improve the performance of our project.

You don't need to compile DPDK inside your project.
Just install the package or compile it externally.

> You are not able to call meson/ninja commands from bazel?
> >> I have found any examples using meson inside bazel. According to my 
> >> understanding, Bazel and meson are two parallel building systems and I 
> >> haven't seen anyone could use them together.
> 
> How do you do with other libraries? Does bazel usually reimplement what is 
> packaged with autotools?
> >> That is the reason why https://github.com/bazelment exists. If some 
> >> libraries are popular and do not support bazel, then these guys will help 
> >> generate some modified version of these libraries, so that developers can 
> >> integrate the library into bazel. It is not completely reimplementation, 
> >> but it indeed needs much extraction to adapt the previous library so that 
> >> it can be used in bazel.
> Unfortunately, these guys stop supporting DPDK atop bazel now.

That's crazy trying to replace the build system of projects.
Even if you find new volunteers, it will not be always up-to-date.

You should look for how to link an existing library from bazel.


> 
From: Thomas Monjalon 
> 11/03/2021 00:42, Jinkun Geng:
> > For any project using bazel, if we want to use DPDK, then we need to 
> > compile DPDK stuff into bazel by ourselves. It is not a trivial thing and 
> > the bazelment (https://github.com/bazelment/dpdk) guys have spent much 
> > effort extracting the core files in DPDK and write the BUILD files for 
> > DPDK. But now it seems they have stopped maintaining that repo since DPDK 
> > 16.04. Even in that version, it has some runtime failure when we use DPDK 
> > in our bazel project.
> 
> Sorry I don't know bazel.
> Do you mean all components must be built inside bazel?
> You are not able to call meson/ninja commands from bazel?
> How do you do with other libraries?
> Does bazel usually reimplement what is packaged with autotools?
> 
> 
> > 
> From: Thomas Monjalon 
> > 09/03/2021 05:11, Jinkun Geng:
> > > Too bad. :<
> >
> > Why is it too bad?
> > How the choice of an internal build system
> > can affect other projects?
> >
> >
> > From: Stephen Hemminger 
> > > On Tue, 9 Mar 2021 01:32:16 +
> > > Jinkun Geng  wrote:
> > >
> > > > Hi, all.
> > > > Since bazel building system is becoming more and more popular, 
> > > > sometimes we need to integrate DPDK library into a bazel project. 
> > > > However, it seems there is no much support for bazel from DPDK 
> > > > community.
> >
> > Why the DPDK community would support building with Bazel?
> > What is the benefit?
> > Bazel projects cannot just link with DPDK using pkg-config?
> >
> >
> > > > The only support at https://github.com/bazelment/dpdk has been 
> > > > outdated. Based on our experience, it can only compile successfully 
> > > > with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed 
> > > > to DPDK 21.02, but the bazel support fails to catch up.
> > > >
> > > > It would be great if the experts in DPDK community can provide some 
> > > > portable BUILD files to facilitate the integration of the newest DPDK 
> > > > into bazel project (just like bazelment). After all, writing the bazel 
> > > > files can be really challenging, especially if we do not have a very 
> > > > deep understanding of the whole DPDK codes.
> > > >
> > > > Jinkun
> > >
> > > DPDK is on meson now. The core team is unlikely to change build systems 
> > > again.
> >
> > DPDK supports library standards for compiling, installing and linking.
> > What else is needed?
> 
> 
> 
> 
> 
> 

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Thomas Monjalon]

11/03/2021 00:42, Jinkun Geng:
> For any project using bazel, if we want to use DPDK, then we need to compile 
> DPDK stuff into bazel by ourselves. It is not a trivial thing and the 
> bazelment (https://github.com/bazelment/dpdk) guys have spent much effort 
> extracting the core files in DPDK and write the BUILD files for DPDK. But now 
> it seems they have stopped maintaining that repo since DPDK 16.04. Even in 
> that version, it has some runtime failure when we use DPDK in our bazel 
> project.

Sorry I don't know bazel.
Do you mean all components must be built inside bazel?
You are not able to call meson/ninja commands from bazel?
How do you do with other libraries?
Does bazel usually reimplement what is packaged with autotools?


> ________
From: Thomas Monjalon 
> 09/03/2021 05:11, Jinkun Geng:
> > Too bad. :<
> 
> Why is it too bad?
> How the choice of an internal build system
> can affect other projects?
> 
> 
> From: Stephen Hemminger 
> > On Tue, 9 Mar 2021 01:32:16 +
> > Jinkun Geng  wrote:
> >
> > > Hi, all.
> > > Since bazel building system is becoming more and more popular, sometimes 
> > > we need to integrate DPDK library into a bazel project. However, it seems 
> > > there is no much support for bazel from DPDK community.
> 
> Why the DPDK community would support building with Bazel?
> What is the benefit?
> Bazel projects cannot just link with DPDK using pkg-config?
> 
> 
> > > The only support at https://github.com/bazelment/dpdk has been outdated. 
> > > Based on our experience, it can only compile successfully with dpdk-16.04 
> > > (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but 
> > > the bazel support fails to catch up.
> > >
> > > It would be great if the experts in DPDK community can provide some 
> > > portable BUILD files to facilitate the integration of the newest DPDK 
> > > into bazel project (just like bazelment). After all, writing the bazel 
> > > files can be really challenging, especially if we do not have a very deep 
> > > understanding of the whole DPDK codes.
> > >
> > > Jinkun
> >
> > DPDK is on meson now. The core team is unlikely to change build systems 
> > again.
> 
> DPDK supports library standards for compiling, installing and linking.
> What else is needed?

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Thomas Monjalon]

09/03/2021 05:11, Jinkun Geng:
> Too bad. :<

Why is it too bad?
How the choice of an internal build system
can affect other projects?


From: Stephen Hemminger 
> On Tue, 9 Mar 2021 01:32:16 +
> Jinkun Geng  wrote:
> 
> > Hi, all.
> > Since bazel building system is becoming more and more popular, sometimes we 
> > need to integrate DPDK library into a bazel project. However, it seems 
> > there is no much support for bazel from DPDK community.

Why the DPDK community would support building with Bazel?
What is the benefit?
Bazel projects cannot just link with DPDK using pkg-config?


> > The only support at https://github.com/bazelment/dpdk has been outdated. 
> > Based on our experience, it can only compile successfully with dpdk-16.04 
> > (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but 
> > the bazel support fails to catch up.
> >
> > It would be great if the experts in DPDK community can provide some 
> > portable BUILD files to facilitate the integration of the newest DPDK into 
> > bazel project (just like bazelment). After all, writing the bazel files can 
> > be really challenging, especially if we do not have a very deep 
> > understanding of the whole DPDK codes.
> >
> > Jinkun
> 
> DPDK is on meson now. The core team is unlikely to change build systems again.

DPDK supports library standards for compiling, installing and linking.
What else is needed?

Re: [dpdk-users] MLX5: Using packet send scheduling / packet pacing

[Thomas Monjalon]

30/01/2021 11:54, Carsten Andrich:
> Hi Slava,
> 
> thank you for the prompt response. I think the requirements for Packet 
> Pacing should be added to Table 34.2(?) of the MLX5 docs [1].

+1 for improving the doc

Re: [dpdk-users] MLX5: Using packet send scheduling / packet pacing

[Thomas Monjalon]

+Cc Slava

29/01/2021 17:30, Carsten Andrich:
> Hello everyone,
> 
> I'm trying to use packet send scheduling [1] with DPDK 20.11 and the 
> MLX5 PMD (NIC: ConnectX-5 MCX516A-CDAT). This patch contains some 
> additional information on this feature also know as packet pacing [2].
> 
> According to MLX5's docs, packet pacing requires the "tx_pp" parameter 
> [3, CTRL+F: "tx_pp"]. However, when firing up testpmd with that 
> parameter, it fails as follows:
> 
> > # dpdk-testpmd -a 81:00.0,tx_pp=500 -- -i
> > ...
> > EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: :81:00.0 (socket 0)
> > mlx5_pci: WQE rate mode is required for packet pacing
> > mlx5_pci: probe of PCI device :81:00.0 aborted after encountering an 
> > error: No such device
> > common_mlx5: Failed to load driver = mlx5_pci.
> >
> > EAL: Requested device :81:00.0 cannot be used
> The error message originates here [4] and is caused by what to me 
> appears to be a value read from the NIC [5]. Unfortunately, that leaves 
> me clueless on how to activate the required "WQE rate mode".  According 
> to the output of ibv_devinfo, my NIC does support packet pacing:
> 
> > # ibv_devinfo -v 81:00.0
> > ...
> > packet_pacing_caps:
> > qp_rate_limit_min:  1kbps
> > qp_rate_limit_max:  1kbps
> > supported_qp:
> > SUPPORT_RAW_PACKET
> I'd be grateful for any information on how to get packet pacing up and 
> running. Am I just missing another required option (which is not given 
> in the docs) or does my NIC lack packet pacing support?
> 
> Thank you very much in advance.
> 
> Best regards,
> Carsten
> 
> [1] 
> https://doc.dpdk.org/api/rte__ethdev_8h.html#a990d8351447a710628cbb24a28d3252d
> [2] https://patches.dpdk.org/patch/73742/
> [3] https://doc.dpdk.org/guides/nics/mlx5.html#run-time-configuration
> [4] 
> http://code.dpdk.org/dpdk/v20.11/source/drivers/net/mlx5/linux/mlx5_os.c#L1278
> [5] 
> http://code.dpdk.org/dpdk/v20.11/source/drivers/common/mlx5/mlx5_devx_cmds.c#L748
> 
> 

Re: [dpdk-users] DPDK: MPLS packet processing

[Thomas Monjalon]

18/01/2021 09:46, Raslan Darawsheh:
> From: raktim bhatt
> 
> > Hi All,
> > 
> > I am trying to build a multi-RX-queue dpdk program, using RSS to split the
> > incoming traffic into RX queues on a single port. Mellanox ConnectX-5 and
> > DPDK Version 19.11 is used for this purpose. It works fine when I use IP
> > over Ethernet packets as input. However when the packet contains IP over
> > MPLS over Ethernet, RSS does not seem to work. As a result, all packets
> > belonging to various flows (with different src & dst IPs, ports over MPLS)
> > are all sent into the same RX queue.
> > 
> > 
> > My queries are
> > 
> > 1. Is there any parameter/techniques in DPDK to distribute MPLS packets to
> > multiple RX queues?
> > 
> I've tried it over my setup with testpmd:
> ./build/app/dpdk-testpmd -n 4 -w :08:00.0 -- --mbcache=512 -i 
> --nb-cores=27 --rxq=4 --txq=4 --rss-ip
> testpmd> set verbose 1
> testpmd> start
> 
> then tried to send two MPLS packets with different src IP:
> packet1 =  Ether()/MPLS()/IP(src='1.1.1.1')
> packet2 =  Ether()/MPLS()/IP(src='1.1.1.2')
> 
> and I see that both packets are being spread over the queues, see the bellow 
> testpmd dump output:
> testpmd> port 0/queue 3: received 1 packets
>   src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 - 
> nb_segs=1 - RSS hash=0x43781943 - RSS queue=0x3 - hw ptype: L2_ETHER 
> L3_IPV4_EXT_UNKNOWN L4_NONFRAG  - sw ptype: L2_ETHER  - l2_len=14 - Receive 
> queue=0x3
>   ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_GOOD 
> PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> port 0/queue 1: received 1 packets
>   src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 - 
> nb_segs=1 - RSS hash=0xb8631e05 - RSS queue=0x1 - hw ptype: L2_ETHER 
> L3_IPV4_EXT_UNKNOWN L4_NONFRAG  - sw ptype: L2_ETHER  - l2_len=14 - Receive 
> queue=0x1
>   ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_GOOD 
> PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> 
> first packet was received on queue 3 and the second one was received over 
> queue 1,
> by the way, this is with both 19.11.0 and v19.11.6 
> 
> > 2. Is there any way to strip off MPLS tags (between Eth and IP) in
> > hardware, something like hw_vlan_strip?
> > 
> For this I'm not sure we have such thing in dpdk maybe Thomas can confirm 
> this here?

Look for "POP_MPLS" in rte_flow.

Re: [dpdk-users] Does vmxnet3 PMD supports LSC=1 ?

[Thomas Monjalon]

+Cc Yong Wang, maintainer of this PMD.

14/01/2021 18:30, madhukar mythri:
> Hi,
> 
> Does vmxnet3 PMD support LSC=1(i.e with interrupt mode) for link changes ?
> 
> When i enable LSC=1 the functionality works fine, but, when pumping traffic
> i'm seeing increasing in CPU load on some cores which is running
> "eal-intr-thread" epoll_wait() function for more CPU-time.
> 
> Actually, interrupt should come only when Link changes, but, we are seeing
> interrupt for each incoming Rx-packet and also a lot of spurious interrupts.
> =
> ~ # cat /proc/interrupts |grep igb
>  58:1254293  0  0  0   PCI-MSI 1572864-edge
>  igb_uio
>  59:1278105  0  0  0   PCI-MSI 5767168-edge
>  igb_uio
> ~ # cat /proc/irq/58/spurious
> count 98035
> unhandled 0
> last_unhandled 0 ms
> ~ #
> ==
> 
> Does anyone tried LSC=1 in vmxnet3 PMD based apps and faced similar issues
> ? If so, please let me know.
> 
> Tried with DPDK-18.11, DPDK-19.11 and DPDK-20.05.
> 
> Thanks,
> Madhukar.

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Thomas Monjalon]

11/12/2020 17:19, Slava Ovsiienko:
> From: Thomas Monjalon 
> > 09/12/2020 17:03, Alessandro Pagani:
> > > Hi all,
> > >
> > > I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5 driver).
> > >
> > > I am specifying the tx_pp parameter to provide the packet send
> > > scheduling on mbuf timestamps, but the testpmd fails with the following
> > error:
> > [...]
> > > EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: :3b:00.0
> 
> This is ConnectX-4LX (DevID is 1015), it does not support scheduling.
> Tx scheduling is supported since ConnectX-6DX. 
> 
> > > (socket 0)
> > > mlx5_pci: No available register for Sampler.
> > > mlx5_pci: WQE index ignore feature is required for packet pacing
> > > mlx5_pci: probe of PCI device :3b:00.0 aborted after encountering
> > > an
> > > error: No such device
> > > common_mlx5: Failed to load driver = mlx5_pci.
> > >
> > > EAL: Requested device :3b:00.0 cannot be used
> > [...]
> > > The error messages suggest that "WQE index ignore feature is required
> > > for packet pacing".
> > >
> > > Anyone knows the reason of this error and how to solve it?
> > 
> > I think it means your device does not support this feature.
> > But I realize it is not documented here:
> 
> Yes, indeed. I'll provide the patch, thank you for noticing that.

I think we should also improve the error message to
something like "not supported on this device".

Re: [dpdk-users] Questions about public IP included in DPDK driver code

[Thomas Monjalon]

+Cc ark maintainers

07/12/2020 15:17, HuangLiming:
> I found that the driver contains public IP(169.254.10.240), which leads to my 
> binary
> with public IP too, which is not what I want.
> What is the function of this IP, and could the community delete it for 
> security reasons?
> 
> drivers/net/ark/ark_pktgen.c: {{"dst_ip"}, OTSTRING, .v.STR = 
> "169.254.10.240"},
> drivers/net/ark/ark_pktchkr.c:{{"dst_ip"}, OTSTRING, .v.STR = 
> "169.254.10.240"}

Re: [dpdk-users] DPDK 20.11 MLX5 testpmd tx_pp 'WQE index ignore feature is required for packet pacing'

[Thomas Monjalon]

09/12/2020 17:03, Alessandro Pagani:
> Hi all,
> 
> I am trying to run dpdk testpmd with Mellanox ConnectX4 Lx (mlx5 driver).
> 
> I am specifying the tx_pp parameter to provide the packet send scheduling
> on mbuf timestamps, but the testpmd fails with the following error:
[...]
> EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: :3b:00.0 (socket 0)
> mlx5_pci: No available register for Sampler.
> mlx5_pci: WQE index ignore feature is required for packet pacing
> mlx5_pci: probe of PCI device :3b:00.0 aborted after encountering an
> error: No such device
> common_mlx5: Failed to load driver = mlx5_pci.
> 
> EAL: Requested device :3b:00.0 cannot be used
[...]
> The error messages suggest that "WQE index ignore feature is required for
> packet pacing".
> 
> Anyone knows the reason of this error and how to solve it?

I think it means your device does not support this feature.
But I realize it is not documented here:
http://doc.dpdk.org/guides/nics/mlx5.html#supported-hardware-offloads

Re: [dpdk-users] Changes to DPDK kmod drivers / backward compatibility within LTS rel.

[Thomas Monjalon]

24/09/2020 19:12, Alexander Kotliarov:
> Hi there!
> I would like to find out what is the policy regarding changes to the DPDK's
> kmod drivers such as igb_uio.ko within a DPDK's LTS release. Are these
> changes backward compatible?
> For example, is there a guarantee that an application built
> against 19.11.5, where igb_uio.ko received changes,would run with this
> driver built from 19.11.1 version?
> 
> Does http://doc.dpdk.org/guides/contributing/stable.html section 8.4 apply
> to kmod drivers as well?

There is no such formal guarantee, but in my opinion,
it should be the case. Do you imagine a change in kmod
which could break a DPDK version?

Re: [dpdk-users] Error while building DPDK version 19.11 over latest kernel version

[Thomas Monjalon]

01/10/2020 19:04, Klei rama:
> Hei,
> 
> I am trying to build DPDK version 19.11 in my ubuntu machine (18.04) with
> the latest kernel version 5.9. It gives me an error while I try to build
> it. The error is when I try to build linux/kni module.
> The error looks something like this:
> 
> *dpdk/kernel/linux/kni/kni_dev.h:104:30:* *error: *passing argument 1 of ‘
> *get_user_pages_remote*’ from incompatible pointer type [
> *-Werror=incompatible-pointer-types*]
> 
> 
> Is there any workaround?

Yes, it will be fixed with the backport of this patch:
http://git.dpdk.org/dpdk/commit/?id=87efaea6376c8

> Do I need to disable that module or should I
> downgrade the kernel version? I wanted to debug my application and see if
> it needs kni module but I did not not how to disable kni module. I tried
> common_base and common_linux under config directory but could not find the
> line which disable this module.

You can disable KNI in the config file:
CONFIG_RTE_KNI_KMOD=n

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #7 from Thomas Monjalon  ---
Created attachment 131686
  --> https://bugs.kde.org/attachment.cgi?id=131686=edit
proposed fix

I think I have a fix in kmime Content::parse()
+if (ct->isEmpty())
+ct->setMimeType("text/plain");

Please check the attachment, thanks.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #7 from Thomas Monjalon  ---
Created attachment 131686
  --> https://bugs.kde.org/attachment.cgi?id=131686=edit
proposed fix

I think I have a fix in kmime Content::parse()
+if (ct->isEmpty())
+ct->setMimeType("text/plain");

Please check the attachment, thanks.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #6 from Thomas Monjalon  ---
The bug is not present with emails having a Content-Type field.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #6 from Thomas Monjalon  ---
The bug is not present with emails having a Content-Type field.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #5 from Thomas Monjalon  ---
After testing above URLs, they work fine for me as well.
The problem is with emails as they are locally. Please check with the
attachment.

Note: I do not compose as HTML, all emails are in the same folder, and there is
no special template.
I started seeing this regression recently (5.15 I believe).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #5 from Thomas Monjalon  ---
After testing above URLs, they work fine for me as well.
The problem is with emails as they are locally. Please check with the
attachment.

Note: I do not compose as HTML, all emails are in the same folder, and there is
no special template.
I started seeing this regression recently (5.15 I believe).

-- 
You are receiving this mail because:
You are watching all bug changes.

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #4 from Thomas Monjalon  ---
Created attachment 131460
  --> https://bugs.kde.org/attachment.cgi?id=131460=edit
email which is not quoted on reply

-- 
You are receiving this mail because:
You are watching all bug changes.

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #4 from Thomas Monjalon  ---
Created attachment 131460
  --> https://bugs.kde.org/attachment.cgi?id=131460=edit
email which is not quoted on reply

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #2 from Thomas Monjalon  ---
This email is not quoted on reply:
https://patches.dpdk.org/patch/76484/mbox/

This email is quoted well when replying:
https://patches.dpdk.org/patch/76166/mbox/

It seems it depends on the sender (probably its MTA).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmail2] [Bug 426238] Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

--- Comment #2 from Thomas Monjalon  ---
This email is not quoted on reply:
https://patches.dpdk.org/patch/76484/mbox/

This email is quoted well when replying:
https://patches.dpdk.org/patch/76166/mbox/

It seems it depends on the sender (probably its MTA).

-- 
You are receiving this mail because:
You are watching all bug changes.

[kmail2] [Bug 384249] midnight shuffle

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=384249

Thomas Monjalon  changed:

   What|Removed |Added

 Status|REPORTED|CONFIRMED
 Ever confirmed|0   |1

--- Comment #1 from Thomas Monjalon  ---
I've disabled grouping to avoid re-sorting mails every days, but it is still
restarting sorting after midnight.
Any tip?

-- 
You are receiving this mail because:
You are watching all bug changes.

[kmail2] [Bug 384249] midnight shuffle

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=384249

Thomas Monjalon  changed:

   What|Removed |Added

 Status|REPORTED|CONFIRMED
 Ever confirmed|0   |1

--- Comment #1 from Thomas Monjalon  ---
I've disabled grouping to avoid re-sorting mails every days, but it is still
restarting sorting after midnight.
Any tip?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmail2] [Bug 426239] New: Mailing-list address is Cc'ed 2 times

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426239

Bug ID: 426239
   Summary: Mailing-list address is Cc'ed 2 times
   Product: kmail2
   Version: unspecified
  Platform: Other
OS: Linux
Status: REPORTED
  Severity: minor
  Priority: NOR
 Component: composer
  Assignee: kdepim-bugs@kde.org
  Reporter: tho...@monjalon.net
  Target Milestone: ---

When replying to a mailing list message, the mailing list address is added as
Cc even if it is already part of the original recipients.
The consequence is to have to mailing list added 2 times.

Given a mailing-list address does not need to have a name, I would prefer
removing the original one (which may have a name added by the original sender)
and keep the one added by kmail in Cc.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmail2] [Bug 426239] New: Mailing-list address is Cc'ed 2 times

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426239

Bug ID: 426239
   Summary: Mailing-list address is Cc'ed 2 times
   Product: kmail2
   Version: unspecified
  Platform: Other
OS: Linux
Status: REPORTED
  Severity: minor
  Priority: NOR
 Component: composer
  Assignee: kdepim-b...@kde.org
  Reporter: tho...@monjalon.net
  Target Milestone: ---

When replying to a mailing list message, the mailing list address is added as
Cc even if it is already part of the original recipients.
The consequence is to have to mailing list added 2 times.

Given a mailing-list address does not need to have a name, I would prefer
removing the original one (which may have a name added by the original sender)
and keep the one added by kmail in Cc.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kmail2] [Bug 426238] New: Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

Bug ID: 426238
   Summary: Reply quote is not always pre-filled
   Product: kmail2
   Version: 5.15.0
  Platform: Archlinux Packages
OS: Linux
Status: REPORTED
  Severity: normal
  Priority: NOR
 Component: composer
  Assignee: kdepim-b...@kde.org
  Reporter: tho...@monjalon.net
  Target Milestone: ---

SUMMARY

When replying a mail, the quote is sometimes missing.

STEPS TO REPRODUCE
1. Click on "Reply"

OBSERVED RESULT

Sometimes the text area is blank, sometimes it is correctly pre-filled with
quoted message. It depends on which email is replied.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kmail2] [Bug 426238] New: Reply quote is not always pre-filled

[Thomas Monjalon]

https://bugs.kde.org/show_bug.cgi?id=426238

Bug ID: 426238
   Summary: Reply quote is not always pre-filled
   Product: kmail2
   Version: 5.15.0
  Platform: Archlinux Packages
OS: Linux
Status: REPORTED
  Severity: normal
  Priority: NOR
 Component: composer
  Assignee: kdepim-bugs@kde.org
  Reporter: tho...@monjalon.net
  Target Milestone: ---

SUMMARY

When replying a mail, the quote is sometimes missing.

STEPS TO REPRODUCE
1. Click on "Reply"

OBSERVED RESULT

Sometimes the text area is blank, sometimes it is correctly pre-filled with
quoted message. It depends on which email is replied.

-- 
You are receiving this mail because:
You are the assignee for the bug.

Re: [dpdk-users] Resolved Issues per release

[Thomas Monjalon]

12/08/2020 15:32, Jim Holland (jimholla):
> The release notes at https://doc.dpdk.org/guides-20.02/rel_notes/index.html 
> don't list the resolved issues/bugs. Is there a way to find what bugs are 
> resolved for each dpdk release?

When looking at the git history,
every commit fixing a bug should contain a line starting with "Fixes:"
identifying the root cause.

Catching such commits will give you all details of bugs fixed.

Re: [dpdk-users] meson: is there a mechanism for controlling compilation configuration

[Thomas Monjalon]

29/07/2020 09:20, Chengchang Tang:
> Hi,
> DPDK with 'make' will be deprecated in a future release. I have some
> questions about using meson to build DPDK.
> 
> When using the make, we can change the macros in config/common_base to
> control the compiling macros. For example, if i want to debug the mbuf,
> i can set CONFIG_RTE_LIBRTE_MBUF_DEBUG=y in config/common_base to change
> the compiling macros.
> 
> According to my understanding. DPDK meson build dose not generate
> rte_config.h based on common_base content during compilation. Is there
> any convenient way to modify the compiling macro in meson build. If all
> the compilation macros need to be modified using environment variable,
> it is inconvenient.

Please look at the work done in this series:
https://patches.dpdk.org/project/dpdk/list/?series=10930=*