Hi Sephen,
Thank you for your help. I made good progress up to now.
When I try to use the dpdk_simple_mp application to send a message to my
application I get a Segmentation fault.
First, I re-verified the dpdk_simple_mp process=primary and dpdk_simple_mp
process-secondary does pass messages successfully. So, my hugepages are
created and DPDK initializes successfully on both at startup.
In my application I created the send and recv rings and message_pool as the
primary process. The logs I added do not show any errors.
Once my application starts and settles I started the dpdk_simple_mp
application: # ./dpdk-simple_mp_dbg -l 30-31 -n 4 --legacy-mem --proc-type
secondary --
However, on the dpdk_simple_mp side do "send hello" and I then get a
segmentation fault.
The debugger takes me deep within the dpdk libraries which I am not too
familiar with.
The rte_ring_elem.h file function: rte_ring_dequeue_build_elem() is where I end
up with segmentation fault. I notice that the variables are optimized out, not
sure why since I built the dpdk libraries with debug flag.
Here is the back trace and could you point me in the direction to look.
# gdb dpdk-simple_mp /core/core.dpdk-simple_mp.241269
warning: Unexpected size of section `.reg-xstate/241269' in core file.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `./dpdk-simple_mp -l 30-31 -n 4 --legacy-mem --proc-type
secondary --'.
Program terminated with signal SIGSEGV, Segmentation fault.
warning: Unexpected size of section `.reg-xstate/241269' in core file.
#0 0x0000000000cf446a in bucket_dequeue ()
[Current thread is 1 (Thread 0x7f946f835c00 (LWP 241269))]
Missing separate debuginfos, use: dnf debuginfo-install
elfutils-libelf-0.189-3.el9.x86_64 glibc-2.34-83.0.1.el9_3.7.x86_64
libibverbs-46.0-1.el9.x86_64 libnl3-3.7.0-1.el9.x86_64
libpcap-1.10.0-4.el9.x86_64 libzstd-1.5.1-2.el9.x86_64
numactl-libs-2.0.16-1.el9.x86_64 openssl-libs-3.0.7-25.0.1.el9_3.x86_64
zlib-1.2.11-40.el9.x86_64
(gdb) bt
#0 0x0000000000cf446a in bucket_dequeue ()
#1 0x00000000007ce77d in cmd_send_parsed ()
#2 0x0000000000aa5d96 in __cmdline_parse ()
#3 0x0000000000aa4d70 in cmdline_valid_buffer ()
#4 0x0000000000aa826b in rdline_char_in ()
#5 0x0000000000aa4e41 in cmdline_in ()
#6 0x0000000000aa4f60 in cmdline_interact ()
#7 0x00000000004fe47a in main.cold ()
#8 0x00007f946f03feb0 in __libc_start_call_main () from /lib64/libc.so.6
#9 0x00007f946f03ff60 in __libc_start_main_impl () from /lib64/libc.so.6
#10 0x00000000007ce605 in _start ()
Gdb - stepping through the code, gdb attached to dpdk_simple_mp_debug
(gdb)
0x0000000000cf42c5 in rte_ring_dequeue_bulk_elem (available=<optimized out>,
n=<optimized out>, esize=<optimized out>,
obj_table=<optimized out>, r=<optimized out>) at
../lib/ring/rte_ring_elem.h:375
375 ../lib/ring/rte_ring_elem.h: No such file or directory.
(gdb) p r
$17 = <optimized out>
(gdb) p obj_table
$18 = <optimized out>
(gdb) p available
$19 = <optimized out>
(gdb) n
Thread 1 "dpdk-simple_mp_" received signal SIGSEGV, Segmentation fault.
bucket_dequeue_orphans (n_orphans=33, obj_table=0x14f09b5c0, bd=0x14f05aa80)
at ../drivers/mempool/bucket/rte_mempool_bucket.c:191
191 ../drivers/mempool/bucket/rte_mempool_bucket.c: No such file or
directory.
(gdb) bt
#0 bucket_dequeue_orphans (n_orphans=33, obj_table=0x14f09b5c0, bd=0x14f05aa80)
at ../drivers/mempool/bucket/rte_mempool_bucket.c:191
#1 bucket_dequeue (mp=<optimized out>, obj_table=0x14f09b5c0, n=33) at
../drivers/mempool/bucket/rte_mempool_bucket.c:289
#2 0x00000000007ce77d in rte_mempool_ops_dequeue_bulk (n=<optimized out>,
obj_table=0x14f09b5c0, mp=0x14f05ed40)
at ../lib/mempool/rte_mempool.h:793
#3 rte_mempool_do_generic_get (cache=0x14f09b580, n=1,
obj_table=0x7fff8df066f0, mp=0x14f05ed40)
at ../lib/mempool/rte_mempool.h:1570
#4 rte_mempool_generic_get (cache=0x14f09b580, n=1, obj_table=0x7fff8df066f0,
mp=0x14f05ed40)
at ../lib/mempool/rte_mempool.h:1649
#5 rte_mempool_get_bulk (n=1, obj_table=0x7fff8df066f0, mp=0x14f05ed40) at
../lib/mempool/rte_mempool.h:1684
#6 rte_mempool_get (obj_p=0x7fff8df066f0, mp=0x14f05ed40) at
../lib/mempool/rte_mempool.h:1710
#7 cmd_send_parsed (parsed_result=parsed_result@entry=0x7fff8df06790,
cl=cl@entry=0x2f73220, data=data@entry=0x0)
at ../examples/multi_process/simple_mp/mp_commands.c:18
#8 0x0000000000aa5d96 in __cmdline_parse (cl=cl@entry=0x2f73220, buf=0x2f73268
"send hello\n",
call_fn=call_fn@entry=true) at ../lib/cmdline/cmdline_parse.c:294
#9 0x0000000000aa5f1a in cmdline_parse (cl=cl@entry=0x2f73220, buf=<optimized
out>) at ../lib/cmdline/cmdline_parse.c:302
#10 0x0000000000aa4d70 in cmdline_valid_buffer (rdl=<optimized out>,
buf=<optimized out>, size=<optimized out>)
at ../lib/cmdline/cmdline.c:24
#11 0x0000000000aa826b in rdline_char_in (rdl=rdl@entry=0x2f73230, c=<optimized
out>)
at ../lib/cmdline/cmdline_rdline.c:444
#12 0x0000000000aa4e41 in cmdline_in (size=<optimized out>, buf=<optimized
out>, cl=<optimized out>)
at ../lib/cmdline/cmdline.c:146
#13 cmdline_in (cl=0x2f73220, buf=0x7fff8df0c89f "\n\200", size=<optimized
out>) at ../lib/cmdline/cmdline.c:135
#14 0x0000000000aa4f60 in cmdline_interact (cl=cl@entry=0x2f73220) at
../lib/cmdline/cmdline.c:192
#15 0x00000000004fe47a in main (argc=<optimized out>, argv=<optimized out>)
at ../examples/multi_process/simple_mp/main.c:122
Appreciate if you can help.
Thanks,
Ed
-----Original Message-----
From: Stephen Hemminger <step...@networkplumber.org>
Sent: Wednesday, March 19, 2025 7:17 PM
To: Lombardo, Ed <ed.lomba...@netscout.com>
Cc: users@dpdk.org
Subject: Re: tailqs issue
External Email: This message originated outside of NETSCOUT. Do not click links
or open attachments unless you recognize the sender and know the content is
safe.
On Wed, 19 Mar 2025 21:52:39 +0000
"Lombardo, Ed" <ed.lomba...@netscout.com> wrote:
> Hi Stephen,
> I added the fib library, but I now see there are many more dpdk libraries I
> need to add. Is this typically the case with the example files working with
> primary DPDK application?
>
> I am using meson and ninja to build the examples, but I don't know how to
> know the library dependencies.
>
> How do I learn ahead of building my Application as to what extra libraries I
> need to include for the DPDK example to work?
>
> I am doing incremental build-test-find_missing_library.
>
> So far, I needed to add these: -lrte_fib -lrte_rib -lrte_stack -lrte_member
> -lrte_efd
>
> Thanks,
> Ed
The typical case is to make sure that primary and secondary are built with the
same libraries.
