RE: I need DPDK MLX5 Probe error support
Hi Thomas, I attached the testpmd result which is gathered on the host sever. Could you please take a look at the mlx5_core PCI issue? Thank you in advance. BR/Jaeeun -Original Message- From: Thomas Monjalon Sent: Sunday, October 3, 2021 4:51 PM To: Jaeeun Ham Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com Subject: Re: I need DPDK MLX5 Probe error support Hi, I think you need to read the documentation. For DPDK install on Linux: https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d8a30ca42b-d871f122b4a0a61a&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fbuild_dpdk.html%23compiling-and-installing-dpdk-system-wide For mlx5 specific dependencies, install rdma-core package: https://protect2.fireeye.com/v1/url?k=9bce4984-c45570e5-9bce091f-86d8a30ca42b-25bd3d467b5f290d&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23linux-prerequisites 02/10/2021 12:57, Jaeeun Ham: > Hi, > > Could you teach me how to install dpdk-testpmd? > I have to run the application on the host server, not a development server. > So, I don't know how to get dpdk-testpmd. > > By the way, testpmd run result is as below. > root@seroics05590:~/ejaeham# testpmd > EAL: Detected 64 lcore(s) > EAL: libmlx4.so.1: cannot open shared object file: No such file or > directory > EAL: FATAL: Cannot init plugins > > EAL: Cannot init plugins > > PANIC in main(): > Cannot init EAL > 5: [testpmd(_start+0x2a) [0x55d301d98e1a]] > 4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) > [0x7f5e044a4bf7]] > 3: [testpmd(main+0x907) [0x55d301d98d07]] > 2: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(__rte_panic+0xbd) > [0x7f5e04ca3cfd]] > 1: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(rte_dump_stack+0x2e) > [0x7f5e04cac19e]] Aborted > > > I added option below when the process is starting in the docker. > dv_flow_en=0 \ > --log-level=pmd,8 \ > < MLX5 log > > 415a695ba348:/tmp/logs # cat epp.log > MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1 > MIDHAUL_IP_ADDR:10.255.21.177, BACKHAUL_IP_ADDR:10.255.21.178 > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.0 cannot be used > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.1 cannot be used > EAL: Bus (pci) probe failed. > EAL: Trying to obtain current memory policy. > EAL: Setting policy MPOL_PREFERRED for socket 1 Caught signal 15 > EAL: Restoring previous memory policy: 0 > EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)' > EAL: request: mp_malloc_sync > EAL: Heap on socket 1 was expanded by 5120MB > FATAL: epp_init.c::copy_mac_addr:130: Call to > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, &port_id) failed: -19 > (Unknown error -19), rte_errno=0 (not set) > > Caught signal 6 > Obtained 7 stack frames, tid=713. > tid=713, /usr/local/bin/ericsson-packet-processor() [0x40a4a4] > tid=713, /lib64/libpthread.so.0(+0x13f80) [0x7f7e1eae8f80] tid=713, > /lib64/libc.so.6(gsignal+0x10b) [0x7f7e1c5f818b] tid=713, > /lib64/libc.so.6(abort+0x175) [0x7f7e1c5f9585] tid=713, > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] > tid=713, /lib64/libc.so.6(__libc_start_main+0xed) [0x7f7e1c5e334d] > tid=713, /usr/local/bin/ericsson-packet-processor(_start+0x2a) > [0x4091ca] > > < i40e log > > cat epp.log > MIDHAUL_PCI_ADDR::3b:0d.5, BACKHAUL_PCI_ADDR::3b:0d.4 > MIDHAUL_IP_ADDR:10.51.21.112, BACKHAUL_IP_ADDR:10.51.21.113 > EAL: Trying to obtain current memory policy. > EAL: Setting policy MPOL_PREFERRED for socket 1 > EAL: Restoring previous memory policy: 0 > EAL: Calling mem event callback 'vfio_mem_event_clb:(nil)' > EAL: request: mp_malloc_sync > EAL: Heap on socket 1 was expanded by 5120MB > i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported > i40evf_handle_aq_msg(): adminq response is received, opcode = 28 > i40e_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are satisfied. > Rx Burst Bulk Alloc function will be used on port=0, queue=0. > i40e_set_tx_function_flag(): Neither simple nor vector Tx enabled on > Tx queue 0 > > i40evf_dev_start(): >> > i40evf_config_rss(): No hash flag is set > i40e_set_rx_function(): Vector Rx path will be used on port=0. > i40e_set_tx_function(): Xmit tx finally be used. > i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported > i40evf_handle_aq_msg(): adminq
RE: I need DPDK MLX5 Probe error support
Hi Thomas, I attached both testpmd results. (host server, docker container) Could you interpret what is the fault cause? BTW, I added LD_LIBRARY_PATH=libjansson.so.4 to run the dpdk-testpmd on the host. mlx5_nl_ifindex() seems to require linux kernel 4.17 at least. mlx5_nl_send() and mlx5_nl_recv() looks normal. However, it is a bit strange that data.flags is set to MLX5_NL_CMD_GET_INDEX instead of MLX5_CMD_GET_INDEX in the second command, Anyway, rather than these functions, we estimate that the RDMA_NLDEV_CMD_PORT_GET command does not work in this function. * This is the only somewhat safe method to avoid resorting to heuristics * when faced with port representors. Unfortunately it requires at least * Linux 4.17. BR/Jaeeun -Original Message- From: Jaeeun Ham Sent: Saturday, October 9, 2021 10:15 AM To: 'Thomas Monjalon' mailto:tho...@monjalon.net>> Cc: 'users@dpdk.org' mailto:users@dpdk.org>>; 'alia...@nvidia.com' mailto:alia...@nvidia.com>>; 'rasl...@nvidia.com' mailto:rasl...@nvidia.com>>; 'as...@nvidia.com' mailto:as...@nvidia.com>>; Keunho Shin mailto:keunho.s...@ericsson.com>>; Joonmo Koo mailto:joonmo@ericsson.com>> Subject: RE: I need DPDK MLX5 Probe error support +Keunho Shin mailto:keunho.s...@ericsson.com>>; Joonmo Koo +mailto:joonmo@ericsson.com>> -Original Message- From: Jaeeun Ham Sent: Saturday, October 9, 2021 10:12 AM To: Thomas Monjalon mailto:tho...@monjalon.net>> Cc: users@dpdk.org<mailto:users@dpdk.org>; alia...@nvidia.com<mailto:alia...@nvidia.com>; rasl...@nvidia.com<mailto:rasl...@nvidia.com>; as...@nvidia.com<mailto:as...@nvidia.com> Subject: RE: I need DPDK MLX5 Probe error support Hi Thomas, I am sorry for the inconvenience. Docker is compiled as a layered, so three attached Dockerfiles are used. Dockerfile.prod.base: application base Dockerfile.prod: application Dockerfile.build: tool chains < docker container version > 4a9d999f0c15:/ # uname -a Linux 4a9d999f0c15 4.15.0-46-generic #49-Ubuntu SMP Wed Feb 6 09:33:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux BR/Jaeeun -Original Message- From: Thomas Monjalon mailto:tho...@monjalon.net>> Sent: Wednesday, October 6, 2021 10:19 PM To: Jaeeun Ham mailto:jaeeun@ericsson.com>> Cc: users@dpdk.org<mailto:users@dpdk.org>; alia...@nvidia.com<mailto:alia...@nvidia.com>; rasl...@nvidia.com<mailto:rasl...@nvidia.com>; as...@nvidia.com<mailto:as...@nvidia.com> Subject: Re: I need DPDK MLX5 Probe error support I don't even know which Linux distribution you are using. Please send the Dockerfile. If it compiles in Docker, it should run. 06/10/2021 14:27, Jaeeun Ham: > Hi Thomas, > > The cause is that I fail to load mlx5 driver using pci address on the docker > container. > So, I tried to add rdma-core library to solve dependency issue you mentioned > as below. > Docker image is built with these Dockerfiles. > This docker image is built with DPDK20.11. > How should I add rdma-core library? > > I don't find any rdma related so files in the docker container. > b273016e5be8:/usr/local/lib # ls *mlx* librte_common_mlx5.so > librte_common_mlx5.so.21 librte_common_mlx5.so.21.0 > librte_net_mlx5.so librte_net_mlx5.so.21 librte_net_mlx5.so.21.0 > b273016e5be8:/usr/local/lib # ls *rdma* > ls: cannot access '*rdma*': No such file or directory > > dpdk-20.11/doc/guides/rel_notes/release_20_11.rst > 911: * rdma-core: > 913:* rdma-core-31.0-1 and above > > > > < error log > > f1d23550a947:/ # cat /tmp/logs/epp.log MIDHAUL_PCI_ADDR::12:01.0, > BACKHAUL_PCI_ADDR::12:01.1 MIDHAUL_IP_ADDR:10.255.20.125, > BACKHAUL_IP_ADDR:10.255.20.124 > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.0 cannot be used > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.1 cannot be used > EAL: Bus (pci) probe failed. > FATAL: epp_init.c::copy_mac_addr:130: Call to > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, &port_id) failed: -19 > (Unknown error -19), rte_errno=0 (not set) > > Caught signal 6 > Obtained 7 stack frames, tid=1377. > tid=1377, /usr/local/bin/ericsson-packet-processor() [0x40a3c4] > tid=1377, /lib64/libpthread.so.0(+0x13f80) [0x7f56c4786f80] tid=1377, > /lib64/libc.so.6(gsignal+0x10b) [0x7f56c229018b] tid=1377, > /lib64/libc.so.6(abort+0x175) [0x7f56c2291585] tid=1377, > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] > tid=1377, /lib64/libc.so.6(__libc_start_main+0xed) [
RE: I need DPDK MLX5 Probe error support
Hi Thomas, I am sorry for the inconvenience. Docker is compiled as a layered, so three attached Dockerfiles are used. Dockerfile.prod.base: application base Dockerfile.prod: application Dockerfile.build: tool chains < docker container version > 4a9d999f0c15:/ # uname -a Linux 4a9d999f0c15 4.15.0-46-generic #49-Ubuntu SMP Wed Feb 6 09:33:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux BR/Jaeeun -Original Message- From: Thomas Monjalon Sent: Wednesday, October 6, 2021 10:19 PM To: Jaeeun Ham Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com Subject: Re: I need DPDK MLX5 Probe error support I don't even know which Linux distribution you are using. Please send the Dockerfile. If it compiles in Docker, it should run. 06/10/2021 14:27, Jaeeun Ham: > Hi Thomas, > > The cause is that I fail to load mlx5 driver using pci address on the docker > container. > So, I tried to add rdma-core library to solve dependency issue you mentioned > as below. > Docker image is built with these Dockerfiles. > This docker image is built with DPDK20.11. > How should I add rdma-core library? > > I don't find any rdma related so files in the docker container. > b273016e5be8:/usr/local/lib # ls *mlx* librte_common_mlx5.so > librte_common_mlx5.so.21 librte_common_mlx5.so.21.0 > librte_net_mlx5.so librte_net_mlx5.so.21 librte_net_mlx5.so.21.0 > b273016e5be8:/usr/local/lib # ls *rdma* > ls: cannot access '*rdma*': No such file or directory > > dpdk-20.11/doc/guides/rel_notes/release_20_11.rst > 911: * rdma-core: > 913:* rdma-core-31.0-1 and above > > > > < error log > > f1d23550a947:/ # cat /tmp/logs/epp.log MIDHAUL_PCI_ADDR::12:01.0, > BACKHAUL_PCI_ADDR::12:01.1 MIDHAUL_IP_ADDR:10.255.20.125, > BACKHAUL_IP_ADDR:10.255.20.124 > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.0 cannot be used > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.1 cannot be used > EAL: Bus (pci) probe failed. > FATAL: epp_init.c::copy_mac_addr:130: Call to > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, &port_id) failed: -19 > (Unknown error -19), rte_errno=0 (not set) > > Caught signal 6 > Obtained 7 stack frames, tid=1377. > tid=1377, /usr/local/bin/ericsson-packet-processor() [0x40a3c4] > tid=1377, /lib64/libpthread.so.0(+0x13f80) [0x7f56c4786f80] tid=1377, > /lib64/libc.so.6(gsignal+0x10b) [0x7f56c229018b] tid=1377, > /lib64/libc.so.6(abort+0x175) [0x7f56c2291585] tid=1377, > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] > tid=1377, /lib64/libc.so.6(__libc_start_main+0xed) [0x7f56c227b34d] > tid=1377, /usr/local/bin/ericsson-packet-processor(_start+0x2a) > [0x4090ea] > > > > BR/Jaeeun > > -----Original Message----- > From: Thomas Monjalon > Sent: Wednesday, October 6, 2021 7:59 PM > To: Jaeeun Ham > Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; > as...@nvidia.com > Subject: Re: I need DPDK MLX5 Probe error support > > Installing dependencies is not an issue. > I don't understand which support you need. > > > 06/10/2021 11:57, Jaeeun Ham: > > Hi Thomas, > > > > Could you take a look at the attached file? > > My engineer managed to compile DPDK 20.11 to support MLX5. Please find the > > output from dpdk-testpmd command in attached file. As you can see testpmd > > was able to probe mlx5_pci drivers and get MAC addresses. > > The key issue in his case for enabling MLX5 support was to export rdma-core > > lib path to shared libs for meson/ninja commands as new build system > > automatically enables MLX5 support if needed dependencies are available. > > > > BR/Jaeeun > > > > -Original Message- > > From: Thomas Monjalon > > mailto:tho...@monjalon.net>> > > Sent: Sunday, October 3, 2021 4:51 PM > > To: Jaeeun Ham > > mailto:jaeeun@ericsson.com>> > > Cc: users@dpdk.org<mailto:users@dpdk.org>; > > alia...@nvidia.com<mailto:alia...@nvidia.com>; > > rasl...@nvidia.com<mailto:rasl...@nvidia.com>; > > as...@nvidia.com<mailto:as...@nvidia.com> > > Subject: Re: I need DPDK MLX5 Probe error support > > > > Hi, > > > > I think you need to read the documentation. > > For DPDK install on Linux: > > https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d > > 8a > > 30ca42b-d871f122b4a0a61a&q=1&e=
RE: I need DPDK MLX5 Probe error support
Hi Thomas, Could you take a look at the attached file? My engineer managed to compile DPDK 20.11 to support MLX5. Please find the output from dpdk-testpmd command in attached file. As you can see testpmd was able to probe mlx5_pci drivers and get MAC addresses. The key issue in his case for enabling MLX5 support was to export rdma-core lib path to shared libs for meson/ninja commands as new build system automatically enables MLX5 support if needed dependencies are available. BR/Jaeeun -Original Message- From: Thomas Monjalon Sent: Sunday, October 3, 2021 4:51 PM To: Jaeeun Ham Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com Subject: Re: I need DPDK MLX5 Probe error support Hi, I think you need to read the documentation. For DPDK install on Linux: https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d8a30ca42b-d871f122b4a0a61a&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fbuild_dpdk.html%23compiling-and-installing-dpdk-system-wide For mlx5 specific dependencies, install rdma-core package: https://protect2.fireeye.com/v1/url?k=9bce4984-c45570e5-9bce091f-86d8a30ca42b-25bd3d467b5f290d&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23linux-prerequisites 02/10/2021 12:57, Jaeeun Ham: > Hi, > > Could you teach me how to install dpdk-testpmd? > I have to run the application on the host server, not a development server. > So, I don't know how to get dpdk-testpmd. > > By the way, testpmd run result is as below. > root@seroics05590:~/ejaeham# testpmd > EAL: Detected 64 lcore(s) > EAL: libmlx4.so.1: cannot open shared object file: No such file or > directory > EAL: FATAL: Cannot init plugins > > EAL: Cannot init plugins > > PANIC in main(): > Cannot init EAL > 5: [testpmd(_start+0x2a) [0x55d301d98e1a]] > 4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) > [0x7f5e044a4bf7]] > 3: [testpmd(main+0x907) [0x55d301d98d07]] > 2: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(__rte_panic+0xbd) > [0x7f5e04ca3cfd]] > 1: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(rte_dump_stack+0x2e) > [0x7f5e04cac19e]] Aborted > > > I added option below when the process is starting in the docker. > dv_flow_en=0 \ > --log-level=pmd,8 \ > < MLX5 log > > 415a695ba348:/tmp/logs # cat epp.log > MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1 > MIDHAUL_IP_ADDR:10.255.21.177, BACKHAUL_IP_ADDR:10.255.21.178 > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.0 cannot be used > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.1 cannot be used > EAL: Bus (pci) probe failed. > EAL: Trying to obtain current memory policy. > EAL: Setting policy MPOL_PREFERRED for socket 1 Caught signal 15 > EAL: Restoring previous memory policy: 0 > EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)' > EAL: request: mp_malloc_sync > EAL: Heap on socket 1 was expanded by 5120MB > FATAL: epp_init.c::copy_mac_addr:130: Call to > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, &port_id) failed: -19 > (Unknown error -19), rte_errno=0 (not set) > > Caught signal 6 > Obtained 7 stack frames, tid=713. > tid=713, /usr/local/bin/ericsson-packet-processor() [0x40a4a4] > tid=713, /lib64/libpthread.so.0(+0x13f80) [0x7f7e1eae8f80] tid=713, > /lib64/libc.so.6(gsignal+0x10b) [0x7f7e1c5f818b] tid=713, > /lib64/libc.so.6(abort+0x175) [0x7f7e1c5f9585] tid=713, > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] > tid=713, /lib64/libc.so.6(__libc_start_main+0xed) [0x7f7e1c5e334d] > tid=713, /usr/local/bin/ericsson-packet-processor(_start+0x2a) > [0x4091ca] > > < i40e log > > cat epp.log > MIDHAUL_PCI_ADDR::3b:0d.5, BACKHAUL_PCI_ADDR::3b:0d.4 > MIDHAUL_IP_ADDR:10.51.21.112, BACKHAUL_IP_ADDR:10.51.21.113 > EAL: Trying to obtain current memory policy. > EAL: Setting policy MPOL_PREFERRED for socket 1 > EAL: Restoring previous memory policy: 0 > EAL: Calling mem event callback 'vfio_mem_event_clb:(nil)' > EAL: request: mp_malloc_sync > EAL: Heap on socket 1 was expanded by 5120MB > i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported > i40evf_handle_aq_msg(): adminq response is received, opcode = 28 > i40e_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are satisfied. > Rx Burst Bulk Alloc function will be used on port=0, queue=0. > i40e_set_tx_function_flag(): Neither simple nor vector Tx enabled on > Tx
RE: I need DPDK MLX5 Probe error support
+Keunho Shin ; Joonmo Koo -Original Message- From: Jaeeun Ham Sent: Saturday, October 9, 2021 10:12 AM To: Thomas Monjalon Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com Subject: RE: I need DPDK MLX5 Probe error support Hi Thomas, I am sorry for the inconvenience. Docker is compiled as a layered, so three attached Dockerfiles are used. Dockerfile.prod.base: application base Dockerfile.prod: application Dockerfile.build: tool chains < docker container version > 4a9d999f0c15:/ # uname -a Linux 4a9d999f0c15 4.15.0-46-generic #49-Ubuntu SMP Wed Feb 6 09:33:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux BR/Jaeeun -Original Message- From: Thomas Monjalon Sent: Wednesday, October 6, 2021 10:19 PM To: Jaeeun Ham Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com Subject: Re: I need DPDK MLX5 Probe error support I don't even know which Linux distribution you are using. Please send the Dockerfile. If it compiles in Docker, it should run. 06/10/2021 14:27, Jaeeun Ham: > Hi Thomas, > > The cause is that I fail to load mlx5 driver using pci address on the docker > container. > So, I tried to add rdma-core library to solve dependency issue you mentioned > as below. > Docker image is built with these Dockerfiles. > This docker image is built with DPDK20.11. > How should I add rdma-core library? > > I don't find any rdma related so files in the docker container. > b273016e5be8:/usr/local/lib # ls *mlx* librte_common_mlx5.so > librte_common_mlx5.so.21 librte_common_mlx5.so.21.0 > librte_net_mlx5.so librte_net_mlx5.so.21 librte_net_mlx5.so.21.0 > b273016e5be8:/usr/local/lib # ls *rdma* > ls: cannot access '*rdma*': No such file or directory > > dpdk-20.11/doc/guides/rel_notes/release_20_11.rst > 911: * rdma-core: > 913:* rdma-core-31.0-1 and above > > > > < error log > > f1d23550a947:/ # cat /tmp/logs/epp.log MIDHAUL_PCI_ADDR::12:01.0, > BACKHAUL_PCI_ADDR::12:01.1 MIDHAUL_IP_ADDR:10.255.20.125, > BACKHAUL_IP_ADDR:10.255.20.124 > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.0 cannot be used > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.1 cannot be used > EAL: Bus (pci) probe failed. > FATAL: epp_init.c::copy_mac_addr:130: Call to > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, &port_id) failed: -19 > (Unknown error -19), rte_errno=0 (not set) > > Caught signal 6 > Obtained 7 stack frames, tid=1377. > tid=1377, /usr/local/bin/ericsson-packet-processor() [0x40a3c4] > tid=1377, /lib64/libpthread.so.0(+0x13f80) [0x7f56c4786f80] tid=1377, > /lib64/libc.so.6(gsignal+0x10b) [0x7f56c229018b] tid=1377, > /lib64/libc.so.6(abort+0x175) [0x7f56c2291585] tid=1377, > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] > tid=1377, /lib64/libc.so.6(__libc_start_main+0xed) [0x7f56c227b34d] > tid=1377, /usr/local/bin/ericsson-packet-processor(_start+0x2a) > [0x4090ea] > > > > BR/Jaeeun > > -----Original Message----- > From: Thomas Monjalon > Sent: Wednesday, October 6, 2021 7:59 PM > To: Jaeeun Ham > Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; > as...@nvidia.com > Subject: Re: I need DPDK MLX5 Probe error support > > Installing dependencies is not an issue. > I don't understand which support you need. > > > 06/10/2021 11:57, Jaeeun Ham: > > Hi Thomas, > > > > Could you take a look at the attached file? > > My engineer managed to compile DPDK 20.11 to support MLX5. Please find the > > output from dpdk-testpmd command in attached file. As you can see testpmd > > was able to probe mlx5_pci drivers and get MAC addresses. > > The key issue in his case for enabling MLX5 support was to export rdma-core > > lib path to shared libs for meson/ninja commands as new build system > > automatically enables MLX5 support if needed dependencies are available. > > > > BR/Jaeeun > > > > -Original Message- > > From: Thomas Monjalon > > mailto:tho...@monjalon.net>> > > Sent: Sunday, October 3, 2021 4:51 PM > > To: Jaeeun Ham > > mailto:jaeeun@ericsson.com>> > > Cc: users@dpdk.org<mailto:users@dpdk.org>; > > alia...@nvidia.com<mailto:alia...@nvidia.com>; > > rasl...@nvidia.com<mailto:rasl...@nvidia.com>; > > as...@nvidia.com<mailto:as...@nvidia.com> > > Subject: Re: I need DPDK MLX5 Probe error support > > > > Hi
Re: I need DPDK MLX5 Probe error support
I don't even know which Linux distribution you are using. Please send the Dockerfile. If it compiles in Docker, it should run. 06/10/2021 14:27, Jaeeun Ham: > Hi Thomas, > > The cause is that I fail to load mlx5 driver using pci address on the docker > container. > So, I tried to add rdma-core library to solve dependency issue you mentioned > as below. > Docker image is built with these Dockerfiles. > This docker image is built with DPDK20.11. > How should I add rdma-core library? > > I don't find any rdma related so files in the docker container. > b273016e5be8:/usr/local/lib # ls *mlx* > librte_common_mlx5.so librte_common_mlx5.so.21 librte_common_mlx5.so.21.0 > librte_net_mlx5.so librte_net_mlx5.so.21 librte_net_mlx5.so.21.0 > b273016e5be8:/usr/local/lib # ls *rdma* > ls: cannot access '*rdma*': No such file or directory > > dpdk-20.11/doc/guides/rel_notes/release_20_11.rst > 911: * rdma-core: > 913:* rdma-core-31.0-1 and above > > > > < error log > > f1d23550a947:/ # cat /tmp/logs/epp.log > MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1 > MIDHAUL_IP_ADDR:10.255.20.125, BACKHAUL_IP_ADDR:10.255.20.124 > mlx5_pci: unable to recognize master/representors on the multiple IB devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.0 cannot be used > mlx5_pci: unable to recognize master/representors on the multiple IB devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.1 cannot be used > EAL: Bus (pci) probe failed. > FATAL: epp_init.c::copy_mac_addr:130: Call to > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, &port_id) failed: -19 > (Unknown error -19), rte_errno=0 (not set) > > Caught signal 6 > Obtained 7 stack frames, tid=1377. > tid=1377, /usr/local/bin/ericsson-packet-processor() [0x40a3c4] tid=1377, > /lib64/libpthread.so.0(+0x13f80) [0x7f56c4786f80] tid=1377, > /lib64/libc.so.6(gsignal+0x10b) [0x7f56c229018b] tid=1377, > /lib64/libc.so.6(abort+0x175) [0x7f56c2291585] tid=1377, > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] tid=1377, > /lib64/libc.so.6(__libc_start_main+0xed) [0x7f56c227b34d] tid=1377, > /usr/local/bin/ericsson-packet-processor(_start+0x2a) [0x4090ea] > > > > BR/Jaeeun > > -Original Message----- > From: Thomas Monjalon > Sent: Wednesday, October 6, 2021 7:59 PM > To: Jaeeun Ham > Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com > Subject: Re: I need DPDK MLX5 Probe error support > > Installing dependencies is not an issue. > I don't understand which support you need. > > > 06/10/2021 11:57, Jaeeun Ham: > > Hi Thomas, > > > > Could you take a look at the attached file? > > My engineer managed to compile DPDK 20.11 to support MLX5. Please find the > > output from dpdk-testpmd command in attached file. As you can see testpmd > > was able to probe mlx5_pci drivers and get MAC addresses. > > The key issue in his case for enabling MLX5 support was to export rdma-core > > lib path to shared libs for meson/ninja commands as new build system > > automatically enables MLX5 support if needed dependencies are available. > > > > BR/Jaeeun > > > > -Original Message- > > From: Thomas Monjalon mailto:tho...@monjalon.net>> > > Sent: Sunday, October 3, 2021 4:51 PM > > To: Jaeeun Ham mailto:jaeeun@ericsson.com>> > > Cc: users@dpdk.org<mailto:users@dpdk.org>; > > alia...@nvidia.com<mailto:alia...@nvidia.com>; > > rasl...@nvidia.com<mailto:rasl...@nvidia.com>; > > as...@nvidia.com<mailto:as...@nvidia.com> > > Subject: Re: I need DPDK MLX5 Probe error support > > > > Hi, > > > > I think you need to read the documentation. > > For DPDK install on Linux: > > https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d8a > > 30ca42b-d871f122b4a0a61a&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u= > > https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fbuild_dpdk.html%23co > > mpiling-and-installing-dpdk-system-wide > > For mlx5 specific dependencies, install rdma-core package: > > https://protect2.fireeye.com/v1/url?k=9bce4984-c45570e5-9bce091f-86d8a > > 30ca42b-25bd3d467b5f290d&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u= > > https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23linux-prerequ > > isites > > > > > > 02/10/2021 12:57, Jaeeun Ham: > > > Hi, > > > > > > Could you teach me how to install dpdk-testpmd? > > > I have to run the applicatio
Re: I need DPDK MLX5 Probe error support
Installing dependencies is not an issue. I don't understand which support you need. 06/10/2021 11:57, Jaeeun Ham: > Hi Thomas, > > Could you take a look at the attached file? > My engineer managed to compile DPDK 20.11 to support MLX5. Please find the > output from dpdk-testpmd command in attached file. As you can see testpmd was > able to probe mlx5_pci drivers and get MAC addresses. > The key issue in his case for enabling MLX5 support was to export rdma-core > lib path to shared libs for meson/ninja commands as new build system > automatically enables MLX5 support if needed dependencies are available. > > BR/Jaeeun > > -Original Message- > From: Thomas Monjalon > Sent: Sunday, October 3, 2021 4:51 PM > To: Jaeeun Ham > Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com > Subject: Re: I need DPDK MLX5 Probe error support > > Hi, > > I think you need to read the documentation. > For DPDK install on Linux: > https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d8a30ca42b-d871f122b4a0a61a&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fbuild_dpdk.html%23compiling-and-installing-dpdk-system-wide > For mlx5 specific dependencies, install rdma-core package: > https://protect2.fireeye.com/v1/url?k=9bce4984-c45570e5-9bce091f-86d8a30ca42b-25bd3d467b5f290d&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23linux-prerequisites > > > 02/10/2021 12:57, Jaeeun Ham: > > Hi, > > > > Could you teach me how to install dpdk-testpmd? > > I have to run the application on the host server, not a development server. > > So, I don't know how to get dpdk-testpmd. > > > > By the way, testpmd run result is as below. > > root@seroics05590:~/ejaeham# testpmd > > EAL: Detected 64 lcore(s) > > EAL: libmlx4.so.1: cannot open shared object file: No such file or > > directory > > EAL: FATAL: Cannot init plugins > > > > EAL: Cannot init plugins > > > > PANIC in main(): > > Cannot init EAL > > 5: [testpmd(_start+0x2a) [0x55d301d98e1a]] > > 4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) > > [0x7f5e044a4bf7]] > > 3: [testpmd(main+0x907) [0x55d301d98d07]] > > 2: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(__rte_panic+0xbd) > > [0x7f5e04ca3cfd]] > > 1: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(rte_dump_stack+0x2e) > > [0x7f5e04cac19e]] Aborted > > > > > > I added option below when the process is starting in the docker. > > dv_flow_en=0 \ > > --log-level=pmd,8 \ > > < MLX5 log > > > 415a695ba348:/tmp/logs # cat epp.log > > MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1 > > MIDHAUL_IP_ADDR:10.255.21.177, BACKHAUL_IP_ADDR:10.255.21.178 > > mlx5_pci: unable to recognize master/representors on the multiple IB > > devices > > common_mlx5: Failed to load driver = mlx5_pci. > > > > EAL: Requested device :12:01.0 cannot be used > > mlx5_pci: unable to recognize master/representors on the multiple IB > > devices > > common_mlx5: Failed to load driver = mlx5_pci. > > > > EAL: Requested device :12:01.1 cannot be used > > EAL: Bus (pci) probe failed. > > EAL: Trying to obtain current memory policy. > > EAL: Setting policy MPOL_PREFERRED for socket 1 Caught signal 15 > > EAL: Restoring previous memory policy: 0 > > EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)' > > EAL: request: mp_malloc_sync > > EAL: Heap on socket 1 was expanded by 5120MB > > FATAL: epp_init.c::copy_mac_addr:130: Call to > > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, &port_id) failed: -19 > > (Unknown error -19), rte_errno=0 (not set) > > > > Caught signal 6 > > Obtained 7 stack frames, tid=713. > > tid=713, /usr/local/bin/ericsson-packet-processor() [0x40a4a4] > > tid=713, /lib64/libpthread.so.0(+0x13f80) [0x7f7e1eae8f80] tid=713, > > /lib64/libc.so.6(gsignal+0x10b) [0x7f7e1c5f818b] tid=713, > > /lib64/libc.so.6(abort+0x175) [0x7f7e1c5f9585] tid=713, > > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] > > tid=713, /lib64/libc.so.6(__libc_start_main+0xed) [0x7f7e1c5e334d] > > tid=713, /usr/local/bin/ericsson-packet-processor(_start+0x2a) > > [0x4091ca] > > > > < i40e log > > > cat epp.log > > MIDHAUL_PCI_ADDR::3b:0d.5, BACKHAUL_PCI_ADDR::3b:0d.4 > > MIDHAUL_IP_ADDR:10.51.21.112, BACKHAUL_IP_ADDR:10.51.21.113 > > EAL: Trying to obtain current memory p
Re: I need DPDK MLX5 Probe error support
05/10/2021 03:17, Jaeeun Ham: > Hi Thomas, > > I attached the testpmd result which is gathered on the host sever. > Could you please take a look at the mlx5_core PCI issue? I see no real issue in the log. For doing more tests, I recommend using the latest DPDK version. > Thank you in advance. > > BR/Jaeeun > > -Original Message- > From: Thomas Monjalon > Sent: Sunday, October 3, 2021 4:51 PM > To: Jaeeun Ham > Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com > Subject: Re: I need DPDK MLX5 Probe error support > > Hi, > > I think you need to read the documentation. > For DPDK install on Linux: > https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d8a30ca42b-d871f122b4a0a61a&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fbuild_dpdk.html%23compiling-and-installing-dpdk-system-wide > For mlx5 specific dependencies, install rdma-core package: > https://protect2.fireeye.com/v1/url?k=9bce4984-c45570e5-9bce091f-86d8a30ca42b-25bd3d467b5f290d&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23linux-prerequisites > > > 02/10/2021 12:57, Jaeeun Ham: > > Hi, > > > > Could you teach me how to install dpdk-testpmd? > > I have to run the application on the host server, not a development server. > > So, I don't know how to get dpdk-testpmd. > > > > By the way, testpmd run result is as below. > > root@seroics05590:~/ejaeham# testpmd > > EAL: Detected 64 lcore(s) > > EAL: libmlx4.so.1: cannot open shared object file: No such file or > > directory > > EAL: FATAL: Cannot init plugins > > > > EAL: Cannot init plugins > > > > PANIC in main(): > > Cannot init EAL > > 5: [testpmd(_start+0x2a) [0x55d301d98e1a]] > > 4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) > > [0x7f5e044a4bf7]] > > 3: [testpmd(main+0x907) [0x55d301d98d07]] > > 2: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(__rte_panic+0xbd) > > [0x7f5e04ca3cfd]] > > 1: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(rte_dump_stack+0x2e) > > [0x7f5e04cac19e]] Aborted > > > > > > I added option below when the process is starting in the docker. > > dv_flow_en=0 \ > > --log-level=pmd,8 \ > > < MLX5 log > > > 415a695ba348:/tmp/logs # cat epp.log > > MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1 > > MIDHAUL_IP_ADDR:10.255.21.177, BACKHAUL_IP_ADDR:10.255.21.178 > > mlx5_pci: unable to recognize master/representors on the multiple IB > > devices > > common_mlx5: Failed to load driver = mlx5_pci. > > > > EAL: Requested device :12:01.0 cannot be used > > mlx5_pci: unable to recognize master/representors on the multiple IB > > devices > > common_mlx5: Failed to load driver = mlx5_pci. > > > > EAL: Requested device :12:01.1 cannot be used > > EAL: Bus (pci) probe failed. > > EAL: Trying to obtain current memory policy. > > EAL: Setting policy MPOL_PREFERRED for socket 1 Caught signal 15 > > EAL: Restoring previous memory policy: 0 > > EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)' > > EAL: request: mp_malloc_sync > > EAL: Heap on socket 1 was expanded by 5120MB > > FATAL: epp_init.c::copy_mac_addr:130: Call to > > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, &port_id) failed: -19 > > (Unknown error -19), rte_errno=0 (not set) > > > > Caught signal 6 > > Obtained 7 stack frames, tid=713. > > tid=713, /usr/local/bin/ericsson-packet-processor() [0x40a4a4] > > tid=713, /lib64/libpthread.so.0(+0x13f80) [0x7f7e1eae8f80] tid=713, > > /lib64/libc.so.6(gsignal+0x10b) [0x7f7e1c5f818b] tid=713, > > /lib64/libc.so.6(abort+0x175) [0x7f7e1c5f9585] tid=713, > > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] > > tid=713, /lib64/libc.so.6(__libc_start_main+0xed) [0x7f7e1c5e334d] > > tid=713, /usr/local/bin/ericsson-packet-processor(_start+0x2a) > > [0x4091ca] > > > > < i40e log > > > cat epp.log > > MIDHAUL_PCI_ADDR::3b:0d.5, BACKHAUL_PCI_ADDR::3b:0d.4 > > MIDHAUL_IP_ADDR:10.51.21.112, BACKHAUL_IP_ADDR:10.51.21.113 > > EAL: Trying to obtain current memory policy. > > EAL: Setting policy MPOL_PREFERRED for socket 1 > > EAL: Restoring previous memory policy: 0 > > EAL: Calling mem event callback 'vfio_mem_event_clb:(nil)' > > EAL: request: mp_malloc_sync > > EAL: Heap on socket 1 was expanded by 5120MB > > i40evf_dev_alarm_handler(): ICR01_ADMINQ is
RE: I need DPDK MLX5 Probe error support
Hi Thomas, Thank you so much for your sincere support. I will follow your suggestion and do my best to solve this issue. By the way, is it okay to use mlx5_core driver by different applications which have different DPDK versions? :12:01.0 (DPDK 20.11 - mlx5_pci: unable to recognize master/representors on the multiple IB) :12:01.1 (DPDK 20.11 - mlx5_pci: unable to recognize master/representors on the multiple IB) :12:01.2 (DPDK 18.11 - currently used) root@seroics05111:~# ethtool -i enp18s1 driver: mlx5_core version: 5.3-1.0.0 firmware-version: 14.27.4000 (HP_2420110034) expansion-rom-version: bus-info: :12:01.0 supports-statistics: yes supports-test: yes supports-eeprom-access: no supports-register-dump: no supports-priv-flags: yes root@seroics05111:~# ethtool -i enp18s1f1 driver: mlx5_core version: 5.3-1.0.0 firmware-version: 14.27.4000 (HP_2420110034) expansion-rom-version: bus-info: :12:01.1 supports-statistics: yes supports-test: yes supports-eeprom-access: no supports-register-dump: no supports-priv-flags: yes root@seroics05111:~# ethtool -i enp18s1f2 driver: mlx5_core version: 5.3-1.0.0 firmware-version: 14.27.4000 (HP_2420110034) expansion-rom-version: bus-info: :12:01.2 supports-statistics: yes supports-test: yes supports-eeprom-access: no supports-register-dump: no supports-priv-flags: yes BR/Jaeeun -Original Message- From: Thomas Monjalon Sent: Sunday, October 3, 2021 4:51 PM To: Jaeeun Ham Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com Subject: Re: I need DPDK MLX5 Probe error support Hi, I think you need to read the documentation. For DPDK install on Linux: https://protect2.fireeye.com/v1/url?k=7925aba3-26be92c2-7925eb38-86d8a30ca42b-d871f122b4a0a61a&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fbuild_dpdk.html%23compiling-and-installing-dpdk-system-wide For mlx5 specific dependencies, install rdma-core package: https://protect2.fireeye.com/v1/url?k=9bce4984-c45570e5-9bce091f-86d8a30ca42b-25bd3d467b5f290d&q=1&e=88eca0f4-aa71-4ba8-a332-179f08406da3&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23linux-prerequisites 02/10/2021 12:57, Jaeeun Ham: > Hi, > > Could you teach me how to install dpdk-testpmd? > I have to run the application on the host server, not a development server. > So, I don't know how to get dpdk-testpmd. > > By the way, testpmd run result is as below. > root@seroics05590:~/ejaeham# testpmd > EAL: Detected 64 lcore(s) > EAL: libmlx4.so.1: cannot open shared object file: No such file or > directory > EAL: FATAL: Cannot init plugins > > EAL: Cannot init plugins > > PANIC in main(): > Cannot init EAL > 5: [testpmd(_start+0x2a) [0x55d301d98e1a]] > 4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) > [0x7f5e044a4bf7]] > 3: [testpmd(main+0x907) [0x55d301d98d07]] > 2: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(__rte_panic+0xbd) > [0x7f5e04ca3cfd]] > 1: [/usr/lib/x86_64-linux-gnu/librte_eal.so.17.11(rte_dump_stack+0x2e) > [0x7f5e04cac19e]] Aborted > > > I added option below when the process is starting in the docker. > dv_flow_en=0 \ > --log-level=pmd,8 \ > < MLX5 log > > 415a695ba348:/tmp/logs # cat epp.log > MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1 > MIDHAUL_IP_ADDR:10.255.21.177, BACKHAUL_IP_ADDR:10.255.21.178 > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.0 cannot be used > mlx5_pci: unable to recognize master/representors on the multiple IB > devices > common_mlx5: Failed to load driver = mlx5_pci. > > EAL: Requested device :12:01.1 cannot be used > EAL: Bus (pci) probe failed. > EAL: Trying to obtain current memory policy. > EAL: Setting policy MPOL_PREFERRED for socket 1 Caught signal 15 > EAL: Restoring previous memory policy: 0 > EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)' > EAL: request: mp_malloc_sync > EAL: Heap on socket 1 was expanded by 5120MB > FATAL: epp_init.c::copy_mac_addr:130: Call to > rte_eth_dev_get_port_by_name(src_dpdk_dev_name, &port_id) failed: -19 > (Unknown error -19), rte_errno=0 (not set) > > Caught signal 6 > Obtained 7 stack frames, tid=713. > tid=713, /usr/local/bin/ericsson-packet-processor() [0x40a4a4] > tid=713, /lib64/libpthread.so.0(+0x13f80) [0x7f7e1eae8f80] tid=713, > /lib64/libc.so.6(gsignal+0x10b) [0x7f7e1c5f818b] tid=713, > /lib64/libc.so.6(abort+0x175) [0x7f7e1c5f9585] tid=713, > /usr/local/bin/ericsson-packet-processor(main+0x458) [0x406818] > tid=713, /lib64/libc.so.6(__libc_start_main+0xed) [0x7f7e1c5e334d] > tid=713, /usr/local/bin/ericsson-packet-processor(_start
RE: I need DPDK MLX5 Probe error support
15
i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
i40evf_handle_aq_msg(): adminq response is received, opcode = 15
i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
i40evf_handle_aq_msg(): adminq response is received, opcode = 15
process start option which is triggered by shell script is as below.
< start-epp.sh >
exec /usr/local/bin/ericsson-packet-processor \
$(get_dpdk_core_list_parameter) \
$(get_dpdk_mem_parameter) \
$(get_dpdk_hugepage_parameters) \
-d /usr/local/lib/librte_mempool_ring.so \
-d /usr/local/lib/librte_mempool_stack.so \
-d /usr/local/lib/librte_net_pcap.so \
-d /usr/local/lib/librte_net_i40e.so \
-d /usr/local/lib/librte_net_mlx5.so \
-d /usr/local/lib/librte_event_dsw.so \
$DPDK_PCI_OPTIONS \
--vdev=event_dsw0 \
--vdev=eth_pcap0,iface=midhaul_edk \
--vdev=eth_pcap1,iface=backhaul_edk \
--file-prefix=container \
--log-level lib.eal:debug \
dv_flow_en=0 \
--log-level=pmd,8 \
-- \
$(get_epp_mempool_parameter) \
"--neighbor-discovery-interface=midhaul_ker,${MIDHAUL_IP_ADDR},mac_addr_dev=${MIDHAUL_MAC_ADDR_DEV},vr_id=0"
\
"--neighbor-discovery-interface=backhaul_ker,${BACKHAUL_IP_ADDR},mac_addr_dev=${BACKHAUL_MAC_ADDR_DEV},vr_id=1"
BR/Jaeeun
-Original Message-
From: Thomas Monjalon
Sent: Wednesday, September 29, 2021 8:16 PM
To: Jaeeun Ham
Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com
Subject: Re: I need DPDK MLX5 Probe error support
27/09/2021 02:18, Jaeeun Ham:
> Hi,
>
> I hope you are well.
> My name is Jaeeun Ham and I have been working for the Ericsson.
>
> I am suffering from enabling MLX5 NIC, so could you take a look at how to run
> it?
> There are two pci address for the SRIOV(vfio) mlx5 nic support but it
> doesn't run correctly. (12:01.0, 12:01.1)
>
> I started one process which is running inside the docker process that is on
> the MLX5 NIC support host server.
> The process started to run with following option.
> -d /usr/local/lib/librte_net_mlx5.so And the docker process has
> mlx5 libraries as below.
Did you try on the host outside of any container?
Please could you try following commands (variables to be replaced)?
dpdk-hugepages.py --reserve 1G
ip link set $netdev netns $container
docker run --cap-add SYS_NICE --cap-add IPC_LOCK --cap-add NET_ADMIN \
--device /dev/infiniband/ $image
echo show port summary all | dpdk-testpmd --in-memory -- -i
> 706a37a35d29:/usr/local/lib # ls -1 | grep mlx librte_common_mlx5.so
> librte_common_mlx5.so.21
> librte_common_mlx5.so.21.0
> librte_net_mlx5.so
> librte_net_mlx5.so.21
> librte_net_mlx5.so.21.0
>
> But I failed to run the process with following error.
> (MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1)
>
> ---
>
> mlx5_pci: unable to recognize master/representors on the multiple IB
> devices
> common_mlx5: Failed to load driver = mlx5_pci.
> EAL: Requested device :12:01.0 cannot be used
> mlx5_pci: unable to recognize master/representors on the multiple IB
> devices
> common_mlx5: Failed to load driver = mlx5_pci.
> EAL: Requested device :12:01.1 cannot be used
> EAL: Bus (pci) probe failed.
>
> ---
>
> For the success case of pci address 12:01.2, it showed following messages.
>
> ---
>
> EAL: Detected 64 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/nah2/mp_socket
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: PCI device :12:01.2 on NUMA socket 0
> EAL: probe driver: 15b3:1016 net_mlx5
> net_mlx5: MPLS over GRE/UDP tunnel offloading disabled due to old
> OFED/rdma-core version or firmware configuration
> net_mlx5: port 0 the requested maximum Rx packet size (2056) is larger
> than a single mbuf (2048) and scattered mode has not been requested
> USER1: rte_ip_frag_table_create: allocated of 6291584 bytes at socket
> 0
>
> ---
>
> BR/Jaeeun
Re: I need DPDK MLX5 Probe error support
03/10/2021 10:10, Jaeeun Ham: > Hi Thomas, > > Thank you so much for your sincere support. > I will follow your suggestion and do my best to solve this issue. > > By the way, is it okay to use mlx5_core driver by different applications > which have different DPDK versions? > :12:01.0 (DPDK 20.11 - mlx5_pci: unable to recognize master/representors > on the multiple IB) > :12:01.1 (DPDK 20.11 - mlx5_pci: unable to recognize master/representors > on the multiple IB) > :12:01.2 (DPDK 18.11 - currently used) I think it should be OK but it is not well tested.
Re: I need DPDK MLX5 Probe error support
_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> Caught signal 10
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
> i40evf_dev_alarm_handler(): ICR01_ADMINQ is reported
> i40evf_handle_aq_msg(): adminq response is received, opcode = 15
>
>
> process start option which is triggered by shell script is as below.
>
> < start-epp.sh >
> exec /usr/local/bin/ericsson-packet-processor \
> $(get_dpdk_core_list_parameter) \
> $(get_dpdk_mem_parameter) \
> $(get_dpdk_hugepage_parameters) \
> -d /usr/local/lib/librte_mempool_ring.so \
> -d /usr/local/lib/librte_mempool_stack.so \
> -d /usr/local/lib/librte_net_pcap.so \
> -d /usr/local/lib/librte_net_i40e.so \
> -d /usr/local/lib/librte_net_mlx5.so \
> -d /usr/local/lib/librte_event_dsw.so \
> $DPDK_PCI_OPTIONS \
> --vdev=event_dsw0 \
> --vdev=eth_pcap0,iface=midhaul_edk \
> --vdev=eth_pcap1,iface=backhaul_edk \
> --file-prefix=container \
> --log-level lib.eal:debug \
> dv_flow_en=0 \
> --log-level=pmd,8 \
> -- \
> $(get_epp_mempool_parameter) \
>
> "--neighbor-discovery-interface=midhaul_ker,${MIDHAUL_IP_ADDR},mac_addr_dev=${MIDHAUL_MAC_ADDR_DEV},vr_id=0"
> \
>
> "--neighbor-discovery-interface=backhaul_ker,${BACKHAUL_IP_ADDR},mac_addr_dev=${BACKHAUL_MAC_ADDR_DEV},vr_id=1"
>
> BR/Jaeeun
>
> -Original Message-
> From: Thomas Monjalon
> Sent: Wednesday, September 29, 2021 8:16 PM
> To: Jaeeun Ham
> Cc: users@dpdk.org; alia...@nvidia.com; rasl...@nvidia.com; as...@nvidia.com
> Subject: Re: I need DPDK MLX5 Probe error support
>
> 27/09/2021 02:18, Jaeeun Ham:
> > Hi,
> >
> > I hope you are well.
> > My name is Jaeeun Ham and I have been working for the Ericsson.
> >
> > I am suffering from enabling MLX5 NIC, so could you take a look at how to
> > run it?
> > There are two pci address for the SRIOV(vfio) mlx5 nic support but it
> > doesn't run correctly. (12:01.0, 12:01.1)
> >
> > I started one process which is running inside the docker process that is on
> > the MLX5 NIC support host server.
> > The process started to run with following option.
> > -d /usr/local/lib/librte_net_mlx5.so And the docker process has
> > mlx5 libraries as below.
>
> Did you try on the host outside of any container?
>
> Please could you try following commands (variables to be replaced)?
>
> dpdk-hugepages.py --reserve 1G
> ip link set $netdev netns $container
> docker run --cap-add SYS_NICE --cap-add IPC_LOCK --cap-add NET_ADMIN \
>--device /dev/infiniband/ $image
> echo show port summary all | dpdk-testpmd --in-memory -- -i
>
>
>
> > 706a37a35d29:/usr/local/lib # ls -1 | grep mlx librte_common_mlx5.so
> > librte_common_mlx5.so.21
> > librte_common_mlx5.so.21.0
> > librte_net_mlx5.so
> > librte_net_mlx5.so.21
> > librte_net_mlx5.so.21.0
> >
> > But I failed to run the process with following error.
> > (MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1)
> >
> > ---
> >
> > mlx5_pci: unable to recognize master/representors on the multiple IB
> > devices
> > common_mlx5: Failed to load driver = mlx5_pci.
> > EAL: Requested device :12:01.0 cannot be used
> > mlx5_pci: unable to recognize master/representors on the multiple IB
> > devices
> > common_mlx5: Failed to load driver = mlx5_pci.
> > EAL: Requested device :12:01.1 cannot be used
> > EAL: Bus (pci) probe failed.
> >
> > ---
> >
> > For the success case of pci address 12:01.2, it showed following messages.
> >
> > ---
> >
> > EAL: Detected 64 lcore(s)
> > EAL: Detected 2 NUMA nodes
> > EAL: Multi-process socket /var/run/dpdk/nah2/mp_socket
> > EAL: Probing VFIO support...
> > EAL: VFIO support initialized
> > EAL: PCI device :12:01.2 on NUMA socket 0
> > EAL: probe driver: 15b3:1016 net_mlx5
> > net_mlx5: MPLS over GRE/UDP tunnel offloading disabled due to old
> > OFED/rdma-core version or firmware configuration
> > net_mlx5: port 0 the requested maximum Rx packet size (2056) is larger
> > than a single mbuf (2048) and scattered mode has not been requested
> > USER1: rte_ip_frag_table_create: allocated of 6291584 bytes at socket
> > 0
> >
> > ---
> >
> > BR/Jaeeun
>
Re: I need DPDK MLX5 Probe error support
27/09/2021 02:18, Jaeeun Ham: > Hi, > > I hope you are well. > My name is Jaeeun Ham and I have been working for the Ericsson. > > I am suffering from enabling MLX5 NIC, so could you take a look at how to run > it? > There are two pci address for the SRIOV(vfio) mlx5 nic support but it doesn't > run correctly. (12:01.0, 12:01.1) > > I started one process which is running inside the docker process that is on > the MLX5 NIC support host server. > The process started to run with following option. > -d /usr/local/lib/librte_net_mlx5.so > And the docker process has mlx5 libraries as below. Did you try on the host outside of any container? Please could you try following commands (variables to be replaced)? dpdk-hugepages.py --reserve 1G ip link set $netdev netns $container docker run --cap-add SYS_NICE --cap-add IPC_LOCK --cap-add NET_ADMIN \ --device /dev/infiniband/ $image echo show port summary all | dpdk-testpmd --in-memory -- -i > 706a37a35d29:/usr/local/lib # ls -1 | grep mlx > librte_common_mlx5.so > librte_common_mlx5.so.21 > librte_common_mlx5.so.21.0 > librte_net_mlx5.so > librte_net_mlx5.so.21 > librte_net_mlx5.so.21.0 > > But I failed to run the process with following error. > (MIDHAUL_PCI_ADDR::12:01.0, BACKHAUL_PCI_ADDR::12:01.1) > > --- > > mlx5_pci: unable to recognize master/representors on the multiple IB devices > common_mlx5: Failed to load driver = mlx5_pci. > EAL: Requested device :12:01.0 cannot be used > mlx5_pci: unable to recognize master/representors on the multiple IB devices > common_mlx5: Failed to load driver = mlx5_pci. > EAL: Requested device :12:01.1 cannot be used > EAL: Bus (pci) probe failed. > > --- > > For the success case of pci address 12:01.2, it showed following messages. > > --- > > EAL: Detected 64 lcore(s) > EAL: Detected 2 NUMA nodes > EAL: Multi-process socket /var/run/dpdk/nah2/mp_socket > EAL: Probing VFIO support... > EAL: VFIO support initialized > EAL: PCI device :12:01.2 on NUMA socket 0 > EAL: probe driver: 15b3:1016 net_mlx5 > net_mlx5: MPLS over GRE/UDP tunnel offloading disabled due to old > OFED/rdma-core version or firmware configuration > net_mlx5: port 0 the requested maximum Rx packet size (2056) is larger than a > single mbuf (2048) and scattered mode has not been requested > USER1: rte_ip_frag_table_create: allocated of 6291584 bytes at socket 0 > > --- > > BR/Jaeeun
