Hi Mohammed I verified that this patch fixes it.
Thanks a lot Julius On Tue, 20 Apr 2021 at 10:25, Mohammed Hawari <moham...@hawari.fr> wrote: > Hi Julius, > > Thanks for finding that issue on master, could you verify that this patch > fixes it? https://gerrit.fd.io/r/c/vpp/+/32042 > > Best regards, > > Mohammed > > On 19 Apr 2021, at 09:08, Július Milan <jmilan....@gmail.com> wrote: > > Hi Mohammed > > Thanks a lot, I confirm that it worked well on branch stable/2101. > > However on master (ae34872077c956aa34ee816b55ccb5c5f6ab40a1), I got the > following error: > $ make run > /home/pantheontech/dev/vpp_master/build-root/install-vpp_debug-native/vpp/bin/vpp: > symbol lookup error: > /home/pantheontech/dev/vpp_master/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/dpdk_plugin.so: > undefined symbol: ibv_fork_init > > Following the same procedure as for stable/2101, compiling from the clean > workspace, having the only changes: > $ git diff > diff --git a/build/external/packages/dpdk.mk b/build/external/packages/ > dpdk.mk > index c7eb0fc3f..f1663a1aa 100644 > --- a/build/external/packages/dpdk.mk > +++ b/build/external/packages/dpdk.mk > @@ -14,9 +14,9 @@ > DPDK_PKTMBUF_HEADROOM ?= 128 > DPDK_USE_LIBBSD ?= n > DPDK_DEBUG ?= n > -DPDK_MLX4_PMD ?= n > -DPDK_MLX5_PMD ?= n > -DPDK_MLX5_COMMON_PMD ?= n > +DPDK_MLX4_PMD := y > +DPDK_MLX5_PMD := y > +DPDK_MLX5_COMMON_PMD := y > DPDK_TAP_PMD ?= n > DPDK_FAILSAFE_PMD ?= n > DPDK_MACHINE ?= default > > Best regards > Julius > > On Fri, 16 Apr 2021 at 17:27, Mohammed Hawari <moham...@hawari.fr> wrote: > >> Hi Julius, >> >> You don’t need any patch for Ubuntu it should work out of the box on >> 21.01… This patch is CentOS specific because the default libdir on CentOS >> is ‘lib64' (while it is ‘lib' on Ubuntu/Debian). On Ubuntu, you only need >> to edit dpdk.mk to enable MLX4_PMD, MLX5_PMD and MLX5_COMMON_PMD. >> Internally it works like this: >> - rdma-core (and the libibverbs *static* library) is built and installed >> in some internal staging directory >> (build-root/install-vpp-native/external/lib/). >> - DPDK is configured to be statically linked against this libibverb >> statis library, and is also statically built. >> - then the VPP dpdk plugin is statically linked against DPDK. >> >> The patch fixes the first step, as on CentOS, cmake will install the >> rdma-core libraries in build-root/install-vpp-native/external/lib64 while >> the DPDK build system will look for them in >> build-root/install-vpp-native/external/lib. Note that this patch is not >> “specific” to CentOS, but is *required* for CentOS support (and is merged >> in master). >> >> Best regards, >> Mohammed >> >> On 16 Apr 2021, at 16:55, Július Milan <jmilan....@gmail.com> wrote: >> >> Hi Mohammed >> >> I am trying to create the patch similar to yours: >> https://gerrit.fd.io/r/c/vpp/+/31876 >> But being able to run on Ubuntu. >> Can you point me a bit? I would like to understand how your patch is >> CentOS specific. >> Probably it has something to do with linking of libibverbs.so into >> dpdk_plugin, which I was not able to achieve yet. >> Or am I completely wrong? >> >> Best regards >> Julius >> >> On Fri, 16 Apr 2021 at 09:49, Mohammed Hawari <moham...@hawari.fr> wrote: >> >>> >>> Hi Chetan, >>> >>> On 16 Apr 2021, at 07:04, chetan bhasin <chetan.bhasin...@gmail.com> >>> wrote: >>> >>> Hi Mohammed , >>> >>> After applying the patch you have mentioned , I am no longer facing the >>> compilation issue. >>> >>> Thanks for your support. >>> >>> With Static RDMA linking , do we need to do anything extra while >>> bringing our application on the server having Mellanox Nic ? >>> >>> >>> No we don’t, it should work out of the box! >>> >>> >>> Thanks & Regards, >>> >>> Chetan >>> >>> >>> Regards >>> >>> Mohammed >>> >>> >>> On Wed, Apr 14, 2021 at 8:33 PM chetan bhasin via lists.fd.io >>> <chetan.bhasin017=gmail....@lists.fd.io> wrote: >>> >>>> Thanks a lot Mohammed! >>>> >>>> Let me try with the rdma-core and patch mentioned by you and >>>> >>>> Thanks, >>>> Chetan >>>> >>>> On Wed, Apr 14, 2021 at 6:20 PM Mohammed Hawari <moham...@hawari.fr> >>>> wrote: >>>> >>>>> Hi Chetan >>>>> >>>>> You are building DPDK with Mellanox support relying on dlopen and the >>>>> glue library which is fine. The default approach implemented in the VPP >>>>> build system consists of statically linking DPDK with the the libraries >>>>> provided by rdma-core (as it’s also built by the VPP build system). The >>>>> default approach should work fine, provided this patch >>>>> https://gerrit.fd.io/r/c/vpp/+/31876 is applied (specific to CentOS). >>>>> But what you should work just fine. Now the reason why you need to >>>>> comment >>>>> dpdk_depends is kind of mysterious here… >>>>> >>>>> Regards, >>>>> >>>>> Mohammed >>>>> >>>>> On 14 Apr 2021, at 10:07, chetan bhasin <chetan.bhasin...@gmail.com> >>>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> I have to do the following to enable Mellanox compilation under >>>>> VPP21.01 with dpdk 20.11. >>>>> >>>>> If I dont comment rdma-core dependencies , it will lead to undefined >>>>> symbols . >>>>> >>>>> Can anybody please correct , is this the right way to do it? >>>>> >>>>> -DPDK_MLX4_PMD ?= n >>>>> -DPDK_MLX5_PMD ?= n >>>>> -DPDK_MLX5_COMMON_PMD ?= n >>>>> *+DPDK_MLX4_PMD := y* >>>>> *+DPDK_MLX5_PMD := y* >>>>> *+DPDK_MLX5_COMMON_PMD = y* >>>>> >>>>> -DPDK_MLX_IBV_LINK ?= static >>>>> *+DPDK_MLX_IBV_LINK := dlopen* >>>>> >>>>> -dpdk_depends := rdma-core $(if $(ARCH_X86_64), >>>>> ipsec-mb) >>>>> *+#dpdk_depends := rdma-core $(if $(ARCH_X86_64), >>>>> ipsec-mb)* >>>>> >>>>> Thanks, >>>>> Chetan >>>>> >>>>> On Tue, Apr 6, 2021 at 4:10 PM chetan bhasin via lists.fd.io >>>>> <chetan.bhasin017=gmail....@lists.fd.io> wrote: >>>>> >>>>>> Thanks Guys ! >>>>>> >>>>>> We have a requirement to compile vppp with DPDK mellanox driver . >>>>>> >>>>>> Right now issue what i understood is , when we compile vpp_2101 , >>>>>> using make build-release , it builds mellanox , but when it is trying to >>>>>> install vpp-ext-dep rpm , it got failed because glue library created of >>>>>> mellanox has some undefined symbols , >>>>>> For your reference , i have set following in dpdk.mk file >>>>>> >>>>>> -DPDK_MLX4_PMD ?= n >>>>>> -DPDK_MLX5_PMD ?= n >>>>>> +DPDK_MLX4_PMD := y >>>>>> +DPDK_MLX5_PMD := y >>>>>> -DPDK_MLX_IBV_LINK ?= static >>>>>> +DPDK_MLX_IBV_LINK := shared >>>>>> >>>>>> please check below logs for reference- >>>>>> >>>>>> FAILED: drivers/net/mlx4/librte_net_mlx4_glue.so.21.0 >>>>>> cc -o drivers/net/mlx4/librte_net_mlx4_glue.so.21.0 >>>>>> 'drivers/net/mlx4/8672f8e@@rte_net_mlx4_glue@sha/mlx4_glue.c.o' >>>>>> -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC >>>>>> -Wl,--start-group >>>>>> -Wl,-soname,librte_net_mlx4_glue.so.21.0 -Wl,--no-as-needed -pthread -lm >>>>>> -ldl -lnuma -Wl,-export-dynamic -Wl,-h,librte_net_mlx4_glue.so.21.0 >>>>>> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libmlx4.so >>>>>> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libibverbs.so >>>>>> -Wl,--end-group >>>>>> drivers/net/mlx4/8672f8e@@rte_net_mlx4_glue@sha/mlx4_glue.c.o: In >>>>>> function `mlx4_glue_reg_mr': >>>>>> mlx4_glue.c:(.text+0x277): undefined reference to `ibv_reg_mr_iova2' >>>>>> collect2: error: ld returned 1 exit status >>>>>> [8/547] Compiling C object 'drivers/a715181@@tmp_rte_net_mlx4@sta >>>>>> /net_mlx4_mlx4_flow.c.o' >>>>>> >>>>>> ninja: build stopped: subcommand failed. >>>>>> >>>>>> On Thu, Apr 1, 2021 at 4:33 PM Юрий Иванов <format_...@outlook.com> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Or you can use nativa rdma driver as written in this post and build >>>>>>> vpp as usual. 😉 >>>>>>> >>>>>>> Regards. >>>>>>> ------------------------------ >>>>>>> *От:* vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> от имени Mohammed >>>>>>> Hawari <moham...@hawari.fr> >>>>>>> *Отправлено:* 1 апреля 2021 г. 13:16 >>>>>>> *Кому:* chetan bhasin <chetan.bhasin...@gmail.com> >>>>>>> *Копия:* vpp-dev <vpp-dev@lists.fd.io> >>>>>>> *Тема:* Re: [vpp-dev] How to enable Mellanox compilation in VPP >>>>>>> 21.01 >>>>>>> >>>>>>> >>>>>>> Hi Chetan, >>>>>>> >>>>>>> If you are using CentOS, I’d suggest to cherry-pick >>>>>>> https://gerrit.fd.io/r/c/vpp/+/31876. Also please change the dpdk.mk >>>>>>> to also set DPDK_MLX5_COMMON_PMD = y. I hope this solves your >>>>>>> issue. Otherwise, please consider using the rdma native driver that does >>>>>>> not rely on DPDK. >>>>>>> >>>>>>> Regards >>>>>>> >>>>>>> Mohammed >>>>>>> >>>>>>> On 30 Mar 2021, at 12:41, chetan bhasin <chetan.bhasin...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>> Hello Everyone, >>>>>>> >>>>>>> I am upgrading to vpp 2101 . I am facing a compilation issue after >>>>>>> enabling Mellanox compilation in dpdk.mk . >>>>>>> >>>>>>> --- a/build/external/packages/dpdk.mk >>>>>>> +++ b/build/external/packages/dpdk.mk >>>>>>> @@ -14,8 +14,8 @@ >>>>>>> DPDK_PKTMBUF_HEADROOM ?= 128 >>>>>>> DPDK_USE_LIBBSD ?= n >>>>>>> DPDK_DEBUG ?= n >>>>>>> *-DPDK_MLX4_PMD ?= n* >>>>>>> *-DPDK_MLX5_PMD ?= n* >>>>>>> *+DPDK_MLX4_PMD ?= y* >>>>>>> *+DPDK_MLX5_PMD ?= y* >>>>>>> DPDK_MLX5_COMMON_PMD ?= n >>>>>>> >>>>>>> >>>>>>> Getting below errors . Anybody please help here ? >>>>>>> >>>>>>> [1344/1846] Compiling C object 'drivers/a715181@@tmp_rte_net_nfp@sta >>>>>>> /net_nfp_nfpcore_nfp_nsp_cmds.c.o' >>>>>>> [1345/1846] Linking target drivers/librte_net_mlx4.so.21.0 >>>>>>> FAILED: drivers/librte_net_mlx4.so.21.0 >>>>>>> cc -o drivers/librte_net_mlx4.so.21.0 'drivers/a715181@ >>>>>>> @rte_net_mlx4@sha/meson-generated_.._rte_net_mlx4.pmd.c.o' >>>>>>> 'drivers/a715181@@tmp_r >>>>>>> te_net_mlx4@sta/net_mlx4_mlx4.c.o' 'drivers/a715181@ >>>>>>> @tmp_rte_net_mlx4@sta/net_mlx4_mlx4_ethdev.c.o' 'drivers/a715181@ >>>>>>> @tmp_rte_net_mlx4@sta >>>>>>> /net_mlx4_mlx4_flow.c.o' >>>>>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_intr.c.o' >>>>>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_ml >>>>>>> x4_mp.c.o' 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_mr.c.o' >>>>>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_rxq.c.o' 'dri >>>>>>> vers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_rxtx.c.o' >>>>>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_txq.c.o' >>>>>>> 'drivers/a715181@ >>>>>>> @tmp_rte_net_mlx4@sta/net_mlx4_mlx4_utils.c.o' 'drivers/a715181@ >>>>>>> @tmp_rte_net_mlx4@sta/net_mlx4_mlx4_glue.c.o' -Wl,--as-needed >>>>>>> -Wl,--no-und >>>>>>> efined -Wl,-O1 -shared -fPIC -Wl,--start-group >>>>>>> -Wl,-soname,librte_net_mlx4.so.21 -Wl,--no-as-needed -pthread -lm -ldl >>>>>>> -lnuma lib/librte_et >>>>>>> hdev.so.21.0 lib/librte_eal.so.21.0 lib/librte_kvargs.so.21.0 >>>>>>> lib/librte_telemetry.so.21.0 lib/librte_net.so.21.0 >>>>>>> lib/librte_mbuf.so.21.0 >>>>>>> lib/librte_mempool.so.21.0 lib/librte_ring.so.21.0 >>>>>>> lib/librte_meter.so.21.0 drivers/librte_bus_pci.so.21.0 >>>>>>> lib/librte_pci.so.21.0 drivers/ >>>>>>> librte_bus_vdev.so.21.0 >>>>>>> -Wl,--version-script=/root/tmp/vpp/build-root/build-vpp-native/external/src-dpdk/drivers/net/mlx4/version.map >>>>>>> -lpt >>>>>>> hread -L/root/tmp/vpp/build-root/install-vpp-native/external/lib64 >>>>>>> -l:libbnxt_re-rdmav25.a -l:libcxgb4-rdmav25.a -l:libefa.a -l:libhns-rdm >>>>>>> av25.a -l:libi40iw-rdmav25.a -l:libmlx4.a -l:libmlx5.a >>>>>>> -l:libmthca-rdmav25.a -l:libocrdma-rdmav25.a -l:libqedr-rdmav25.a >>>>>>> -l:libvmw_pvrdma-rdmav25.a -l:libhfi1verbs-rdmav25.a >>>>>>> -l:libipathverbs-rdmav25.a -l:librxe-rdmav25.a -l:libsiw-rdmav25.a >>>>>>> -l:libibverbs.a -l:librdma_util.a -l:libccan.a -Wl,--end-group >>>>>>> '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' >>>>>>> -Wl,-rpath-link,/root/tmp/vpp/build-root/build-vpp-native/external/build-dpdk/lib >>>>>>> -Wl,-rpath-link,/root/tmp/vpp/build-root/build-vpp-native/external/build-dpdk/drivers >>>>>>> /bin/ld: cannot find -l:libbnxt_re-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libcxgb4-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libefa.a >>>>>>> /bin/ld: cannot find -l:libhns-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libi40iw-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libmlx4.a >>>>>>> /bin/ld: cannot find -l:libmlx5.a >>>>>>> /bin/ld: cannot find -l:libmthca-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libocrdma-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libqedr-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libvmw_pvrdma-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libhfi1verbs-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libipathverbs-rdmav25.a >>>>>>> /bin/ld: cannot find -l:librxe-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libsiw-rdmav25.a >>>>>>> /bin/ld: cannot find -l:libibverbs.a >>>>>>> /bin/ld: cannot find -l:librdma_util.a >>>>>>> /bin/ld: cannot find -l:libccan.a >>>>>>> collect2: error: ld returned 1 exit status >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>> >>> >>> >>> >>> >>> >>> >> >> > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19266): https://lists.fd.io/g/vpp-dev/message/19266 Mute This Topic: https://lists.fd.io/mt/81718770/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
