Hi Damjan, I am trying to build VPP on Mellanox Bluefield with Ubuntu 18.04 (4.18.0-1001-mellanox) without installing MOFED.
I usually build with $ make DPDK_MLX5_PMD=y build-release to build Mellanox PMDs. What changes if I want to use rdma-core ? I don’t need to build with DPDK with Mellanox PMDs ? If I just give the PCIe address it should work ? One thing is building VPP with DPDK Mellanox PMD is breaking in this scenario (I wanted to measure the performance difference between using rdma-core vs DPDK PMD): $ make DPDK_MLX5_PMD=y build-release <<DPDK builds fine like Stephen mentioned>> Build complete [arm64-armv8a-linuxapp-gcc] ================== Installing /home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/ make[7]: Nothing to be done for 'install-kmod'. Installation in /home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/ complete make[4]: Leaving directory '/home/sirdas/code/commita/vpp/build-root/build-vpp-native/external/dpdk-19.02' make[3]: Leaving directory '/home/sirdas/code/commita/vpp/build/external' make[2]: Leaving directory '/home/sirdas/code/commita/vpp/build/external' @@@@ Configuring vpp in /home/sirdas/code/commita/vpp/build-root/build-vpp-native/vpp @@@@ -- ARM AArch64 CPU implementer 0x41 part 0xd08 cacheline size 64 -- Looking for ccache -- Looking for ccache - found -- Looking for libuuid -- Found uuid in /usr/include -- dpdk plugin needs libdpdk.a library - found at /home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/lib/libdpdk.a -- Found DPDK 19.2.0 in /home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/include/dpdk -- dpdk plugin needs numa library - found at /usr/lib/aarch64-linux-gnu/libnuma.so -- dpdk plugin needs ibverbs library - found at /home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/lib/libibverbs.a -- dpdk plugin needs mlx5 library - found at /home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/lib/libmlx5.a -- DPDK depends on libmnl, libibverbs, libmlx5 (Mellanox PMD requirement) -- Marvell MUSDK not found - marvell_plugin disabled -- Looking for quicly -- Found quicly in /home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/include -- rdma plugin needs libibverbs.a library - found at /home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/lib/libibverbs.a -- rdma plugin needs libmlx5.a library - found at /home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/lib/libmlx5.a -- tlsmbedtls plugin needs mbedtls library - found at /usr/lib/aarch64-linux-gnu/libmbedtls.so -- tlsmbedtls plugin needs mbedx509 library - found at /usr/lib/aarch64-linux-gnu/libmbedx509.so -- tlsmbedtls plugin needs mbedcrypto library - found at /usr/lib/aarch64-linux-gnu/libmbedcrypto.so -- Configuration: VPP version : 19.04-rc0~591-g26dc58bfa VPP library version : 19.04 GIT toplevel dir : /home/sirdas/code/commita/vpp Build type : C flags : -Wno-address-of-packed-member -march=armv8-a+crc -g -O2 -DFORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror Linker flags : -g -O2 -DFORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror -pie Host processor : aarch64 Target processor : aarch64 Prefix path : /opt/vpp/external/aarch64;/home/sirdas/code/commita/vpp/build-root/install-vpp-native/external Install prefix : /home/sirdas/code/commita/vpp/build-root/install-vpp-native/vpp -- Configuring done -- Generating done -- Build files have been written to: /home/sirdas/code/commita/vpp/build-root/build-vpp-native/vpp @@@@ Building vpp in /home/sirdas/code/commita/vpp/build-root/build-vpp-native/vpp @@@@ ninja: error: '/home/sirdas/code/commita/vpp/build-root/install-vpp-native/external/lib/libibverbs.a', needed by 'lib/vpp_plugins/dpdk_plugin.so', missing and no known rule to make it Makefile:695: recipe for target 'vpp-build' failed make[1]: *** [vpp-build] Error 1 make[1]: Leaving directory '/home/sirdas/code/commita/vpp/build-root' Makefile:373: recipe for target 'build-release' failed Thank you Sirshak Das -----Original Message----- From: Damjan Marion <[email protected]> Sent: Thursday, March 28, 2019 1:32 PM To: Stephen Hemminger <[email protected]> Cc: Thomas F Herbert <[email protected]>; Sirshak Das <[email protected]>; Artem Belov <[email protected]>; vpp-dev <[email protected]> Subject: Re: [vpp-dev] Mellanox dependency changes > On 28 Mar 2019, at 19:00, Stephen Hemminger <[email protected]> > wrote: > > On Wed, 27 Mar 2019 12:06:22 -0400 > "Thomas F Herbert" <[email protected]> wrote: > >> +Artem >> >> +vpp-dev >> >> >> On 03/27/2019 11:22 AM, Sirshak Das wrote: >>> >>> Hi Thomas >>> >>> Is this the patch under consideration ? >>> >>> https://gerrit.fd.io/r/#/c/18521/ >>> >>> So will this make mlx4/5 dpdk compilation default ? >>> >>> What about MOFED dependencies (Kernel Modules and all userspace verb >>> libraries) ? >>> >> With regard to CentOS, necessary kmods are loaded when rdma-core rpm >> is loaded, >>> >>> I heard its all now Upstreamed in Kernel but that’s post 4.20, so >>> how would that work for distros with kernel version lower and MOFED >>> uninstalled (ubuntu 18.04). >>> >> I don't know the solution for Ubuntu 1604. There is no OFED required >> for CentOS7.6. >> >> CentOS7.6: >> >> This is the good news resolved for CentOS 7.6 and I have verified >> for >> 19.01 with patch above on CNCF packet.net infrastructure. >> >> Ubuntu1604: >> >> For Ubuntu, 1604 there is still have a problem: >> >> https://jenkins.fd.io/job/vpp-verify-master-ubuntu1604/18353/console >> ... >> >> *21:27:39* >> /w/workspace/vpp-verify-master-ubuntu1604/build/external/deb/_build/d >> pdk-19.02/drivers/net/mlx5/mlx5_mr.h:19:31: fatal error: >> infiniband/mlx5dv.h: No such file or directory >> *21:27:39* compilation terminated. >> >> Artem's patch, https://gerrit.fd.io/r/#/c/18199/ Also fails for >> Ubuntu1604 >> >> Ubuntu1804: >> >> Seems to be buiding: >> https://jenkins.fd.io/job/vpp-beta-verify-master-ubuntu1804/6150/ > > > In DPDK 18.05 or later, the Mellanox code changed from requiring out > of tree OFED drivers; to using upstream kernel drivers. Since VPP use > 18.11 or 19.02 there is no need for OFED. Distributions have their own > timeline. BTW we are in process of crafting native rdma-core driver to avoid use of DPDK as additional unnecesary layer.
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12673): https://lists.fd.io/g/vpp-dev/message/12673 Mute This Topic: https://lists.fd.io/mt/30795618/21656 Group Owner: [email protected] Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
