Package: openmpi-bin Version: 4.0.3-8 Severity: wishlist Dear Maintainer,
We use the mpiGraph (https://github.com/LLNL/mpiGraph) program to test regularly openmpi basic functionalities on our clusters. For Infiniband networks, and since version 4.0.3 of openmpi available in testing, we use the following command line to run this test: mpirun -v --allow-run-as-root -mca mtl_base_verbose 1 -mca btl_base_verbose 1 -mca btl openib,self -hostfile /root/hostfile --map-by ppr:1:node ./mpiGraph 20000000 50 1 along with the following configuration file (/etc/openmpi/openmpi-mca-params.conf): mtl_ofi_provider_exclude = shm,sockets,tcp,udp,rstream,verbs btl_openib_allow_ib = true We must add the "btl_openib_allow_ib = true" instruction to the configuration, because: "By default, for Open MPI 4.0 and later, infiniband ports on a device are not used by default. The intent is to use UCX for these devices. You can override this policy by setting the btl_openib_allow_ib MCA parameter to true." Mpirun outputs this message when this parameter is not set, and "-mca btl ^openib" is not specified on the command line. (in this case the IB interface is not used, and for one of our cluster, the test runs at Gigabit speed instead of Infiniband speed) As UCX is the recommended solution now, we wonder why ucx is not packaged in the testing distribution (and used for building openmpi). Best regards Patrice -- System Information: Debian Release: bullseye/sid APT prefers testing APT policy: (500, 'testing') Architecture: amd64 (x86_64) Kernel: Linux 5.6.0-2-amd64 (SMP w/32 CPU cores) Locale: LANG=en_US.UTF8, LC_CTYPE=en_US.UTF8 (charmap=UTF-8), LANGUAGE=en_US.UTF8 (charmap=UTF-8) Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages openmpi-bin depends on: ii libc6 2.30-8 ii libevent-2.1-7 2.1.11-stable-1 ii libopenmpi3 4.0.3-8 ii openmpi-common 4.0.3-8 ii openssh-client [ssh-client] 1:8.2p1-4 openmpi-bin recommends no packages. Versions of packages openmpi-bin suggests: ii gfortran [fortran-compiler] 4:9.2.1-3.1 -- Configuration Files: /etc/openmpi/openmpi-mca-params.conf changed: mtl_ofi_provider_exclude = shm,sockets,tcp,udp,rstream,verbs btl_openib_allow_ib = true -- no debconf information
