Hi,
Simply add
btl = tcp,self
If the openib error message persists, try also adding
osc_rdma_btls = ugni,uct,ucp
or simply
osc = ^rdma
Cheers,
Gilles
On 11/29/2022 5:16 PM, Gestió Servidors via users wrote:
Hi,
If I run “mpirun --mca btl tcp,self --mca allow_ib 0 -n 12
./my_program”, I get to disable some “extra” info in the output file like:
The OpenFabrics (openib) BTL failed to initialize while trying to
allocate some locked memory. This typically can indicate that the
memlock limits are set too low. For most HPC installations, the
memlock limits should be set to "unlimited". The failure occured
here:
Local host: clus11
OMPI source: btl_openib.c:757
Function: opal_free_list_init()
Device: qib0
Memlock limit: 65536
You may need to consult with your system administrator to get this
problem fixed. This FAQ entry on the Open MPI web site may also be
helpful:
http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
--------------------------------------------------------------------------
[clus11][[33029,1],0][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],1][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],9][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],8][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],2][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],6][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],10][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],11][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],5][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],3][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],4][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
[clus11][[33029,1],7][btl_openib.c:1062:mca_btl_openib_add_procs]
could not prepare openib device for use
or like
By default, for Open MPI 4.0 and later, infiniband ports on a device
are not used by default. The intent is to use UCX for these devices.
You can override this policy by setting the btl_openib_allow_ib MCA
parameter
to true.
Local host: clus11
Local adapter: qib0
Local port: 1
--------------------------------------------------------------------------
--------------------------------------------------------------------------
WARNING: There was an error initializing an OpenFabrics device.
Local host: clus11
Local device: qib0
--------------------------------------------------------------------------
so, now, I would like to force that parameters in file
$OMPI/etc/openmpi-mca-params.conf. I have run “ompi_info --param all
all --level 9” to get all parameters, but I don’t know exactly what
parameters I need to add to $OMPI/etc/openmpi-mca-params.conf and what
is the correcty syntax of them to force always “--mca btl tcp,self
--mca allow_ib 0”. I have already added “btl_openib_allow_ib = “ and
it works, but for parametes “--mca btl tcp,self”, what would be the
correct syntax in $OMPI/etc/openmpi-mca-params.conf file?
Thanks!!
