[OMPI users] Announcing Open MPI v5.0.0rc1

2021-09-30 Thread Austen W Lauria via users 
The first release candidate for the Open MPI v5.0.0 release is posted at: https://www.open-mpi.org/software/ompi/v5.0/
 
Major changes include:
v5.0.0rc1 -- September, 2021- ORTE, the underlying OMPI launcher has been removed, and replaced  with PRTE.- Reworked how Open MPI integrates with 3rd party packages.  The decision was made to stop building 3rd-party packages  such as Libevent, HWLOC, PMIx, and PRRTE as MCA components  and instead 1) start relying on external libraries whenever  possible and 2) Open MPI builds the 3rd party libraries (if needed)  as independent libraries, rather than linked into libopen-pal.- Update to use PMIx v4.1.1rc2- Update to use PRRTE v2.0.1rc2- Change the default component build behavior to prefer building  components as part of libmpi.so instead of individual DSOs.- Remove pml/yalla, mxm, mtl/psm, and ikrit components.- Remove all vestiges of the C/R support.- Various ROMIO v3.4.1 updates.- Use Pandoc to generate manpages- 32 bit atomics are now only supported via C11 compliant compilers.- Do not build Open SHMEM layer when there are no SPMLs available.  Currently, this means the Open SHMEM layer will only build if  the UCX library is found.- Fix rank-by algorithms to properly rank by object and span.- Updated the "-mca pml" option to only accept one pml, not a list.- vprotocol/pessimist: Updated to support MPI_THREAD_MULLTIPLE.- btl/tcp: Updated to use reachability and graph solving for global  interface matching. This has been shown to improve MPI_Init()  performance under btl/tcp.- fs/ime: Fixed compilation errors due to missing header inclusion  Thanks to Sylvain Didelot  for finding  and fixing this issue.- Fixed bug where MPI_Init_thread can give wrong error messages by  delaying error reporting until all infrastructure is running.- Atomics support removed: S390/s390x, Sparc v9, ARMv4 and ARMv5 CMA  support.- autogen.pl now supports a "-j" option to run multi-threaded.  Users can also use environment variable "AUTOMAKE_JOBS".- PMI support has been removed for Open MPI apps.- Legacy btl/sm has been removed, and replaced with btl/vader, which  was renamed to "btl/sm".- Update btl/sm to not use CMA in user namespaces.- C++ bindings have been removed.- The "--am" and "--amca" options have been deprecated.- opal/mca/threads framework added. Currently supports  argobots, qthreads, and pthreads. See the --with-threads=x option  in configure.- Various README.md fixes - thanks to:  Yixin Zhang ,  Samuel Cho ,  rlangefe ,  Alex Ross ,  Sophia Fang ,  mitchelltopaloglu ,  Evstrife , and  Hao Tong  for their  contributions.- osc/pt2pt: Removed. Users can use osc/rdma + btl/tcp  for OSC support using TCP, or other providers.- Open MPI now links -levent_core instead of -levent.- MPI-4: Added ERRORS_ABORT infrastructure.- common/cuda docs: Various fixes. Thanks to  Simon Byrne  for finding and fixing.- osc/ucx: Add support for acc_single_intrinsic.- Fixed buildrpm.sh "-r" option used for RPM options specification.  Thanks to John K. McIver III  for  reporting and fixing.- configure: Added support for setting the wrapper C compiler.  Adds new option "--with-wrapper-cc=" .- mpi_f08: Fixed Fortran-8-byte-INTEGER vs. C-4-byte-int issue.  Thanks to @ahaichen for reporting the bug.- MPI-4: Added support for 'initial error handler'.- opal/thread/tsd: Added thread-specific-data (tsd) api.- MPI-4: Added error handling for 'unbound' errors to MPI_COMM_SELF.- Add missing MPI_Status conversion subroutines:  MPI_Status_c2f08(), MPI_Status_f082c(), MPI_Status_f082f(),  MPI_Status_f2f08() and the PMPI_* related subroutines.- patcher: Removed the Linux component.- opal/util: Fixed typo in error string. Thanks to  NARIBAYASHI Akira  for finding  and fixing the bug.- fortran/use-mpi-f08: Generate PMPI bindings from the MPI bindings.- Converted man pages to markdown.  Thanks to Fangcong Yin  for their contribution  to this effort.- Fixed ompi_proc_world error string and some comments in pml/ob1.  Thanks to Julien EMMANUEL  for  finding and fixing these issues.- oshmem/tools/oshmem_info: Fixed Fortran keyword issue when  compiling param.c. Thanks to Pak Lui  for  finding and fixing the bug.- autogen.pl: Patched libtool.m4 for OSX Big Sur. Thanks to  @fxcoudert for reporting the issue.- Updgraded to HWLOC v2.4.0.- Removed config/opal_check_pmi.m4.  Thanks to Zach Osman  for the contribution.- opal/atomics: Added load-linked, store-conditional atomics for  AArch6.- Fixed envvar names to OMPI_MCA_orte_precondition_transports.  Thanks to Marisa Roman   for the contribution.- fcoll/two_phase: Removed the component. All scenerios it was  used for has been replaced.- btl/uct: Bumped UCX allowed version to v1.9.x.- ULFM Fault Tolerance has been added. See README.FT.ULFM.md.- Fixed a crash during CUDA initialization.  Thanks to Yaz Saito  for finding  and fixing the bug.- Added CUDA support to the OFI MTL.- ompio: Added atomicity support.- Singleton comm spawn support has been fixed.-

Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, buildhints?

2021-09-30 Thread Ray Muno via users 
Hmm, trying to keep this all straight. It turns out I had the module loaded for the nv-hpc 
compilers, version 21.7, instead of 21.9 in what I thought was asuccesful build.


The nuance is that with the version 21.9 compiler suite, adding this to 
configure,

--with-platform=/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/contrib/platform/mellanox/optimized

brings back an error in avx.

 CCLD mca_op_avx.la
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0): multiple 
definition of `ompi_op_avx_functions_avx2'

./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
 first defined here
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o): In function 
`ompi_op_avx_2buff_min_uint16_t_avx2':
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651: multiple 
definition of `ompi_op_avx_3buff_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651: 
first defined here

make[2]: *** [mca_op_avx.la] Error 2
make[2]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mca/op/avx'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi'
make: *** [all-recursive] Error 1

Similar, to the error without --enable-mca-no-build=op-avx, but appears to referencing a different 
line in the source file.


  CCLD mca_op_avx.la
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0): multiple 
definition of `ompi_op_avx_functions_avx2'

./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
 first defined here
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o): In function 
`ompi_op_avx_2buff_min_uint16_t_avx2':
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:54: multiple 
definition of `ompi_op_avx_3buff_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:54: 
first defined here

make[2]: *** [mca_op_avx.la] Error 2
make[2]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9-clean/ompi/mca/op/avx'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9-clean/ompi'
make: *** [all-recursive] Error 1

For reference, ompi_info from the two succesful builds.

NV-HPC Version 21.7
+++

ompi_info
 Package: Open MPI muno@loki.local Distribution
Open MPI: 4.1.1
  Open MPI repo revision: v4.1.1
   Open MPI release date: Apr 24, 2021
Open RTE: 4.1.1
  Open RTE repo revision: v4.1.1
   Open RTE release date: Apr 24, 2021
OPAL: 4.1.1
  OPAL repo revision: v4.1.1
   OPAL release date: Apr 24, 2021
 MPI API: 3.1.0
Ident string: 4.1.1
  Prefix: /stage/opt/OpenMPI/ROME/4.1.1/NV-HPC/21.7
 Configured architecture: x86_64-pc-linux-gnu
  Configure host: loki.local
   Configured by: muno
   Configured on: Thu Sep 30 18:12:10 UTC 2021
  Configure host: loki.local
  Configure command line: 'CC=nvc' 'CXX=nvc++' 'FC=nvfortran' 'FCFLAGS=-fPIC'
  '--enable-mca-no-build=op-avx'
  '--prefix=/stage/opt/OpenMPI/ROME/4.1.1/NV-HPC/21.7'
  '--with-libevent=internal'
  '--enable-mpi1-compatibility' '--without-xpmem'
  '--with-pmi' '--enable-mpi-cxx'
  '--with-hwloc=/stage/opt/HWLOC/2.5.0'
  '--with-hcoll=/opt/mellanox/hcoll'
  '--with-knem=/opt/knem-1.1.4.90mlnx1'
  
'--with-cuda=/stage/opt/NV_hpc_sdk/Linux_x86_64/21.9/cuda'

'--with-platform=/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/contrib/platform/mellanox/optimized'
Built by: muno
Built on: Thu Sep 30 18:28:56 UTC 2021
  Built host: loki.local
  C bindings: yes
C++ bindings: yes
 Fort mpif.h: yes (all)
Fort use mpi: yes (full: ignore TKR)
   Fort use mpi size: deprecated-ompi-info-value
Fort use mpi_f08: yes
 Fort mpi_f08 compliance: The mpi_f08 module is available, but due to
  limitations in the nvfortran compiler and/or Open
  MPI, does not support the following: array
  subsections, direct passthru (where possible) to
  underlying Open MPI's C functionality
  Fort mpi_f08 subarrays: no
   Java bindings: no
  Wrapper compiler rpath: runpath
  C compiler: nvc
 C compiler absolute:

Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, buildhints?

2021-09-30 Thread Ray Muno via users 

OK, starting clean.

OS CentOS 7.9  (7.9.2009)
mlnxofed 5.4-1.0.3.0
UCX 1.11.0 (from mlnxofed)
hcoll-4.7.3199 (from mlnxofed)
knem-1.1.4.90  (from mlnxofed)
nVidia HPC-SDK  21.9
OpenMPI 4.1.1
HWLOC 2.5.0

Straight configure

configure CC=nvc CXX=nvc++  FC=nvfortran

dies in

 FCLD libmpi_usempif08.la
/usr/bin/ld: .libs/comm_spawn_multiple_f08.o: relocation R_X86_64_32S against `.rodata' can not be 
used when making a shared object; recompile with -fPIC


configure CC=nvc CXX=nvc++  FC=nvfortran FCFLAGS='-fPIC'

Fixes that, dies in

 CCLD mca_op_avx.la
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0): multiple 
definition of `ompi_op_avx_functions_avx2'

./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
 first defined here

configure CC=nvc CXX=nvc++ FC=nvfortran FCFLAGS=-fPIC 
--enable-mca-no-build=op-avx

succeeds

And, working up to what I really want, I can build  (soemwhat emulating the 
HPC-X 2.9.0 build)

/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/configure CC=nvc CXX=nvc++ FC=nvfortran FCFLAGS=-fPIC 
--enable-mca-no-build=op-avx --prefix=/stage/opt/OpenMPI/ROME/4.1.1/NV-HPC/21.9 
--with-libevent=internal --enable-mpi1-compatibility --without-xpmem --with-pmi --enable-mpi-cxx 
--with-hwloc=/stage/opt/HWLOC/2.5.0 --with-hcoll=/opt/mellanox/hcoll 
--with-knem=/opt/knem-1.1.4.90mlnx1 --with-cuda=/stage/opt/NV_hpc_sdk/Linux_x86_64/21.9/cuda


adding

--with-platform=/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/contrib/platform/mellanox/optimized

brings back

/usr/bin/ld: final link failed: Bad value
make[2]: *** [libmpi_usempif08.la] Error 2

Since it overrides the FCFLAGS command line setting apparently.
Editing the file to add -fPIC to FCLAGS took care of that as did the FC='nvfortran -fPIC' (which is 
kludgey).



-Ray Muno

On 9/30/21 8:13 AM, Gilles Gouaillardet via users wrote:

  Ray,

there is a typo, the configure option is

--enable-mca-no-build=op-avx

Cheers,

Gilles

- Original Message -

  Added -*-enable-mca-no-build=op-avx *to the configure line. Still dies in 
the same place.
history | grep config




CCLD mca_op_avx.la

./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0):
 multiple
definition of `ompi_op_avx_functions_avx2'

./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
 first defined here
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o): 
In function
`ompi_op_avx_2buff_min_uint16_t_avx2':

/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
 multiple
definition of `ompi_op_avx_3buff_functions_avx2'

./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
first defined here
make[2]: *** [mca_op_avx.la] Error 2
make[2]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mca/op/avx'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi'
make: *** [all-recursive] Error 1

On 9/30/21 5:54 AM, Carl Ponder wrote:


For now, you can suppress this error building OpenMPI 4.1.1


./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0):
multiple definition of `ompi_op_avx_functions_avx2'

./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
 first
defined here

./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o): In 
function
`ompi_op_avx_2buff_min_uint16_t_avx2':

/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
multiple definition of `ompi_op_avx_3buff_functions_avx2'

./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:

first defined here 


with the NVHPC/PGI 21.9 compiler by using the setting

configure -*-enable-mca-no-build=op-avx* ...

We're still looking at the cause here. I don't have any advice about the 
problem with 21.7.



Subject:Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, 
build hints?
Date:   Wed, 29 Sep 2021 12:25:43 -0500
From:   Ray Muno via users 
Reply-To:   Open MPI Users 
To: users@lists.open-mpi.org
CC: Ray Muno 



External email: Use caution opening links or attachments


Tried this

configure CC='nvc -fPIC' CXX='nvc++ -fPIC' FC='nvfortran -fPIC'

Configure completes. Compiles quite a way through. Dies in a different 
place. It does get past the
first error, however with libmpi_usempif08.la

Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, build hints?

2021-09-30 Thread Gilles Gouaillardet via users 
Carl,

I opened https://github.com/open-mpi/ompi/issues/9444 to specifically track
the issue related to the op/avx component

TL;DR
nvhpc compilers can compile AVX512 intrinsics (so far so good), but do not
define at least one of these macros
__AVX512BW__
 __AVX512F__
__AVX512VL__
and Open MPI is not happy about it.

If you can have all these 3 macros defined by the nvhpc compilers, that
would be great!
Otherwise, I will let George decide if and how Open MPI addresses this issue

Cheers,

Gilles

On Thu, Sep 30, 2021 at 11:33 PM Carl Ponder via users <
users@lists.open-mpi.org> wrote:

>
> Are you able to fix the problem by adding extern qualifiers?
> We could push this change back to the OpenMPI developers...
>
> --
> Subject: Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk,
> build hints?
> Date: Thu, 30 Sep 2021 08:46:26 -0400
> From: Bennet Fauber  
> To: Carl Ponder  , Open MPI Users
>  
> CC: Ray Muno  
>
> *External email: Use caution opening links or attachments*
> You may be seeing this?
>
> C language issues Default to -fno-common
>
> A common mistake in C is omitting extern when declaring a global variable
> in a header file. If the header is included by several files it results in
> multiple definitions of the same variable. In previous GCC versions this
> error is ignored. GCC 10 defaults to -fno-common, which means a linker
> error will now be reported. To fix this, use extern in header files when
> declaring global variables, and ensure each global is defined in exactly
> one C file. If tentative definitions of particular variables need to be
> placed in a common block, __attribute__((__common__)) can be used to
> force that behavior even in code compiled without -fcommon. As a
> workaround, legacy C code where all tentative definitions should be placed
> into a common block can be compiled with -fcommon.
>
> https://gcc.gnu.org/gcc-10/porting_to.html
> 
>
>
> On Thu, Sep 30, 2021 at 6:56 AM Carl Ponder via users <
> users@lists.open-mpi.org> wrote:
>
>>
>> For now, you can suppress this error building OpenMPI 4.1.1
>>
>> ./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0):
>> multiple definition of `ompi_op_avx_functions_avx2'
>> ./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
>> first defined here
>> ./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
>> In function`ompi_op_avx_2buff_min_uint16_t_avx2':
>> /project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
>> multiple definition of `ompi_op_avx_3buff_functions_avx2'
>>
>> ./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
>> first defined here
>>
>> with the NVHPC/PGI 21.9 compiler by using the setting
>>
>> configure -*-enable-mca-no-build=op-avx* ...
>>
>> We're still looking at the cause here. I don't have any advice about the
>> problem with 21.7.
>>
>> --
>> Subject: Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk,
>> build hints?
>> Date: Wed, 29 Sep 2021 12:25:43 -0500
>> From: Ray Muno via users 
>> 
>> Reply-To: Open MPI Users 
>> 
>> To: users@lists.open-mpi.org
>> CC: Ray Muno  
>>
>> External email: Use caution opening links or attachments
>>
>>
>> Tried this
>>
>> configure CC='nvc -fPIC' CXX='nvc++ -fPIC' FC='nvfortran -fPIC'
>>
>> Configure completes. Compiles quite a way through. Dies in a different
>> place. It does get past the
>> first error, however with libmpi_usempif08.la
>> 
>>
>>
>> FCLD libmpi_usempif08.la
>> 
>> make[2]: Leaving directory
>>
>> `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/use-mpi-f08'
>> Making all in mpi/fortran/mpiext-use-mpi-f08
>> make[2]: Entering directory

Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, build hints?

2021-09-30 Thread Carl Ponder via users 


Are you able to fix the problem by adding extern qualifiers?
We could push this change back to the OpenMPI developers...


Subject: 	Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, 
build hints?

Date:   Thu, 30 Sep 2021 08:46:26 -0400
From:   Bennet Fauber 
To: 	Carl Ponder , Open MPI Users 


CC: Ray Muno 



*External email: Use caution opening links or attachments*


You may be seeing this?


   C language issues


 Default to |-fno-common|

A common mistake in C is omitting |extern| when declaring a global 
variable in a header file. If the header is included by several files it 
results in multiple definitions of the same variable. In previous GCC 
versions this error is ignored. GCC 10 defaults to |-fno-common|, which 
means a linker error will now be reported. To fix this, use |extern| in 
header files when declaring global variables, and ensure each global is 
defined in exactly one C file. If tentative definitions of particular 
variables need to be placed in a common block, 
|__attribute__((__common__))| can be used to force that behavior even in 
code compiled without |-fcommon|. As a workaround, legacy C code where 
all tentative definitions should be placed into a common block can be 
compiled with |-fcommon|.



https://gcc.gnu.org/gcc-10/porting_to.html 




On Thu, Sep 30, 2021 at 6:56 AM Carl Ponder via users 
mailto:users@lists.open-mpi.org>> wrote:



   For now, you can suppress this error building OpenMPI 4.1.1

   
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0):
   multipledefinition of `ompi_op_avx_functions_avx2'
   
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
   first defined here
   ./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
   In function`ompi_op_avx_2buff_min_uint16_t_avx2':
   
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
   multipledefinition of `ompi_op_avx_3buff_functions_avx2'
   
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
   first defined here

   with the NVHPC/PGI 21.9 compiler by using the setting

   configure -*-enable-mca-no-build=op-avx* ...

   We're still looking at the cause here. I don't have any advice about
   the problem with 21.7.

   
   Subject: Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia
   HPC-SDk, build hints?
   Date:Wed, 29 Sep 2021 12:25:43 -0500
   From:Ray Muno via users 
   
   Reply-To:Open MPI Users 
   
   To:  users@lists.open-mpi.org 
   CC:  Ray Muno  



   External email: Use caution opening links or attachments


   Tried this

   configure CC='nvc -fPIC' CXX='nvc++ -fPIC' FC='nvfortran -fPIC'

   Configure completes. Compiles quite a way through. Dies in a
   different place. It does get past the
   first error, however with libmpi_usempif08.la
   



   FCLD libmpi_usempif08.la
   

   make[2]: Leaving directory
   
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/use-mpi-f08'
   Making all in mpi/fortran/mpiext-use-mpi-f08
   make[2]: Entering directory
   
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/mpiext-use-mpi-f08'
   PPFC mpi-f08-ext-module.lo
   FCLD libforce_usempif08_module_to_be_built.la

Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, buildhints?

2021-09-30 Thread Gilles Gouaillardet via users 
 Ray,



there is a typo, the configure option is

--enable-mca-no-build=op-avx



Cheers,



Gilles

- Original Message -

 Added --enable-mca-no-build=op-avx to the configure line. Still dies in 
the same place. 


CCLD mca_op_avx.la
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
(.data+0x0): multiple definition of `ompi_op_avx_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.
data+0x0): first defined here
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
 In function `ompi_op_avx_2buff_min_uint16_t_avx2':
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_
functions.c:651: multiple definition of `ompi_op_avx_3buff_functions_
avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/
project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_
functions.c:651: first defined here
make[2]: *** [mca_op_avx.la] Error 2
make[2]: Leaving directory `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-
HPC/21.9/ompi/mca/op/avx'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-
HPC/21.9/ompi'
make: *** [all-recursive] Error 1


On 9/30/21 5:54 AM, Carl Ponder wrote:

For now, you can suppress this error building OpenMPI 4.1.1
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
(.data+0x0): multiple definition of `ompi_op_avx_functions_avx2' 
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.
data+0x0): first defined here 
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
 In function `ompi_op_avx_2buff_min_uint16_t_avx2': 
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_
functions.c:651: multiple definition of `ompi_op_avx_3buff_functions_
avx2' 
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/
project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_
functions.c:651: 
first defined here
with the NVHPC/PGI 21.9 compiler by using the setting
configure --enable-mca-no-build=op-avx ... 
We're still looking at the cause here. I don't have any advice about the 
problem with 21.7.

Subject:Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, 
build hints?
Date:   Wed, 29 Sep 2021 12:25:43 -0500
From:   Ray Muno via users 
Reply-To:   Open MPI Users 
To: users@lists.open-mpi.org
CC: Ray Muno 


External email: Use caution opening links or attachments


Tried this

configure CC='nvc -fPIC' CXX='nvc++ -fPIC' FC='nvfortran -fPIC'

Configure completes. Compiles quite a way through. Dies in a different 
place. It does get past the
first error, however with libmpi_usempif08.la


FCLD libmpi_usempif08.la
make[2]: Leaving directory
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/use
-mpi-f08'
Making all in mpi/fortran/mpiext-use-mpi-f08
make[2]: Entering directory
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/
mpiext-use-mpi-f08'
PPFC mpi-f08-ext-module.lo
FCLD libforce_usempif08_module_to_be_built.la
make[2]: Leaving directory
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/
mpiext-use-mpi-f08'

Dies here now.

CCLD liblocal_ops_avx512.la
CCLD mca_op_avx.la
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
(.data+0x0): multiple
definition of `ompi_op_avx_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.
data+0x0): first defined here
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
 In function
`ompi_op_avx_2buff_min_uint16_t_avx2':
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_
functions.c:651: multiple
definition of `ompi_op_avx_3buff_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/
project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_
functions.c:651:
first defined here
make[2]: *** [mca_op_avx.la] Error 2
make[2]: Leaving directory `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-
HPC/21.9/ompi/mca/op/avx'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-
HPC/21.9/ompi'
make: *** [all-recursive] Error 1


On 9/29/21 11:42 AM, Bennet Fauber via users wrote:
Ray,

If all the errors about not being compiled with -fPIC are still 
appearing, there may be a bug that
is preventing the option from getting through to the compiler(s).  It 
might be worth looking through
the logs to see the full compile command for one or more of them to see 
whether that is true?  Say,
libs/comm_spawn_multiple_f08.o for example?

If -fPIC is missing, you may be able to recompile that manually with the 
-fPIC in place, then remake
and see if that also causes the link error to go away, that would be a 
good start.

Hope this helps,-- bennet



On Wed, Sep 29, 2021 at 12:29 PM Ray Muno via users mailto:users@lists.open-mpi.org>> wrote:

Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, build hints?

2021-09-30 Thread Bennet Fauber via users 
You may be seeing this?

C language issues Default to -fno-common

A common mistake in C is omitting extern when declaring a global variable
in a header file. If the header is included by several files it results in
multiple definitions of the same variable. In previous GCC versions this
error is ignored. GCC 10 defaults to -fno-common, which means a linker
error will now be reported. To fix this, use extern in header files when
declaring global variables, and ensure each global is defined in exactly
one C file. If tentative definitions of particular variables need to be
placed in a common block, __attribute__((__common__)) can be used to force
that behavior even in code compiled without -fcommon. As a workaround,
legacy C code where all tentative definitions should be placed into a
common block can be compiled with -fcommon.

https://gcc.gnu.org/gcc-10/porting_to.html


On Thu, Sep 30, 2021 at 6:56 AM Carl Ponder via users <
users@lists.open-mpi.org> wrote:

>
> For now, you can suppress this error building OpenMPI 4.1.1
>
> ./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0):
> multiple definition of `ompi_op_avx_functions_avx2'
> ./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
> first defined here
> ./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
> In function`ompi_op_avx_2buff_min_uint16_t_avx2':
> /project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
> multiple definition of `ompi_op_avx_3buff_functions_avx2'
>
> ./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
> first defined here
>
> with the NVHPC/PGI 21.9 compiler by using the setting
>
> configure -*-enable-mca-no-build=op-avx* ...
>
> We're still looking at the cause here. I don't have any advice about the
> problem with 21.7.
>
> --
> Subject: Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk,
> build hints?
> Date: Wed, 29 Sep 2021 12:25:43 -0500
> From: Ray Muno via users 
> 
> Reply-To: Open MPI Users 
> 
> To: users@lists.open-mpi.org
> CC: Ray Muno  
>
> External email: Use caution opening links or attachments
>
>
> Tried this
>
> configure CC='nvc -fPIC' CXX='nvc++ -fPIC' FC='nvfortran -fPIC'
>
> Configure completes. Compiles quite a way through. Dies in a different
> place. It does get past the
> first error, however with libmpi_usempif08.la
>
>
> FCLD libmpi_usempif08.la
> make[2]: Leaving directory
>
> `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/use-mpi-f08'
> Making all in mpi/fortran/mpiext-use-mpi-f08
> make[2]: Entering directory
>
> `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/mpiext-use-mpi-f08'
> PPFC mpi-f08-ext-module.lo
> FCLD libforce_usempif08_module_to_be_built.la
> make[2]: Leaving directory
>
> `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/mpiext-use-mpi-f08'
>
> Dies here now.
>
> CCLD liblocal_ops_avx512.la
> CCLD mca_op_avx.la
> ./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0):
> multiple
> definition of `ompi_op_avx_functions_avx2'
> ./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
> first defined here
> ./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
> In function
> `ompi_op_avx_2buff_min_uint16_t_avx2':
> /project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
> multiple
> definition of `ompi_op_avx_3buff_functions_avx2'
>
> ./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
> first defined here
> make[2]: *** [mca_op_avx.la] Error 2
> make[2]: Leaving directory
> `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mca/op/avx'
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory
> `/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi'
> make: *** [all-recursive] Error 1
>
>
> On 9/29/21 11:42 AM, Bennet Fauber via users wrote:
>
> Ray,
>
> If all the errors about not being compiled with -fPIC are still appearing, 
> there may be a bug that
> is preventing the option from getting through to the compiler(s).  It might 
> be worth looking through
> the logs to see the full compile command for one or more of them to see 
> whether that is true?  Say,
> libs/comm_spawn_multiple_f08.o for example?
>
> If -fPIC is missing, you may be able to recompile that manually with the 
> -fPIC in place, then remake
> and see if that also causes the link error to go away, that would be a good 
> start.
>
> Hope this helps,-- bennet
>
>
>
> On Wed, Sep 29, 2021 at 12:29 PM Ray Muno via users 
> mailto:users@lists.open-mpi.org> 
> > wrote:
>
> I did try that and it fails at the same place.
>
> Which version of the nVidia HPC-SDK are you

Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, build hints?

2021-09-30 Thread Raymond Muno via users 
 Added -*-enable-mca-no-build=op-avx *to the configure line. Still dies 
in the same place.



CCLD mca_op_avx.la
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0): 
multiple definition of `ompi_op_avx_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0): 
first defined here
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o): 
In function `ompi_op_avx_2buff_min_uint16_t_avx2':
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651: 
multiple definition of `ompi_op_avx_3buff_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651: 
first defined here

make[2]: *** [mca_op_avx.la] Error 2
make[2]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mca/op/avx'

make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi'

make: *** [all-recursive] Error 1

On 9/30/21 5:54 AM, Carl Ponder wrote:


For now, you can suppress this error building OpenMPI 4.1.1


./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0):
multipledefinition of `ompi_op_avx_functions_avx2'

./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
first defined here
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
In function`ompi_op_avx_2buff_min_uint16_t_avx2':

/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
multipledefinition of `ompi_op_avx_3buff_functions_avx2'

./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
first defined here

with the NVHPC/PGI 21.9 compiler by using the setting

configure -*-enable-mca-no-build=op-avx* ...

We're still looking at the cause here. I don't have any advice about 
the problem with 21.7.



Subject: 	Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, 
build hints?

Date:   Wed, 29 Sep 2021 12:25:43 -0500
From:   Ray Muno via users 
Reply-To:   Open MPI Users 
To: users@lists.open-mpi.org
CC: Ray Muno 



External email: Use caution opening links or attachments


Tried this

configure CC='nvc -fPIC' CXX='nvc++ -fPIC' FC='nvfortran -fPIC'

Configure completes. Compiles quite a way through. Dies in a different 
place. It does get past the

first error, however with libmpi_usempif08.la


FCLD libmpi_usempif08.la
make[2]: Leaving directory
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/use-mpi-f08'
Making all in mpi/fortran/mpiext-use-mpi-f08
make[2]: Entering directory
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/mpiext-use-mpi-f08'
PPFC mpi-f08-ext-module.lo
FCLD libforce_usempif08_module_to_be_built.la
make[2]: Leaving directory
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/mpiext-use-mpi-f08'

Dies here now.

CCLD liblocal_ops_avx512.la
CCLD mca_op_avx.la
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0): 
multiple

definition of `ompi_op_avx_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0): 
first defined here
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o): 
In function

`ompi_op_avx_2buff_min_uint16_t_avx2':
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651: 
multiple

definition of `ompi_op_avx_3buff_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
first defined here
make[2]: *** [mca_op_avx.la] Error 2
make[2]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mca/op/avx'

make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi'

make: *** [all-recursive] Error 1


On 9/29/21 11:42 AM, Bennet Fauber via users wrote:

Ray,

If all the errors about not being compiled with -fPIC are still appearing, 
there may be a bug that
is preventing the option from getting through to the compiler(s).  It might be 
worth looking through
the logs to see the full compile command for one or more of them to see whether 
that is true?  Say,
libs/comm_spawn_multiple_f08.o for example?

If -fPIC is missing, you may be able to recompile that manually with the -fPIC 
in place, then remake
and see if that also causes the link error to go away, that would be a good 
start.

Hope this helps,-- bennet



On Wed, Sep 29, 2021 at 12:29 PM Ray Muno via users mailto:users@lists.open-mpi.org>> wrote:

 I did try that

Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, build hints?

2021-09-30 Thread Carl Ponder via users 


For now, you can suppress this error building OpenMPI 4.1.1

   
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0):
   multipledefinition of `ompi_op_avx_functions_avx2'
   
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0):
   first defined here
   ./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):
   In function`ompi_op_avx_2buff_min_uint16_t_avx2':
   
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
   multipledefinition of `ompi_op_avx_3buff_functions_avx2'
   
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
   first defined here

with the NVHPC/PGI 21.9 compiler by using the setting

   configure -*-enable-mca-no-build=op-avx* ...

We're still looking at the cause here. I don't have any advice about the 
problem with 21.7.



Subject: 	Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, 
build hints?

Date:   Wed, 29 Sep 2021 12:25:43 -0500
From:   Ray Muno via users 
Reply-To:   Open MPI Users 
To: users@lists.open-mpi.org
CC: Ray Muno 



External email: Use caution opening links or attachments


Tried this

configure CC='nvc -fPIC' CXX='nvc++ -fPIC' FC='nvfortran -fPIC'

Configure completes. Compiles quite a way through. Dies in a different 
place. It does get past the

first error, however with libmpi_usempif08.la


FCLD libmpi_usempif08.la
make[2]: Leaving directory
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/use-mpi-f08'
Making all in mpi/fortran/mpiext-use-mpi-f08
make[2]: Entering directory
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/mpiext-use-mpi-f08'
PPFC mpi-f08-ext-module.lo
FCLD libforce_usempif08_module_to_be_built.la
make[2]: Leaving directory
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mpi/fortran/mpiext-use-mpi-f08'

Dies here now.

CCLD liblocal_ops_avx512.la
CCLD mca_op_avx.la
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o):(.data+0x0): 
multiple

definition of `ompi_op_avx_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):(.data+0x0): 
first defined here
./.libs/liblocal_ops_avx512.a(liblocal_ops_avx512_la-op_avx_functions.o): 
In function

`ompi_op_avx_2buff_min_uint16_t_avx2':
/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651: 
multiple

definition of `ompi_op_avx_3buff_functions_avx2'
./.libs/liblocal_ops_avx2.a(liblocal_ops_avx2_la-op_avx_functions.o):/project/muno/OpenMPI/BUILD/SRC/openmpi-4.1.1/ompi/mca/op/avx/op_avx_functions.c:651:
first defined here
make[2]: *** [mca_op_avx.la] Error 2
make[2]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi/mca/op/avx'

make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory 
`/project/muno/OpenMPI/BUILD/4.1.1/ROME/NV-HPC/21.9/ompi'

make: *** [all-recursive] Error 1


On 9/29/21 11:42 AM, Bennet Fauber via users wrote:

Ray,

If all the errors about not being compiled with -fPIC are still appearing, 
there may be a bug that
is preventing the option from getting through to the compiler(s).  It might be 
worth looking through
the logs to see the full compile command for one or more of them to see whether 
that is true?  Say,
libs/comm_spawn_multiple_f08.o for example?

If -fPIC is missing, you may be able to recompile that manually with the -fPIC 
in place, then remake
and see if that also causes the link error to go away, that would be a good 
start.

Hope this helps,-- bennet



On Wed, Sep 29, 2021 at 12:29 PM Ray Muno via users mailto:users@lists.open-mpi.org>> wrote:

 I did try that and it fails at the same place.

 Which version of the nVidia HPC-SDK are you using? I a m using 21.7. I see 
there is an upgrade to
 21.9, which came out since I installed.  I have that installed and will 
try to see if they changed
 anything. Not much in the releases notes to indicate any major changes.

 -Ray Muno


 On 9/29/21 10:54 AM, Jing Gong wrote:
  > Hi,
  >
  >
  > Before Nvidia persons look into details,pProbably you can try to add the flag 
"-fPIC" to the
  > nvhpc compiler likes cc="nvc -fPIC", which at least worked with me.
  >
  >
  >
  > /Jing
  >
  >
 

  > *From:* users mailto:users-boun...@lists.open-mpi.org>> on
 behalf of Ray Muno via users
  > mailto:users@lists.open-mpi.org>>
  > *Sent:* Wednesday, September 29, 2021 17:22
  > *To:* Open MPI User's List
  > *Cc:* Ray Muno
  > *Subject:* Re: [OMPI users] OpenMPI 4.1.1, CentOS 7.9, nVidia HPC-SDk, 
build hints?
  > Thanks, I looked through previous emails here in