Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Götz Waschk]

On Fri, Oct 8, 2010 at 8:27 PM, Richard Walsh
 wrote:
> Regarding building HD5 ... the OpenMPI 1.4.1 wrapper using the May 2010
> release of the Intel Compiler Toolkit Cluster Edition (ICTCE) worked for me.
> Here is my config.log header:
>  $ ./configure CC=mpicc CXX=mpiCC F77=mpif77 FC=mpif90 --enable-parallel 
> --prefix=/share/apps/hdf5/1.8.4p --with-zlib=/share/apps/zlib/1.2.3/lib 
> --with-szlib=/share/apps/szip/2.1/lib --disable-shared
Hi,

yes, with static linking it was never a problem to build with the
Intel compiler. The shared library builds are problematic.

-- 
AL I:40: Do what thou wilt shall be the whole of the Law.

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Jeff Squyres (jsquyres)]

Good!

Sent from my PDA. No type good. 

On Oct 8, 2010, at 2:34 PM, "Richard Walsh" <richard.wa...@csi.cuny.edu> wrote:

> 
> All,
> 
> Regarding building HD5 ... the OpenMPI 1.4.1 wrapper using the May 2010
> release of the Intel Compiler Toolkit Cluster Edition (ICTCE) worked for me.
> Here is my config.log header:
> 
>  $ ./configure CC=mpicc CXX=mpiCC F77=mpif77 FC=mpif90 --enable-parallel 
> --prefix=/share/apps/hdf5/1.8.4p --with-zlib=/share/apps/zlib/1.2.3/lib 
> --with-szlib=/share/apps/szip/2.1/lib --disable-shared
> 
> With some tweaking I was able to build the whole WRF and NCAR-NCL
> stack here.
> 
> Regards,
> 
> rbw
> 
> Richard Walsh
> Parallel Applications and Systems Manager
> CUNY HPC Center, Staten Island, NY
> 718-982-3319
> 612-382-4620
> 
> Reason does give the heart pause;
> As the heart gives reason fits.
> 
> Yet, to live where reason always rules;
> Is to kill one's heart with wits.
> 
> From: users-boun...@open-mpi.org [users-boun...@open-mpi.org] On Behalf Of 
> Gus Correa [g...@ldeo.columbia.edu]
> Sent: Friday, October 08, 2010 1:58 PM
> To: Open MPI Users
> Subject: Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran
> 
> Jeff Squyres wrote:
>> On Oct 8, 2010, at 1:00 PM, Götz Waschk wrote:
>> 
>>> I have solved this problem myself. The trick is not to use the
>>> compiler wrappers but icc and ifort directly. But in that case you'll
>>> have to link to libmpi_f77 manually and set the variable RUNPARALLEL
>>> to a working mpirun command.
>> 
>> Strange.
>> 
>> Be sure to see: 
>> http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers
>> 
> Hi Jeff
> 
> Sadly, it is not only HDF5.
> 
> There are several public domain parallel programs which we use here
> that refuse to build using the mpi wrappers (from OpenMPI,
> MPICH2, or MVAPICH2), and require the same "deconstruction"
> of configuration and make files that Gotz probably had to face.
> 
> Typically the Makefiles have hardwired items that cannot be overridden
> (e.g. -lmpich for the MPI library name passed to the linker,
> compiler options and flags that are vendor-specific, logic that
> separates MPI, OpenMP, and serial compilation, etc).
> I have used the FAQ link you sent many times in this regard.
> 
> I would guess that even more than the end users of MPI,
> it is the developers of public domain software
> (and of software projects funded by public grants)
> who need to be convinced that the right thing
> to do is to ensure that MPI wrappers will compile their
> software without problems.
> 
> My $0.02
> Gus Correa
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
> 
> Think green before you print this email.
> 
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Jeff Squyres (jsquyres)]

We've been trying to convince people to use thewrappers unless they have a 
*really, really good* reason not to for years. :(

Sent from my PDA. No type good. 

On Oct 8, 2010, at 1:59 PM, "Gus Correa"  wrote:

> Jeff Squyres wrote:
>> On Oct 8, 2010, at 1:00 PM, Götz Waschk wrote:
>>> I have solved this problem myself. The trick is not to use the
>>> compiler wrappers but icc and ifort directly. But in that case you'll
>>> have to link to libmpi_f77 manually and set the variable RUNPARALLEL
>>> to a working mpirun command.
>> Strange.
>> Be sure to see: 
>> http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers
> Hi Jeff
> 
> Sadly, it is not only HDF5.
> 
> There are several public domain parallel programs which we use here
> that refuse to build using the mpi wrappers (from OpenMPI,
> MPICH2, or MVAPICH2), and require the same "deconstruction"
> of configuration and make files that Gotz probably had to face.
> 
> Typically the Makefiles have hardwired items that cannot be overridden (e.g. 
> -lmpich for the MPI library name passed to the linker,
> compiler options and flags that are vendor-specific, logic that
> separates MPI, OpenMP, and serial compilation, etc).
> I have used the FAQ link you sent many times in this regard.
> 
> I would guess that even more than the end users of MPI,
> it is the developers of public domain software
> (and of software projects funded by public grants)
> who need to be convinced that the right thing
> to do is to ensure that MPI wrappers will compile their
> software without problems.
> 
> My $0.02
> Gus Correa
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Richard Walsh]

All,

Regarding building HD5 ... the OpenMPI 1.4.1 wrapper using the May 2010
release of the Intel Compiler Toolkit Cluster Edition (ICTCE) worked for me.
Here is my config.log header:

  $ ./configure CC=mpicc CXX=mpiCC F77=mpif77 FC=mpif90 --enable-parallel 
--prefix=/share/apps/hdf5/1.8.4p --with-zlib=/share/apps/zlib/1.2.3/lib 
--with-szlib=/share/apps/szip/2.1/lib --disable-shared

With some tweaking I was able to build the whole WRF and NCAR-NCL
stack here.

Regards,

rbw

Richard Walsh
Parallel Applications and Systems Manager
CUNY HPC Center, Staten Island, NY
718-982-3319
612-382-4620

Reason does give the heart pause;
As the heart gives reason fits.

Yet, to live where reason always rules;
Is to kill one's heart with wits.

From: users-boun...@open-mpi.org [users-boun...@open-mpi.org] On Behalf Of Gus 
Correa [g...@ldeo.columbia.edu]
Sent: Friday, October 08, 2010 1:58 PM
To: Open MPI Users
Subject: Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

Jeff Squyres wrote:
> On Oct 8, 2010, at 1:00 PM, Götz Waschk wrote:
>
>> I have solved this problem myself. The trick is not to use the
>> compiler wrappers but icc and ifort directly. But in that case you'll
>> have to link to libmpi_f77 manually and set the variable RUNPARALLEL
>> to a working mpirun command.
>
> Strange.
>
> Be sure to see: 
> http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers
>
Hi Jeff

Sadly, it is not only HDF5.

There are several public domain parallel programs which we use here
that refuse to build using the mpi wrappers (from OpenMPI,
MPICH2, or MVAPICH2), and require the same "deconstruction"
of configuration and make files that Gotz probably had to face.

Typically the Makefiles have hardwired items that cannot be overridden
(e.g. -lmpich for the MPI library name passed to the linker,
compiler options and flags that are vendor-specific, logic that
separates MPI, OpenMP, and serial compilation, etc).
I have used the FAQ link you sent many times in this regard.

I would guess that even more than the end users of MPI,
it is the developers of public domain software
(and of software projects funded by public grants)
who need to be convinced that the right thing
to do is to ensure that MPI wrappers will compile their
software without problems.

My $0.02
Gus Correa
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users

Think green before you print this email.

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Gus Correa]

Jeff Squyres wrote:

On Oct 8, 2010, at 1:00 PM, Götz Waschk wrote:


I have solved this problem myself. The trick is not to use the
compiler wrappers but icc and ifort directly. But in that case you'll
have to link to libmpi_f77 manually and set the variable RUNPARALLEL
to a working mpirun command.


Strange.

Be sure to see: http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers


Hi Jeff

Sadly, it is not only HDF5.

There are several public domain parallel programs which we use here
that refuse to build using the mpi wrappers (from OpenMPI,
MPICH2, or MVAPICH2), and require the same "deconstruction"
of configuration and make files that Gotz probably had to face.

Typically the Makefiles have hardwired items that cannot be overridden 
(e.g. -lmpich for the MPI library name passed to the linker,

compiler options and flags that are vendor-specific, logic that
separates MPI, OpenMP, and serial compilation, etc).
I have used the FAQ link you sent many times in this regard.

I would guess that even more than the end users of MPI,
it is the developers of public domain software
(and of software projects funded by public grants)
who need to be convinced that the right thing
to do is to ensure that MPI wrappers will compile their
software without problems.

My $0.02
Gus Correa

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Jeff Squyres]

On Oct 8, 2010, at 1:00 PM, Götz Waschk wrote:

> I have solved this problem myself. The trick is not to use the
> compiler wrappers but icc and ifort directly. But in that case you'll
> have to link to libmpi_f77 manually and set the variable RUNPARALLEL
> to a working mpirun command.

Strange.

Be sure to see: http://www.open-mpi.org/faq/?category=mpi-apps#cant-use-wrappers

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Götz Waschk]

On Wed, Oct 6, 2010 at 4:35 PM, Jeff Squyres  wrote:
> On Oct 6, 2010, at 10:07 AM, Götz Waschk wrote:
>>> Do -Wl,-rpath and -Wl,-soname= work any better?
>> Yes, with these options, it build fine. But the command line is
>> generated by libtool, so how can I make libtool to use -Wl, in front
>> of the linker options? It seems to strip these from the command.
> This might well be directly related to the version of Libtool that HDF 
> bootstraps their build process with -- you really should talk to them.
Hi everyone,

I have solved this problem myself. The trick is not to use the
compiler wrappers but icc and ifort directly. But in that case you'll
have to link to libmpi_f77 manually and set the variable RUNPARALLEL
to a working mpirun command.

Regards, Götz
-- 
AL I:40: Do what thou wilt shall be the whole of the Law.

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Jeff Squyres]

On Oct 6, 2010, at 10:07 AM, Götz Waschk wrote:

>> Do -Wl,-rpath and -Wl,-soname= work any better?
> Yes, with these options, it build fine. But the command line is
> generated by libtool, so how can I make libtool to use -Wl, in front
> of the linker options? It seems to strip these from the command.

This might well be directly related to the version of Libtool that HDF 
bootstraps their build process with -- you really should talk to them.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Götz Waschk]

On Wed, Oct 6, 2010 at 2:43 PM, Tim Prince  wrote:
>> libtool: link: mpif90 -shared  .libs/H5f90global.o
>> .libs/H5fortran_types.o .libs/H5_ff.o .libs/H5Aff.o .libs/H5Dff.o
>> .libs/H5Eff.o .libs/H5Fff.o .libs/H5Gff.o .libs/H5Iff.o .libs/H5Lff.o
>> .libs/H5Off.o .libs/H5Pff.o .libs/H5Rff.o .libs/H5Sff.o .libs/H5Tff.o
>> .libs/H5Zff.o .libs/H5_DBLE_InterfaceInclude.o .libs/H5f90kit.o
>> .libs/H5_f.o .libs/H5Af.o .libs/H5Df.o .libs/H5Ef.o .libs/H5Ff.o
>> .libs/H5Gf.o .libs/H5If.o .libs/H5Lf.o .libs/H5Of.o .libs/H5Pf.o
>> .libs/H5Rf.o .libs/H5Sf.o .libs/H5Tf.o .libs/H5Zf.o .libs/H5FDmpiof.o
>> .libs/HDF5mpio.o .libs/H5FDmpioff.o    -lmpi -lsz -lz -lm  -m64
>> -mtune=generic -rpath=/usr/lib64/openmpi/1.4-icc/lib   -soname
>> libhdf5_fortran.so.6 -o .libs/libhdf5_fortran.so.6.0.4
>> ifort: command line warning #10156: ignoring option '-r'; no argument
>> required
>> ifort: command line warning #10156: ignoring option '-s'; no argument
>> required
>> ld: libhdf5_fortran.so.6: No such file: No such file or directory
> Do -Wl,-rpath and -Wl,-soname= work any better?
Yes, with these options, it build fine. But the command line is
generated by libtool, so how can I make libtool to use -Wl, in front
of the linker options? It seems to strip these from the command.

Regards, Götz

-- 
AL I:40: Do what thou wilt shall be the whole of the Law.

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Tim Prince]

 On 10/6/2010 12:09 AM, Götz Waschk wrote:

libtool: link: mpif90 -shared  .libs/H5f90global.o
.libs/H5fortran_types.o .libs/H5_ff.o .libs/H5Aff.o .libs/H5Dff.o
.libs/H5Eff.o .libs/H5Fff.o .libs/H5Gff.o .libs/H5Iff.o .libs/H5Lff.o
.libs/H5Off.o .libs/H5Pff.o .libs/H5Rff.o .libs/H5Sff.o .libs/H5Tff.o
.libs/H5Zff.o .libs/H5_DBLE_InterfaceInclude.o .libs/H5f90kit.o
.libs/H5_f.o .libs/H5Af.o .libs/H5Df.o .libs/H5Ef.o .libs/H5Ff.o
.libs/H5Gf.o .libs/H5If.o .libs/H5Lf.o .libs/H5Of.o .libs/H5Pf.o
.libs/H5Rf.o .libs/H5Sf.o .libs/H5Tf.o .libs/H5Zf.o .libs/H5FDmpiof.o
.libs/HDF5mpio.o .libs/H5FDmpioff.o-lmpi -lsz -lz -lm  -m64
-mtune=generic -rpath=/usr/lib64/openmpi/1.4-icc/lib   -soname
libhdf5_fortran.so.6 -o .libs/libhdf5_fortran.so.6.0.4
ifort: command line warning #10156: ignoring option '-r'; no argument required
ifort: command line warning #10156: ignoring option '-s'; no argument required
ld: libhdf5_fortran.so.6: No such file: No such file or directory

Do -Wl,-rpath and -Wl,-soname= work any better?

--
Tim Prince

Re: [OMPI users] hdf5 build error using openmpi and Intel Fortran

[Jeff Squyres (jsquyres)]

This looks like something you should take up with the hdf5 people. 

Sent from my PDA. No type good. 

On Oct 6, 2010, at 3:09 AM, Götz Waschk  wrote:

> Hi everyone,
> 
> I'm trying to build hdf5 1.8.5-patch1 on RHEL5 using openmpi 1.4 and
> the Intel Compiler suite 11.0. I have Fortran, MPI and shared library
> support enabled. I get this error at linking stage:
> 
> /bin/sh ../../libtool --tag=FC   --mode=link mpif90 -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC  -O3 -I../../src
> -I../../fortran/src  -version-info 6:4:0
> -Wl,-rpath=/usr/lib64/openmpi/1.4-icc/lib -lmpi -o libhdf5_fortran.la
> -rpath /usr/lib64 H5f90global.lo H5fortran_types.lo H5_ff.lo H5Aff.lo
> H5Dff.lo H5Eff.lo H5Fff.lo H5Gff.lo H5Iff.lo H5Lff.lo H5Off.lo
> H5Pff.lo H5Rff.lo H5Sff.lo H5Tff.lo H5Zff.lo
> H5_DBLE_InterfaceInclude.lo H5f90kit.lo H5_f.lo H5Af.lo H5Df.lo
> H5Ef.lo H5Ff.lo H5Gf.lo H5If.lo H5Lf.lo H5Of.lo H5Pf.lo H5Rf.lo
> H5Sf.lo H5Tf.lo H5Zf.lo H5FDmpiof.lo HDF5mpio.lo H5FDmpioff.lo  -lsz
> -lz -lm
> libtool: link: mpif90 -shared  .libs/H5f90global.o
> .libs/H5fortran_types.o .libs/H5_ff.o .libs/H5Aff.o .libs/H5Dff.o
> .libs/H5Eff.o .libs/H5Fff.o .libs/H5Gff.o .libs/H5Iff.o .libs/H5Lff.o
> .libs/H5Off.o .libs/H5Pff.o .libs/H5Rff.o .libs/H5Sff.o .libs/H5Tff.o
> .libs/H5Zff.o .libs/H5_DBLE_InterfaceInclude.o .libs/H5f90kit.o
> .libs/H5_f.o .libs/H5Af.o .libs/H5Df.o .libs/H5Ef.o .libs/H5Ff.o
> .libs/H5Gf.o .libs/H5If.o .libs/H5Lf.o .libs/H5Of.o .libs/H5Pf.o
> .libs/H5Rf.o .libs/H5Sf.o .libs/H5Tf.o .libs/H5Zf.o .libs/H5FDmpiof.o
> .libs/HDF5mpio.o .libs/H5FDmpioff.o-lmpi -lsz -lz -lm  -m64
> -mtune=generic -rpath=/usr/lib64/openmpi/1.4-icc/lib   -soname
> libhdf5_fortran.so.6 -o .libs/libhdf5_fortran.so.6.0.4
> ifort: command line warning #10156: ignoring option '-r'; no argument required
> ifort: command line warning #10156: ignoring option '-s'; no argument required
> ld: libhdf5_fortran.so.6: No such file: No such file or directory
> make[1]: *** [libhdf5_fortran.la] Fehler 1
> 
> 
> hdf5 builds fine with static libraries only, but they become huge. It
> looks like libtool or mpif90 or something else is calling ifort with
> the wrong options. Any idea on how to fix this?
> 
> Regards, Götz Waschk
> 
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users