Ben et. al.....
I'm having some crazy issue with getting libMesh executables to link on a
BlueGene/Q (Mira at Argonne). The issue is that libtool is choosing to try
to link dynamic libraries even though I explicitly set --enable-static
--disable-shared. It's weird because libtool is picking up .so's and
putting them right on the link line instead of letting the linker find the
static libraries that are sitting right next to those .so's.
I can manually create link lines that work - and generate working
binaries... but I can't get libtool to do it automatically...
Below is a full description of what is happening... any help I could get
would be awesome...
Derek
---------- Forwarded message ----------
From: Gaston, Derek R <[email protected]>
Date: Thu, Aug 1, 2013 at 12:45 PM
Subject: libtool Trouble on Mira
To: Jeff Hammond <[email protected]>, "John Peterson (INL)" <
[email protected]>
Jeff,
(Note: If there is someone else I should send this to instead let me know)
I've been able to log into Mira and have been working on getting everything
compiled. So far, everything is going well. I've been able to compile
Hypre and PETSc without issue and I can _compile_ libMesh... but something
is going wrong in the linking stage when trying to link an executable with
libMesh.
What's happening is that libtool is picking up shared libraries for system
libraries such as libmpichf90. I cannot see why this is happening. If I
manually edit the link line to just link -lmpichf90 (and others) everything
works fine and my executables are fine.
I am using the +mpiwrappers-gcc softenv.
Here's a typical libtool line that the libMesh build system is trying to
run:
libtool --tag=CXX --mode=link mpicxx -O0 -felide-constructors -g -ansi
-pedantic -W -Wall -Wextra -Wno-long-long -Wunused -Wpointer-arith -Wformat
-Wparentheses -std=c++0x -Woverloaded-virtual -fopenmp -o meshtool-dbg
src/apps/meshtool_dbg-meshtool.o
libmesh_dbg.la-L/home/gastdr/moose-compilers/petsc/petsc-3.3-p4/gcc-opt/lib
-lpetsc
-Wl,-rpath,/home/gastdr/moose-compilers/hypre/hypre-2.8.0b/gcc-opt/lib
-L/home/gastdr/moose-compilers/hypre/hypre-2.8.0b/gcc-opt/lib -lHYPRE
-L/bgsys/drivers/V1R2M0/ppc64/spi/lib
-L/bgsys/drivers/V1R2M0/ppc64/comm/sys/lib
-L/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib
-L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/lib/gcc/powerpc64-bgq-linux/4.4.6
-L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/lib/gcc
-L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib
-Wl,-rpath,/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib
-Wl,-rpath,/home/gastdr/moose-compilers/petsc/petsc-3.3-p4/gcc-opt/lib
-lflapack -lfblas -lmpichf90 -lgfortran -lm -lcxxmpich -lmpich -lopa -lmpl
-lpami -lSPI -lSPI_cnk -lrt -lpthread -lstdc++ -lgomp -lgcc_eh -lnss_files
-lnss_dns -lresolv -ldl
And here is the actual link line that libtool is creating:
mpicxx -O0 -felide-constructors -g -ansi -pedantic -W -Wall -Wextra
-Wno-long-long -Wunused -Wpointer-arith -Wformat -Wparentheses -std=c++0x
-Woverloaded-virtual -fopenmp -o meshtool-dbg
src/apps/meshtool_dbg-meshtool.o -Wl,-rpath
-Wl,/home/gastdr/moose-compilers/hypre/hypre-2.8.0b/gcc-opt/lib -Wl,-rpath
-Wl,/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib -Wl,-rpath
-Wl,/home/gastdr/moose-compilers/petsc/petsc-3.3-p4/gcc-opt/lib
./.libs/libmesh_dbg.a -L./src/mpl -L./src/openpa/src
-L/home/gastdr/moose-compilers/petsc/petsc-3.3-p4/gcc-opt/lib
-L/home/gastdr/moose-compilers/hypre/hypre-2.8.0b/gcc-opt/lib
-L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib
/gpfs/mira-home/gastdr/projects/libmesh/build/contrib/netcdf/v4/liblib/.libs/libnetcdf.a
-lpetsc -lHYPRE -L/bgsys/drivers/V1R2M0/ppc64/spi/lib
-L/bgsys/drivers/V1R2M0/ppc64/comm/sys/lib
-L/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib
-L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/lib/gcc/powerpc64-bgq-linux/4.4.6
-L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/lib/gcc -lflapack
-lfblas /bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib/libmpichf90.so
/bgsys/drivers/toolchain/V1R2M0/gnu-linux/powerpc64-bgq-linux/lib/libgfortran.so
/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib/libgfortran.so
/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib/libcxxmpich.so
/bgsys/drivers/toolchain/V1R2M0/gnu-linux/powerpc64-bgq-linux/lib/libstdc++.so
/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib/libmpich.so
/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib/libopa.so
/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib/libmpl.so -lpami -lSPI -lSPI_cnk
-lpthread
/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib/libstdc++.so
-lm
/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib/libgomp.so
-lrt -lgcc_eh -lnss_files -lnss_dns -lresolv -ldl -fopenmp -Wl,-rpath
-Wl,/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib
-Wl,-rpath -Wl,/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib -Wl,-rpath
-Wl,/bgsys/drivers/toolchain/V1R2M0/gnu-linux/powerpc64-bgq-linux/lib
-Wl,-rpath
-Wl,/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib
-Wl,-rpath -Wl,/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib -Wl,-rpath
-Wl,/bgsys/drivers/toolchain/V1R2M0/gnu-linux/powerpc64-bgq-linux/lib
Note the .so files that are directly on the link line. This gives rise to
the following error:
/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/lib/gcc/powerpc64-bgq-linux/4.4.6/../../../../powerpc64-bgq-linux/bin/ld:
attempted static link of dynamic object
`/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib/libmpichf90.so'
If I remove all of the .so's from that link line (and add -lgfortran) I get
a link line that works:
mpicxx -O0 -felide-constructors -g -ansi -pedantic -W -Wall -Wextra
-Wno-long-long -Wunused -Wpointer-arith -Wformat -Wparentheses -std=c++0x
-Woverloaded-virtual -fopenmp -o meshtool-dbg
src/apps/meshtool_dbg-meshtool.o -Wl,-rpath
-Wl,/home/gastdr/moose-compilers/hypre/hypre-2.8.0b/gcc-opt/lib -Wl,-rpath
-Wl,/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib -Wl,-rpath
-Wl,/home/gastdr/moose-compilers/petsc/petsc-3.3-p4/gcc-opt/lib
./.libs/libmesh_dbg.a -L./src/mpl -L./src/openpa/src
-L/home/gastdr/moose-compilers/petsc/petsc-3.3-p4/gcc-opt/lib
-L/home/gastdr/moose-compilers/hypre/hypre-2.8.0b/gcc-opt/lib
-L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib
/gpfs/mira-home/gastdr/projects/libmesh/build/contrib/netcdf/v4/liblib/.libs/libnetcdf.a
-lpetsc -lHYPRE -L/bgsys/drivers/V1R2M0/ppc64/spi/lib
-L/bgsys/drivers/V1R2M0/ppc64/comm/sys/lib
-L/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib
-L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/lib/gcc/powerpc64-bgq-linux/4.4.6
-L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/lib/gcc -lflapack
-lfblas -lpami -lSPI -lSPI_cnk -lpthread -lm -lrt -lgcc_eh -lnss_files
-lnss_dns -lresolv -ldl -fopenmp -Wl,-rpath
-Wl,/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib
-Wl,-rpath -Wl,/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib -Wl,-rpath
-Wl,/bgsys/drivers/toolchain/V1R2M0/gnu-linux/powerpc64-bgq-linux/lib
-Wl,-rpath
-Wl,/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib
-Wl,-rpath -Wl,/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib -Wl,-rpath
-Wl,/bgsys/drivers/toolchain/V1R2M0/gnu-linux/powerpc64-bgq-linux/lib
-lgfortran
I went and looked at the .la files for those libraries (for
instance
/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib/
libgfortran.la) and they definitely are specifying the "old_library" option
like they are supposed to for the static version of the library.
I'm really stumped on this one...
Derek
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Libmesh-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/libmesh-devel
