It also depends on what exactly you mean by "static binaries" (see
below).
To fill in the background for those on the list, we had some prior
off-list e-mail about static builds where I said:
-----
Keep in mind that "--disable-shared --enable-static" on OMPI's
configure line refers to how *Open MPI's libraries are built*; it
does not refer to how Open MPI's libraries or executables are linked
to external libraries (we didn't make this up; it's a GNU standard).
For example, I just did a "--disable-shared --enable-static" build on
my machine. Running ldd on the generated "mpirun", I see that it is
fully static with respect to all OMPI components/libraries, but it
still links to a variety of other shared libraries:
shell$ ldd mpirun
libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x0000003af3a00000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000002a95585000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003af9300000)
libutil.so.1 => /lib64/libutil.so.1 (0x0000003af9500000)
libm.so.6 => /lib64/tls/libm.so.6 (0x0000003af3f00000)
libpthread.so.0 => /lib64/tls/libpthread.so.0
(0x0000003af4700000)
libc.so.6 => /lib64/tls/libc.so.6 (0x0000003af3c00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003af3800000)
[...snipped...]
Typically, one would use the "-static" (or your compiler's
equivalent) flag to produce fully static executables. I was unable
to make this work on my development systems, however; it seems that
some Linux system calls *require* shared libraries (I'm not sure
why). For example:
[...failed link line omitted, similar to what Josh showed...]
Let me know if this is a priority for you and I can poke around
further. I'm guessing that your main goal is with regards to
libtorque, and the other system libraries aren't that important.
-----
With that background, a few questions on this mail:
- You mentioned that you used "--enable-static" but you did not use
"--disable-shared". Did you intend to build both the static and
dynamic OMPI libraries?
- What exactly is your intent: a) do you want 100% static executables
(to include the normal Linux system libraries), b) do you just want
to avoid having to set LD_LIBRARY_PATH for some of the esoteric
libraries such as libmpi, libtorque, libibverbs, etc., or c)
something else?
Given the messages from the linker and my prior mail, I'm not sure
that a) is possible -- it will require more poking around to know for
sure. b) is definitely possible simply by using --enable-static and
--disable-shared and then using "mpicc" (and friends) as normal.
OMPI will be built statically (to include torque and libibverbs,
assuming you have .a's versions of both of those), but as Scott
mentioned, system libraries such as dl and sysfs will be linked
dynamically.
On Oct 13, 2006, at 10:22 PM, Scott Weitzenkamp ((sweitzen)) wrote:
Shouldn't mpicc always link system libraries (like -ldl and -lsysfs)
dynamically?
Scott Weitzenkamp
SQA and Release Manager
Server Virtualization Business Unit
Cisco Systems
-----Original Message-----
From: users-boun...@open-mpi.org
[mailto:users-boun...@open-mpi.org] On Behalf Of Josh England
Sent: Friday, October 13, 2006 3:53 PM
To: Open MPI Users
Subject: [OMPI users] problem building static w/ 1.1.2_rc4
Hi,
I'm trying to statically compile a simple program, but have run into
some difficulties so far. Trying with my own link line I get many
complaints about symbols being redefined:
[root@dlogin2 examples]# gcc -static -o cpi cpi.c
-I/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/include
-L/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib
-L/apps/torque/lib -L/usr/ofed/lib64 -lmpi -lopal -ldl -lpthread -lrt
-lorte -lopal -ltorque -lutil -libverbs -lsysfs -lm
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(ltdl.o)(.text+0x2ef): In function `sys_dl_open':
: warning: Using 'dlopen' in statically linked applications
requires at
runtime the shared libraries from the glibc version used for linking
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
liborte.a(sys_info.o)(.text+0xa2): In function `orte_sys_info':
: warning: Using 'getpwuid' in statically linked applications
requires
at runtime the shared libraries from the glibc version used
for linking
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(if.o)(.text+0x5d9): In function `opal_ifaddrtoname':
: warning: Using 'gethostbyname' in statically linked applications
requires at runtime the shared libraries from the glibc
version used for
linking
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0x1880): In function `free':
: multiple definition of `free'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x3a8a): first defined here
/usr/bin/ld: Warning: size of symbol `free' changed from 231
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 255 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0x38d0): In function `malloc':
: multiple definition of `malloc'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x2fd5): first defined here
/usr/bin/ld: Warning: size of symbol `malloc' changed from 319
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 461 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0x3fb0): In function `realloc':
: multiple definition of `realloc'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x3b71): first defined here
/usr/bin/ld: Warning: size of symbol `realloc' changed from 448
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 783 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0x4c0): In function `_int_new_arena':
: multiple definition of `_int_new_arena'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x352): first defined here
/usr/bin/ld: Warning: size of symbol `_int_new_arena' changed from
242
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 224 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0xff0): In function `_int_free':
: multiple definition of `_int_free'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x113a): first defined here
/usr/bin/ld: Warning: size of symbol `_int_free' changed from 856
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 1897 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0x1980): In function `_int_malloc':
: multiple definition of `_int_malloc'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x1b0a): first defined here
/usr/bin/ld: Warning: size of symbol `_int_malloc' changed from 3193
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 3548 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0x2840): In function `_int_memalign':
: multiple definition of `_int_memalign'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x28b9): first defined here
/usr/bin/ld: Warning: size of symbol `_int_memalign' changed from 464
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 524 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0x2a50): In function `_int_valloc':
: multiple definition of `_int_valloc'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x3d71): first defined here
/usr/bin/ld: Warning: size of symbol `_int_valloc' changed from 69
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 71 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0x2c10): In function `_int_realloc':
: multiple definition of `_int_realloc'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x2cd8): first defined here
/usr/bin/ld: Warning: size of symbol `_int_realloc' changed from 765
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 1273 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)(.text+0x42c0): In function `__malloc_check_init':
: multiple definition of `__malloc_check_init'
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o)(.text+0x631): first defined here
/usr/bin/ld: Warning: size of symbol `__malloc_check_init'
changed from
141
in
/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
libopal.a(lt1-malloc.o) to 164 in
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
a(malloc.o)
collect2: ld returned 1 exit status
When I try mpicc, its seemingly unable to find some symbols,
even when I
add the right library:
[root@dlogin2 examples]# mpicc -static -o cpi cpi.c
-L/usr/lib64 -lsysfs
...
... Same 'multiple definition' stuff as above, followed by:
...
/usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
x23b): In
function `ibverbs_init':
: undefined reference to `sysfs_open_class'
/usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
x2d0): In
function `ibverbs_init':
: undefined reference to `sysfs_get_class_devices'
/usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
x2e4): In
function `ibverbs_init':
: undefined reference to `dlist_start'
/usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
x2f1): In
function `ibverbs_init':
: undefined reference to `_dlist_mark_move'
/usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
x334): In
function `ibverbs_init':
: undefined reference to `sysfs_open_class_device'
collect2: ld returned 1 exit status
[root@dlogin2 examples]# nm /usr/lib64/libsysfs.a |grep
sysfs_open_class_device
0000000000000d60 T sysfs_open_class_device
0000000000000170 T sysfs_open_class_device_path
I configured ompi with '--with-openib=/usr/ofed --with-tm=/apps/
torque
--enable-static --without-libnuma' .
Is there something I missed doing in order to be able to
compile static
binaries? Any help is greatly appreciated.
-JE
--
-----------------------------------------------
Josh England
Sandia National Laboratory, Albuquerque, NM
Infrastructure Computing Systems
email: jjen...@sandia.gov
phone: (505) 284-0963
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
--
Jeff Squyres
Server Virtualization Business Unit
Cisco Systems
