Re: [OMPI devel] [1.8.2rc3] static linking fails on linux when not building ROMIO
Paul, i missed a step indeed : opal is required by rte, that is in turn required by mpi the attached patch does the job (tested on a solaris10/x86_64 vm with gnu compilers) Cheers, Gilles On 2014/08/06 4:40, Paul Hargrove wrote: > Gilles, > > I have not tested your patch. > I've only read it. > > It looks like it could work, except that libopen-rte.a depends on libsocket > and libnsl on Solaris. > So, one probably needs to add $LIBS to the ORTE wrapper libs as well. > > Additionally,if your approach is the correct one, then I think one can fold: > > OPAL_FLAGS_APPEND_UNIQ([OPAL_WRAPPER_EXTRA_LIBS], > [$wrapper_extra_libs]) > OPAL_WRAPPER_EXTRA_LIBS="$OPAL_WRAPPER_EXTRA_LIBS > $with_wrapper_libs" > + OPAL_FLAGS_APPEND_UNIQ([OPAL_WRAPPER_EXTRA_LIBS], [$LIBS]) > + OPAL_WRAPPER_EXTRA_LIBS="$OPAL_WRAPPER_EXTRA_LIBS > $with_wrapper_libs" > > into just > > -OPAL_FLAGS_APPEND_UNIQ([OPAL_WRAPPER_EXTRA_LIBS], > [$wrapper_extra_libs]) > + OPAL_FLAGS_APPEND_UNIQ([OPAL_WRAPPER_EXTRA_LIBS], > [$wrapper_extra_libs $LIBS]) > > which merges two calls to OPAL_FLAGS_APPEND_UNIQ and avoids double-adding > of the user's $with_wrapper_libs. > And of course the same 1-line change would apply for the OMPI and > eventually ORTE variables too. > > I'd like to wait until Jeff has had a chance to look this over before I > devote time to testing. > Since I've determined already that 1.6.5 did not have the problem while > 1.7.x does, the possibility exists that some smaller change might exist to > restore what ever was lost between the v1.6 and v1.7 branches. > > -Paul > > > On Tue, Aug 5, 2014 at 1:33 AM, Gilles Gouaillardet < > gilles.gouaillar...@iferc.org> wrote: > >> Here is a patch that has been minimally tested. >> >> this is likely an overkill (at least when dynamic libraries can be used), >> but it does the job so far ... >> >> Cheers, >> >> Gilles >> >> On 2014/08/05 16:56, Gilles Gouaillardet wrote: >> >> from libopen-pal.la : >> dependency_libs=' -lrdmacm -libverbs -lscif -lnuma -ldl -lrt -lnsl >> -lutil -lm' >> >> >> i confirm mpicc fails linking >> >> but FWIT, using libtool does work (!) >> >> could the bug come from the mpicc (and other) wrappers ? >> >> Gilles >> >> $ gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c >> -I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath >> -Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib >> -Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte >> -lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi >> >> $ /tmp/install/ompi.noromio/bin/mpicc -g -O0 -o hw -show ~/hw.c >> gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c >> -I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath >> -Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib >> -Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte >> -lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi >> [gouaillardet@soleil build]$ /tmp/install/ompi.noromio/bin/mpicc -g -O0 >> -o hw ~/hw.c >> /tmp/install/ompi.noromio/lib/libmpi.a(fbtl_posix_ipwritev.o): In >> function `mca_fbtl_posix_ipwritev': >> fbtl_posix_ipwritev.c:(.text+0x17b): undefined reference to `aio_write' >> fbtl_posix_ipwritev.c:(.text+0x237): undefined reference to `aio_write' >> fbtl_posix_ipwritev.c:(.text+0x3f4): undefined reference to `aio_write' >> fbtl_posix_ipwritev.c:(.text+0x48e): undefined reference to `aio_write' >> /tmp/install/ompi.noromio/lib/libopen-pal.a(opal_pty.o): In function >> `opal_openpty': >> opal_pty.c:(.text+0x1): undefined reference to `openpty' >> /tmp/install/ompi.noromio/lib/libopen-pal.a(event.o): In function >> `event_add_internal': >> event.c:(.text+0x288d): undefined reference to `clock_gettime' >> >> $ /bin/sh ./static/libtool --silent --tag=CC --mode=compile gcc >> -std=gnu99 -I/tmp/install/ompi.noromio/include -c ~/hw.c >> $ /bin/sh ./static/libtool --silent --tag=CC --mode=link gcc >> -std=gnu99 -o hw hw.o -L/tmp/install/ompi.noromio/lib -lmpi >> $ ldd hw >> linux-vdso.so.1 => (0x7fff7530d000) >> librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x7f0ed541e000) >> libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x7f0ed521) >> libscif.so.0 => /usr/lib64/libscif.so.0 (0x003b9c60) >> libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x003ba560) >> libdl.so.2 => /lib64/libdl.so.2 (0x003b9be0) >> librt.so.1 => /lib64/librt.so.1 (0x003b9ca0) >> libnsl.so.1 => /lib64/libnsl.so.1 (0x003bae20) >> libutil.so.1 => /lib64/libutil.so.1 (0x003bac60) >> libm.so.6 => /lib64/libm.so.6 (0x003b9ba0) >> libpthread.so.0 => /lib64/libpthread.so.0 (0x003b9c20) >> libc.so.6 => /lib64/libc.so.6 (0x003b9b60) >> /lib64/ld-linux-x86-64.so.2 (0x003b9b20) >> >> >> >> >> On 2014/08/05 7:56, Ralph Castain wrote: >> >> My thought was to post initially as a blocker, pending a discussion with
Re: [OMPI devel] [1.8.2rc3] static linking fails on linux when not building ROMIO
Gilles, I have not tested your patch. I've only read it. It looks like it could work, except that libopen-rte.a depends on libsocket and libnsl on Solaris. So, one probably needs to add $LIBS to the ORTE wrapper libs as well. Additionally,if your approach is the correct one, then I think one can fold: OPAL_FLAGS_APPEND_UNIQ([OPAL_WRAPPER_EXTRA_LIBS], [$wrapper_extra_libs]) OPAL_WRAPPER_EXTRA_LIBS="$OPAL_WRAPPER_EXTRA_LIBS $with_wrapper_libs" + OPAL_FLAGS_APPEND_UNIQ([OPAL_WRAPPER_EXTRA_LIBS], [$LIBS]) + OPAL_WRAPPER_EXTRA_LIBS="$OPAL_WRAPPER_EXTRA_LIBS $with_wrapper_libs" into just -OPAL_FLAGS_APPEND_UNIQ([OPAL_WRAPPER_EXTRA_LIBS], [$wrapper_extra_libs]) + OPAL_FLAGS_APPEND_UNIQ([OPAL_WRAPPER_EXTRA_LIBS], [$wrapper_extra_libs $LIBS]) which merges two calls to OPAL_FLAGS_APPEND_UNIQ and avoids double-adding of the user's $with_wrapper_libs. And of course the same 1-line change would apply for the OMPI and eventually ORTE variables too. I'd like to wait until Jeff has had a chance to look this over before I devote time to testing. Since I've determined already that 1.6.5 did not have the problem while 1.7.x does, the possibility exists that some smaller change might exist to restore what ever was lost between the v1.6 and v1.7 branches. -Paul On Tue, Aug 5, 2014 at 1:33 AM, Gilles Gouaillardet < gilles.gouaillar...@iferc.org> wrote: > Here is a patch that has been minimally tested. > > this is likely an overkill (at least when dynamic libraries can be used), > but it does the job so far ... > > Cheers, > > Gilles > > On 2014/08/05 16:56, Gilles Gouaillardet wrote: > > from libopen-pal.la : > dependency_libs=' -lrdmacm -libverbs -lscif -lnuma -ldl -lrt -lnsl > -lutil -lm' > > > i confirm mpicc fails linking > > but FWIT, using libtool does work (!) > > could the bug come from the mpicc (and other) wrappers ? > > Gilles > > $ gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c > -I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath > -Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib > -Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte > -lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi > > $ /tmp/install/ompi.noromio/bin/mpicc -g -O0 -o hw -show ~/hw.c > gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c > -I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath > -Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib > -Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte > -lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi > [gouaillardet@soleil build]$ /tmp/install/ompi.noromio/bin/mpicc -g -O0 > -o hw ~/hw.c > /tmp/install/ompi.noromio/lib/libmpi.a(fbtl_posix_ipwritev.o): In > function `mca_fbtl_posix_ipwritev': > fbtl_posix_ipwritev.c:(.text+0x17b): undefined reference to `aio_write' > fbtl_posix_ipwritev.c:(.text+0x237): undefined reference to `aio_write' > fbtl_posix_ipwritev.c:(.text+0x3f4): undefined reference to `aio_write' > fbtl_posix_ipwritev.c:(.text+0x48e): undefined reference to `aio_write' > /tmp/install/ompi.noromio/lib/libopen-pal.a(opal_pty.o): In function > `opal_openpty': > opal_pty.c:(.text+0x1): undefined reference to `openpty' > /tmp/install/ompi.noromio/lib/libopen-pal.a(event.o): In function > `event_add_internal': > event.c:(.text+0x288d): undefined reference to `clock_gettime' > > $ /bin/sh ./static/libtool --silent --tag=CC --mode=compile gcc > -std=gnu99 -I/tmp/install/ompi.noromio/include -c ~/hw.c > $ /bin/sh ./static/libtool --silent --tag=CC --mode=link gcc > -std=gnu99 -o hw hw.o -L/tmp/install/ompi.noromio/lib -lmpi > $ ldd hw > linux-vdso.so.1 => (0x7fff7530d000) > librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x7f0ed541e000) > libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x7f0ed521) > libscif.so.0 => /usr/lib64/libscif.so.0 (0x003b9c60) > libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x003ba560) > libdl.so.2 => /lib64/libdl.so.2 (0x003b9be0) > librt.so.1 => /lib64/librt.so.1 (0x003b9ca0) > libnsl.so.1 => /lib64/libnsl.so.1 (0x003bae20) > libutil.so.1 => /lib64/libutil.so.1 (0x003bac60) > libm.so.6 => /lib64/libm.so.6 (0x003b9ba0) > libpthread.so.0 => /lib64/libpthread.so.0 (0x003b9c20) > libc.so.6 => /lib64/libc.so.6 (0x003b9b60) > /lib64/ld-linux-x86-64.so.2 (0x003b9b20) > > > > > On 2014/08/05 7:56, Ralph Castain wrote: > > My thought was to post initially as a blocker, pending a discussion with > Jeff at tomorrow's telecon. If he thinks this is something we can fix in some > central point (thus catching it everywhere), then it could be quick and worth > doing. However, I'm skeptical as I tried to do that in the most obvious > place, and it failed (could be operator error). > > Will let you know tomorrow. Truly appreciate your digging on this! > Ralph > > On Aug 4, 20
Re: [OMPI devel] [1.8.2rc3] static linking fails on linux when not building ROMIO
Here is a patch that has been minimally tested. this is likely an overkill (at least when dynamic libraries can be used), but it does the job so far ... Cheers, Gilles On 2014/08/05 16:56, Gilles Gouaillardet wrote: > from libopen-pal.la : > dependency_libs=' -lrdmacm -libverbs -lscif -lnuma -ldl -lrt -lnsl > -lutil -lm' > > > i confirm mpicc fails linking > > but FWIT, using libtool does work (!) > > could the bug come from the mpicc (and other) wrappers ? > > Gilles > > $ gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c > -I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath > -Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib > -Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte > -lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi > > $ /tmp/install/ompi.noromio/bin/mpicc -g -O0 -o hw -show ~/hw.c > gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c > -I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath > -Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib > -Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte > -lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi > [gouaillardet@soleil build]$ /tmp/install/ompi.noromio/bin/mpicc -g -O0 > -o hw ~/hw.c > /tmp/install/ompi.noromio/lib/libmpi.a(fbtl_posix_ipwritev.o): In > function `mca_fbtl_posix_ipwritev': > fbtl_posix_ipwritev.c:(.text+0x17b): undefined reference to `aio_write' > fbtl_posix_ipwritev.c:(.text+0x237): undefined reference to `aio_write' > fbtl_posix_ipwritev.c:(.text+0x3f4): undefined reference to `aio_write' > fbtl_posix_ipwritev.c:(.text+0x48e): undefined reference to `aio_write' > /tmp/install/ompi.noromio/lib/libopen-pal.a(opal_pty.o): In function > `opal_openpty': > opal_pty.c:(.text+0x1): undefined reference to `openpty' > /tmp/install/ompi.noromio/lib/libopen-pal.a(event.o): In function > `event_add_internal': > event.c:(.text+0x288d): undefined reference to `clock_gettime' > > $ /bin/sh ./static/libtool --silent --tag=CC --mode=compile gcc > -std=gnu99 -I/tmp/install/ompi.noromio/include -c ~/hw.c > $ /bin/sh ./static/libtool --silent --tag=CC --mode=link gcc > -std=gnu99 -o hw hw.o -L/tmp/install/ompi.noromio/lib -lmpi > $ ldd hw > linux-vdso.so.1 => (0x7fff7530d000) > librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x7f0ed541e000) > libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x7f0ed521) > libscif.so.0 => /usr/lib64/libscif.so.0 (0x003b9c60) > libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x003ba560) > libdl.so.2 => /lib64/libdl.so.2 (0x003b9be0) > librt.so.1 => /lib64/librt.so.1 (0x003b9ca0) > libnsl.so.1 => /lib64/libnsl.so.1 (0x003bae20) > libutil.so.1 => /lib64/libutil.so.1 (0x003bac60) > libm.so.6 => /lib64/libm.so.6 (0x003b9ba0) > libpthread.so.0 => /lib64/libpthread.so.0 (0x003b9c20) > libc.so.6 => /lib64/libc.so.6 (0x003b9b60) > /lib64/ld-linux-x86-64.so.2 (0x003b9b20) > > > > > On 2014/08/05 7:56, Ralph Castain wrote: >> My thought was to post initially as a blocker, pending a discussion with >> Jeff at tomorrow's telecon. If he thinks this is something we can fix in >> some central point (thus catching it everywhere), then it could be quick and >> worth doing. However, I'm skeptical as I tried to do that in the most >> obvious place, and it failed (could be operator error). >> >> Will let you know tomorrow. Truly appreciate your digging on this! >> Ralph >> >> On Aug 4, 2014, at 3:50 PM, Paul Hargrove wrote: >> >>> Ralph and Jeff, >>> >>> I've been digging and find the problem is wider than just the one library >>> and has manifestations specific to FreeBSD, NetBSD and Solaris. I am >>> adding new info to the ticket as I unearth it. >>> >>> Additionally, it appears this existed in 1.8, 1.8.1 and in the 1.7 series >>> as well. >>> So, would suggest this NOT be a blocker for a 1.8.2 release. >>> >>> Of course I am willing to provide testing if you still want to push for a >>> quick resolution. >>> >>> -Paul >>> >>> >>> On Mon, Aug 4, 2014 at 1:27 PM, Ralph Castain wrote: >>> Okay, I filed a blocker on this for 1.8.2 and assigned it to Jeff. I took a >>> crack at fixing it, but came up short :-( >>> >>> >>> On Aug 3, 2014, at 10:46 PM, Paul Hargrove wrote: >>> I've identified the difference between the platform that does link libutil and the one that does not. 1) libutil is linked (as an OMPI dependency) only on the working system: Working system: $ grep 'checking for .* LIBS' configure.out checking for OPAL LIBS... -lm -lpciaccess -ldl checking for ORTE LIBS... -lm -lpciaccess -ldl -ltorque checking for OMPI LIBS... -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil NON-working system: $ grep 'checking for .* LIBS' configure.out checking for OPAL LIBS... -lm -ldl checking for ORTE L
Re: [OMPI devel] [1.8.2rc3] static linking fails on linux when not building ROMIO
from libopen-pal.la : dependency_libs=' -lrdmacm -libverbs -lscif -lnuma -ldl -lrt -lnsl -lutil -lm' i confirm mpicc fails linking but FWIT, using libtool does work (!) could the bug come from the mpicc (and other) wrappers ? Gilles $ gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c -I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath -Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib -Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte -lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi $ /tmp/install/ompi.noromio/bin/mpicc -g -O0 -o hw -show ~/hw.c gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c -I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath -Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib -Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte -lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi [gouaillardet@soleil build]$ /tmp/install/ompi.noromio/bin/mpicc -g -O0 -o hw ~/hw.c /tmp/install/ompi.noromio/lib/libmpi.a(fbtl_posix_ipwritev.o): In function `mca_fbtl_posix_ipwritev': fbtl_posix_ipwritev.c:(.text+0x17b): undefined reference to `aio_write' fbtl_posix_ipwritev.c:(.text+0x237): undefined reference to `aio_write' fbtl_posix_ipwritev.c:(.text+0x3f4): undefined reference to `aio_write' fbtl_posix_ipwritev.c:(.text+0x48e): undefined reference to `aio_write' /tmp/install/ompi.noromio/lib/libopen-pal.a(opal_pty.o): In function `opal_openpty': opal_pty.c:(.text+0x1): undefined reference to `openpty' /tmp/install/ompi.noromio/lib/libopen-pal.a(event.o): In function `event_add_internal': event.c:(.text+0x288d): undefined reference to `clock_gettime' $ /bin/sh ./static/libtool --silent --tag=CC --mode=compile gcc -std=gnu99 -I/tmp/install/ompi.noromio/include -c ~/hw.c $ /bin/sh ./static/libtool --silent --tag=CC --mode=link gcc -std=gnu99 -o hw hw.o -L/tmp/install/ompi.noromio/lib -lmpi $ ldd hw linux-vdso.so.1 => (0x7fff7530d000) librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x7f0ed541e000) libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x7f0ed521) libscif.so.0 => /usr/lib64/libscif.so.0 (0x003b9c60) libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x003ba560) libdl.so.2 => /lib64/libdl.so.2 (0x003b9be0) librt.so.1 => /lib64/librt.so.1 (0x003b9ca0) libnsl.so.1 => /lib64/libnsl.so.1 (0x003bae20) libutil.so.1 => /lib64/libutil.so.1 (0x003bac60) libm.so.6 => /lib64/libm.so.6 (0x003b9ba0) libpthread.so.0 => /lib64/libpthread.so.0 (0x003b9c20) libc.so.6 => /lib64/libc.so.6 (0x003b9b60) /lib64/ld-linux-x86-64.so.2 (0x003b9b20) On 2014/08/05 7:56, Ralph Castain wrote: > My thought was to post initially as a blocker, pending a discussion with Jeff > at tomorrow's telecon. If he thinks this is something we can fix in some > central point (thus catching it everywhere), then it could be quick and worth > doing. However, I'm skeptical as I tried to do that in the most obvious > place, and it failed (could be operator error). > > Will let you know tomorrow. Truly appreciate your digging on this! > Ralph > > On Aug 4, 2014, at 3:50 PM, Paul Hargrove wrote: > >> Ralph and Jeff, >> >> I've been digging and find the problem is wider than just the one library >> and has manifestations specific to FreeBSD, NetBSD and Solaris. I am adding >> new info to the ticket as I unearth it. >> >> Additionally, it appears this existed in 1.8, 1.8.1 and in the 1.7 series as >> well. >> So, would suggest this NOT be a blocker for a 1.8.2 release. >> >> Of course I am willing to provide testing if you still want to push for a >> quick resolution. >> >> -Paul >> >> >> On Mon, Aug 4, 2014 at 1:27 PM, Ralph Castain wrote: >> Okay, I filed a blocker on this for 1.8.2 and assigned it to Jeff. I took a >> crack at fixing it, but came up short :-( >> >> >> On Aug 3, 2014, at 10:46 PM, Paul Hargrove wrote: >> >>> I've identified the difference between the platform that does link libutil >>> and the one that does not. >>> >>> 1) libutil is linked (as an OMPI dependency) only on the working system: >>> >>> Working system: >>> $ grep 'checking for .* LIBS' configure.out >>> checking for OPAL LIBS... -lm -lpciaccess -ldl >>> checking for ORTE LIBS... -lm -lpciaccess -ldl -ltorque >>> checking for OMPI LIBS... -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil >>> >>> NON-working system: >>> $ grep 'checking for .* LIBS' configure.out >>> checking for OPAL LIBS... -lm -ldl >>> checking for ORTE LIBS... -lm -ldl -ltorque >>> checking for OMPI LIBS... -lm -ldl -ltorque >>> >>> So, the working system that does link libutil is doing so as an OMPI >>> dependency. >>> However it is also needed for opal (only caller of openpty is >>> opal/util/open_pty.c). >>> >>> 2) Only the working system is building ROMIO: >>> >>> Comparing the 'checking if * can compile' lines of configu
Re: [OMPI devel] [1.8.2rc3] static linking fails on linux when not building ROMIO
My thought was to post initially as a blocker, pending a discussion with Jeff at tomorrow's telecon. If he thinks this is something we can fix in some central point (thus catching it everywhere), then it could be quick and worth doing. However, I'm skeptical as I tried to do that in the most obvious place, and it failed (could be operator error). Will let you know tomorrow. Truly appreciate your digging on this! Ralph On Aug 4, 2014, at 3:50 PM, Paul Hargrove wrote: > Ralph and Jeff, > > I've been digging and find the problem is wider than just the one library and > has manifestations specific to FreeBSD, NetBSD and Solaris. I am adding new > info to the ticket as I unearth it. > > Additionally, it appears this existed in 1.8, 1.8.1 and in the 1.7 series as > well. > So, would suggest this NOT be a blocker for a 1.8.2 release. > > Of course I am willing to provide testing if you still want to push for a > quick resolution. > > -Paul > > > On Mon, Aug 4, 2014 at 1:27 PM, Ralph Castain wrote: > Okay, I filed a blocker on this for 1.8.2 and assigned it to Jeff. I took a > crack at fixing it, but came up short :-( > > > On Aug 3, 2014, at 10:46 PM, Paul Hargrove wrote: > >> I've identified the difference between the platform that does link libutil >> and the one that does not. >> >> 1) libutil is linked (as an OMPI dependency) only on the working system: >> >> Working system: >> $ grep 'checking for .* LIBS' configure.out >> checking for OPAL LIBS... -lm -lpciaccess -ldl >> checking for ORTE LIBS... -lm -lpciaccess -ldl -ltorque >> checking for OMPI LIBS... -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil >> >> NON-working system: >> $ grep 'checking for .* LIBS' configure.out >> checking for OPAL LIBS... -lm -ldl >> checking for ORTE LIBS... -lm -ldl -ltorque >> checking for OMPI LIBS... -lm -ldl -ltorque >> >> So, the working system that does link libutil is doing so as an OMPI >> dependency. >> However it is also needed for opal (only caller of openpty is >> opal/util/open_pty.c). >> >> 2) Only the working system is building ROMIO: >> >> Comparing the 'checking if * can compile' lines of configure output shows >> only ONE difference: >> >> checking if MCA component fs:ufs can compile... yes >> checking if MCA component fs:pvfs2 can compile... no >> checking if MCA component io:ompio can compile... yes >> -checking if MCA component io:romio can compile... no >> +checking if MCA component io:romio can compile... yes >> checking if MCA component mpool:grdma can compile... yes >> checking if MCA component mpool:sm can compile... yes >> checking if MCA component mpool:udreg can compile... no >> >> So, it appears that *if* ROMIO is configured in, then "-lutil" gets added to >> OMPI_WRAPPER_EXTRA_LIBS. >> This masks the fact that it is missing from OPAL_WRAPPER_EXTRA_LIBS. >> >> >> I have confirmed that I can reproduce the static linking failure by adding >> --disable-io-romio to the configure options of the system that worked >> previously. >> >> So, I update my report (and the email subject line) to: >>Static linking fails on Linux when not building ROMIO >> >> -Paul >> >> >> >> On Sun, Aug 3, 2014 at 6:22 PM, Paul Hargrove wrote: >> Hmm, >> >> On a different Linux/x86-64 host things work as expected with '-lutil' >> linked explicitly: >> >> $ ./INST/bin/mpicc -showme BLD/examples/hello_c.c >> pgcc BLD/examples/hello_c.c >> -I/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/include >> -L/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib >> -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath >> -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib >> -Wl,-rpath >> -Wl,/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib >> >> -L/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib >> -lmpi -lopen-rte -lopen-pal -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil >> >> Searching for relevant differences now... >> >> -Paul >> >> >> On Sun, Aug 3, 2014 at 4:58 PM, Paul Hargrove wrote: >> >> I've configured the 1.8.2rc3 tarball with "--enable-static --disable-shared" >> on a fairly standard Linux/x86-64 platform. While there are no problems on >> the same platform w/o these configure flags, with them I cannot link any >> application codes. >> >> $ mpicc -ghello_c.c -o hello_c >> /global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib/libopen-pal.a(opal_pty.o): >> In function `opal_openpty': >> opal_pty.c:(.text+0x1): undefined reference to `openpty' >> >> I checked "make openpty" and the manpage says to link with '-lutil'. >> The '-showme' does not show libutil: >> >> $ mpicc -showme hello_c.c >> gcc hello_c.c >> -I/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/include >> -pthread -L/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >> -Wl,/usr/syscom/opt
Re: [OMPI devel] [1.8.2rc3] static linking fails on linux when not building ROMIO
Ralph and Jeff, I've been digging and find the problem is wider than just the one library and has manifestations specific to FreeBSD, NetBSD and Solaris. I am adding new info to the ticket as I unearth it. Additionally, it appears this existed in 1.8, 1.8.1 and in the 1.7 series as well. So, would suggest this NOT be a blocker for a 1.8.2 release. Of course I am willing to provide testing if you still want to push for a quick resolution. -Paul On Mon, Aug 4, 2014 at 1:27 PM, Ralph Castain wrote: > Okay, I filed a blocker on this for 1.8.2 and assigned it to Jeff. I took > a crack at fixing it, but came up short :-( > > > On Aug 3, 2014, at 10:46 PM, Paul Hargrove wrote: > > I've identified the difference between the platform that does link libutil > and the one that does not. > > 1) libutil is linked (as an OMPI dependency) only on the working system: > > Working system: > $ grep 'checking for .* LIBS' configure.out > checking for OPAL LIBS... -lm -lpciaccess -ldl > checking for ORTE LIBS... -lm -lpciaccess -ldl -ltorque > checking for OMPI LIBS... -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil > > NON-working system: > $ grep 'checking for .* LIBS' configure.out > checking for OPAL LIBS... -lm -ldl > checking for ORTE LIBS... -lm -ldl -ltorque > checking for OMPI LIBS... -lm -ldl -ltorque > > So, the working system that does link libutil is doing so as an OMPI > dependency. > However it is also needed for opal (only caller of openpty is > opal/util/open_pty.c). > > 2) Only the working system is building ROMIO: > > Comparing the 'checking if * can compile' lines of configure output shows > only ONE difference: > > checking if MCA component fs:ufs can compile... yes > checking if MCA component fs:pvfs2 can compile... no > checking if MCA component io:ompio can compile... yes > -checking if MCA component io:romio can compile... no > +checking if MCA component io:romio can compile... yes > checking if MCA component mpool:grdma can compile... yes > checking if MCA component mpool:sm can compile... yes > checking if MCA component mpool:udreg can compile... no > > So, it appears that *if* ROMIO is configured in, then "-lutil" gets added > to OMPI_WRAPPER_EXTRA_LIBS. > This masks the fact that it is missing from OPAL_WRAPPER_EXTRA_LIBS. > > > I have confirmed that I can reproduce the static linking failure by adding > --disable-io-romio to the configure options of the system that worked > previously. > > So, I update my report (and the email subject line) to: >Static linking fails on Linux when not building ROMIO > > -Paul > > > > On Sun, Aug 3, 2014 at 6:22 PM, Paul Hargrove wrote: > >> Hmm, >> >> On a different Linux/x86-64 host things work as expected with '-lutil' >> linked explicitly: >> >> $ ./INST/bin/mpicc -showme BLD/examples/hello_c.c >> pgcc BLD/examples/hello_c.c >> -I/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/include >> -L/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib >> -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath >> -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib >> -Wl,-rpath >> -Wl,/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib >> -L/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib >> -lmpi -lopen-rte -lopen-pal -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil >> >> Searching for relevant differences now... >> >> -Paul >> >> >> On Sun, Aug 3, 2014 at 4:58 PM, Paul Hargrove wrote: >> >>> >>> I've configured the 1.8.2rc3 tarball with "--enable-static >>> --disable-shared" on a fairly standard Linux/x86-64 platform. While there >>> are no problems on the same platform w/o these configure flags, with them I >>> cannot link any application codes. >>> >>> $ mpicc -ghello_c.c -o hello_c >>> /global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib/libopen-pal.a(opal_pty.o): >>> In function `opal_openpty': >>> opal_pty.c:(.text+0x1): undefined reference to `openpty' >>> >>> I checked "make openpty" and the manpage says to link with '-lutil'. >>> The '-showme' does not show libutil: >>> >>> $ mpicc -showme hello_c.c >>> gcc hello_c.c >>> -I/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/include >>> -pthread -L/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >>> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >>> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >>> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >>> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >>> -Wl,/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib >>> -Wl,--enable-new-dtags >>> -L/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib >>> -lmpi -lopen-rte -lopen-pal -lm -ldl -ltorque -libverbs -lrdmacm >>> >>> >>> It looks like configure is doing the right thing on some level, but >>> failing to add '-lutil' to the appropriate list of libs >>
Re: [OMPI devel] [1.8.2rc3] static linking fails on linux when not building ROMIO
Okay, I filed a blocker on this for 1.8.2 and assigned it to Jeff. I took a crack at fixing it, but came up short :-( On Aug 3, 2014, at 10:46 PM, Paul Hargrove wrote: > I've identified the difference between the platform that does link libutil > and the one that does not. > > 1) libutil is linked (as an OMPI dependency) only on the working system: > > Working system: > $ grep 'checking for .* LIBS' configure.out > checking for OPAL LIBS... -lm -lpciaccess -ldl > checking for ORTE LIBS... -lm -lpciaccess -ldl -ltorque > checking for OMPI LIBS... -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil > > NON-working system: > $ grep 'checking for .* LIBS' configure.out > checking for OPAL LIBS... -lm -ldl > checking for ORTE LIBS... -lm -ldl -ltorque > checking for OMPI LIBS... -lm -ldl -ltorque > > So, the working system that does link libutil is doing so as an OMPI > dependency. > However it is also needed for opal (only caller of openpty is > opal/util/open_pty.c). > > 2) Only the working system is building ROMIO: > > Comparing the 'checking if * can compile' lines of configure output shows > only ONE difference: > > checking if MCA component fs:ufs can compile... yes > checking if MCA component fs:pvfs2 can compile... no > checking if MCA component io:ompio can compile... yes > -checking if MCA component io:romio can compile... no > +checking if MCA component io:romio can compile... yes > checking if MCA component mpool:grdma can compile... yes > checking if MCA component mpool:sm can compile... yes > checking if MCA component mpool:udreg can compile... no > > So, it appears that *if* ROMIO is configured in, then "-lutil" gets added to > OMPI_WRAPPER_EXTRA_LIBS. > This masks the fact that it is missing from OPAL_WRAPPER_EXTRA_LIBS. > > > I have confirmed that I can reproduce the static linking failure by adding > --disable-io-romio to the configure options of the system that worked > previously. > > So, I update my report (and the email subject line) to: >Static linking fails on Linux when not building ROMIO > > -Paul > > > > On Sun, Aug 3, 2014 at 6:22 PM, Paul Hargrove wrote: > Hmm, > > On a different Linux/x86-64 host things work as expected with '-lutil' linked > explicitly: > > $ ./INST/bin/mpicc -showme BLD/examples/hello_c.c > pgcc BLD/examples/hello_c.c > -I/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/include > -L/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib > -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath > -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib > -Wl,-rpath > -Wl,/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib > > -L/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib > -lmpi -lopen-rte -lopen-pal -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil > > Searching for relevant differences now... > > -Paul > > > On Sun, Aug 3, 2014 at 4:58 PM, Paul Hargrove wrote: > > I've configured the 1.8.2rc3 tarball with "--enable-static --disable-shared" > on a fairly standard Linux/x86-64 platform. While there are no problems on > the same platform w/o these configure flags, with them I cannot link any > application codes. > > $ mpicc -ghello_c.c -o hello_c > /global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib/libopen-pal.a(opal_pty.o): > In function `opal_openpty': > opal_pty.c:(.text+0x1): undefined reference to `openpty' > > I checked "make openpty" and the manpage says to link with '-lutil'. > The '-showme' does not show libutil: > > $ mpicc -showme hello_c.c > gcc hello_c.c > -I/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/include > -pthread -L/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath > -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath > -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath > -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath > -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath > -Wl,/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib > -Wl,--enable-new-dtags > -L/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib > -lmpi -lopen-rte -lopen-pal -lm -ldl -ltorque -libverbs -lrdmacm > > > It looks like configure is doing the right thing on some level, but failing > to add '-lutil' to the appropriate list of libs (OPAL_WRAPPER_EXTRA_LIBS?): > > > == Library and Function tests > > checking if we need -lutil for openpty... yes > checking for openpty... yes > > > -Paul > > -- > Paul H. Hargrove phhargr...@lbl.gov > Future Technologies Group > Computer and Data Sciences Department Tel: +1-510-495-2352 > Lawrence Berkeley National Laboratory Fax: +1-510-486-69
Re: [OMPI devel] [1.8.2rc3] static linking fails on linux when not building ROMIO
I've identified the difference between the platform that does link libutil and the one that does not. 1) libutil is linked (as an OMPI dependency) only on the working system: Working system: $ grep 'checking for .* LIBS' configure.out checking for OPAL LIBS... -lm -lpciaccess -ldl checking for ORTE LIBS... -lm -lpciaccess -ldl -ltorque checking for OMPI LIBS... -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil NON-working system: $ grep 'checking for .* LIBS' configure.out checking for OPAL LIBS... -lm -ldl checking for ORTE LIBS... -lm -ldl -ltorque checking for OMPI LIBS... -lm -ldl -ltorque So, the working system that does link libutil is doing so as an OMPI dependency. However it is also needed for opal (only caller of openpty is opal/util/open_pty.c). 2) Only the working system is building ROMIO: Comparing the 'checking if * can compile' lines of configure output shows only ONE difference: checking if MCA component fs:ufs can compile... yes checking if MCA component fs:pvfs2 can compile... no checking if MCA component io:ompio can compile... yes -checking if MCA component io:romio can compile... no +checking if MCA component io:romio can compile... yes checking if MCA component mpool:grdma can compile... yes checking if MCA component mpool:sm can compile... yes checking if MCA component mpool:udreg can compile... no So, it appears that *if* ROMIO is configured in, then "-lutil" gets added to OMPI_WRAPPER_EXTRA_LIBS. This masks the fact that it is missing from OPAL_WRAPPER_EXTRA_LIBS. I have confirmed that I can reproduce the static linking failure by adding --disable-io-romio to the configure options of the system that worked previously. So, I update my report (and the email subject line) to: Static linking fails on Linux when not building ROMIO -Paul On Sun, Aug 3, 2014 at 6:22 PM, Paul Hargrove wrote: > Hmm, > > On a different Linux/x86-64 host things work as expected with '-lutil' > linked explicitly: > > $ ./INST/bin/mpicc -showme BLD/examples/hello_c.c > pgcc BLD/examples/hello_c.c > -I/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/include > -L/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib > -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath > -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib > -Wl,-rpath > -Wl,/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib > -L/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib > -lmpi -lopen-rte -lopen-pal -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil > > Searching for relevant differences now... > > -Paul > > > On Sun, Aug 3, 2014 at 4:58 PM, Paul Hargrove wrote: > >> >> I've configured the 1.8.2rc3 tarball with "--enable-static >> --disable-shared" on a fairly standard Linux/x86-64 platform. While there >> are no problems on the same platform w/o these configure flags, with them I >> cannot link any application codes. >> >> $ mpicc -ghello_c.c -o hello_c >> /global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib/libopen-pal.a(opal_pty.o): >> In function `opal_openpty': >> opal_pty.c:(.text+0x1): undefined reference to `openpty' >> >> I checked "make openpty" and the manpage says to link with '-lutil'. >> The '-showme' does not show libutil: >> >> $ mpicc -showme hello_c.c >> gcc hello_c.c >> -I/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/include >> -pthread -L/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath >> -Wl,/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib >> -Wl,--enable-new-dtags >> -L/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib >> -lmpi -lopen-rte -lopen-pal -lm -ldl -ltorque -libverbs -lrdmacm >> >> >> It looks like configure is doing the right thing on some level, but >> failing to add '-lutil' to the appropriate list of libs >> (OPAL_WRAPPER_EXTRA_LIBS?): >> >> >> >> == Library and Function tests >> >> >> checking if we need -lutil for openpty... yes >> checking for openpty... yes >> >> >> -Paul >> >> -- >> Paul H. Hargrove phhargr...@lbl.gov >> Future Technologies Group >> Computer and Data Sciences Department Tel: +1-510-495-2352 >> Lawrence Berkeley National Laboratory Fax: +1-510-486-6900 >> > > > > -- > Paul H. Hargrove phhargr...@lbl.gov > Future Technologies Group > Computer and Data Sciences Department Tel: +1-510-495-2352 > Lawrence Berkeley National Laboratory Fax: +1-510-486-6900 > -- Paul H. Hargrove
