On Thu, Dec/25/2008 08:12:49AM, Jeff Squyres wrote: > It's quite possible that we don't handle this situation properly. Won't > you need to libdir's (one for the 32 bit OMPI executables, and one for the > 64 bit MPI apps)?
I don't need an OPAL environment variable for the executables, just a single OPAL_LIBDIR var for the libraries. (One set of 32-bit executables runs with both 32-bit and 64-bit libraries.) I'm guessing OPAL_LIBDIR will not work for you if you configure with a non-standard --libdir option. -Ethan > > On Dec 23, 2008, at 3:58 PM, Ethan Mallove wrote: > >> I think the problem is that I am doing a multi-lib build. I have >> 32-bit libraries in lib/, and 64-bit libraries in lib/64. I assume I >> do not see the issue for 32-bit tests, because all the dependencies >> are where Open MPI expects them to be. For the 64-bit case, I tried >> setting OPAL_LIBDIR to /opt/openmpi-relocated/lib/lib64, but no luck. >> Given the below configure arguments, what do my OPAL_* env vars need >> to be? (Also, could using --enable-orterun-prefix-by-default interfere >> with OPAL_PREFIX?) >> >> $ ./configure CC=cc CXX=CC F77=f77 FC=f90 --with-openib >> --without-udapl --disable-openib-ibcm --enable-heterogeneous >> --enable-cxx-exceptions --enable-shared --enable-orterun-prefix-by-default >> --with-sge --enable-mpi-f90 --with-mpi-f90-size=small >> --disable-mpi-threads --disable-progress-threads --disable-debug >> CFLAGS="-m32 -xO5" CXXFLAGS="-m32 -xO5" FFLAGS="-m32 -xO5" FCFLAGS="-m32 >> -xO5" >> --prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install >> >> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man >> >> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib >> >> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include >> >> --without-mx --with-tm=/ws/ompi-tools/orte/torque/current/shared-install32 >> --with-contrib-vt-flags="--prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v! >> 20z-12/ompi-tarball-testing/installs/DGQx/install >> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man >> >> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib >> >> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include >> >> LDFLAGS=-R/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib" >> >> $ ./confgiure CC=cc CXX=CC F77=f77 FC=f90 --with-openib >> --without-udapl --disable-openib-ibcm --enable-heterogeneous >> --enable-cxx-exceptions --enable-shared --enable-orterun-prefix-by-default >> --with-sge --enable-mpi-f90 --with-mpi-f90-size=small >> --disable-mpi-threads --disable-progress-threads --disable-debug >> CFLAGS="-m64 -xO5" CXXFLAGS="-m64 -xO5" FFLAGS="-m64 -xO5" FCFLAGS="-m64 >> -xO5" >> --prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install >> >> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man >> >> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib/lib64 >> >> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include/64 >> >> --without-mx --with-tm=/ws/ompi-tools/orte/torque/current/shared-install64 >> --with-contrib-vt-flags="--prefix=/workspace/em162155/hpc/mtt-scratch/! >> burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install >> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man >> >> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib/lib64 >> >> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include/64 >> >> LDFLAGS=-R/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib" >> >> --disable-binaries >> >> -Ethan >> >>> >>> >>> On Dec 22, 2008, at 12:42 PM, Ethan Mallove wrote: >>> >>>> Can anyone get OPAL_PREFIX to work on Linux? A simple test is to see >>>> if the following works for any mpicc/mpirun: >>>> >>>> $ mv <openmpi-installation> /tmp/foo >>>> $ set OPAL_PREFIX /tmp/foo >>>> $ mpicc ... >>>> $ mpirun ... >>>> >>>> If you are able to get the above to run successfully, I'm interested >>>> in your config.log file. >>>> >>>> Thanks, >>>> Ethan >>>> >>>> >>>> On Thu, Dec/18/2008 11:03:25AM, Ethan Mallove wrote: >>>>> Hello, >>>>> >>>>> The below FAQ lists instructions on how to use a relocated Open MPI >>>>> installation: >>>>> >>>>> http://www.open-mpi.org/faq/?category=building#installdirs >>>>> >>>>> On Solaris, OPAL_PREFIX and friends (documented in the FAQ) work for >>>>> me with both MPI (hello_c) and non-MPI (hostname) programs. On Linux, >>>>> I can only get the non-MPI case to work. Here are the environment >>>>> variables I am setting: >>>>> >>>>> $ cat setenv_opal_prefix.csh >>>>> set opal_prefix = "/opt/openmpi-relocated" >>>>> >>>>> setenv OPAL_PREFIX $opal_prefix >>>>> setenv OPAL_BINDIR $opal_prefix/bin >>>>> setenv OPAL_SBINDIR $opal_prefix/sbin >>>>> setenv OPAL_DATAROOTDIR $opal_prefix/share >>>>> setenv OPAL_SYSCONFDIR $opal_prefix/etc >>>>> setenv OPAL_SHAREDSTATEDIR $opal_prefix/com >>>>> setenv OPAL_LOCALSTATEDIR $opal_prefix/var >>>>> setenv OPAL_LIBDIR $opal_prefix/lib >>>>> setenv OPAL_INCLUDEDIR $opal_prefix/include >>>>> setenv OPAL_INFODIR $opal_prefix/info >>>>> setenv OPAL_MANDIR $opal_prefix/man >>>>> >>>>> setenv PATH $opal_prefix/bin:$PATH >>>>> setenv LD_LIBRARY_PATH $opal_prefix/lib:$opal_prefix/lib/64 >>>>> >>>>> Here is the error I get: >>>>> >>>>> $ mpirun -np 2 hello_c >>>>> >>>>> -------------------------------------------------------------------------- >>>>> It looks like opal_init failed for some reason; your parallel process >>>>> is >>>>> likely to abort. There are many reasons that a parallel process can >>>>> fail during opal_init; some of which are due to configuration or >>>>> environment problems. This failure appears to be an internal failure; >>>>> here's some additional information (which may only be relevant to an >>>>> Open MPI developer): >>>>> >>>>> opal_carto_base_select failed >>>>> --> Returned value -13 instead of OPAL_SUCCESS >>>>> >>>>> -------------------------------------------------------------------------- >>>>> [burl-ct-v20z-0:27737] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in >>>>> file runtime/orte_init.c at line 77 >>>>> >>>>> Any ideas on what's going on? >>>>> >>>>> Thanks, >>>>> Ethan >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>> >>> >>> -- >>> Jeff Squyres >>> Cisco Systems >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users > > > -- > Jeff Squyres > Cisco Systems > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
