Re: [OMPI users] openmpi/slurm/pmix
> On Apr 25, 2018, at 8:16 AM, Michael Di Domenico > wrote: > > On Mon, Apr 23, 2018 at 6:07 PM, r...@open-mpi.org wrote: >> Looks like the problem is that you didn’t wind up with the external PMIx. >> The component listed in your error is the internal PMIx one which shouldn’t >> have built given that configure line. >> >> Check your config.out and see what happened. Also, ensure that your >> LD_LIBRARY_PATH is properly pointing to the installation, and that you built >> into a “clean” prefix. > > the "clean prefix" part seemed to fix my issue. i'm not exactly sure > i understand why/how though. i recompiled pmix and removed the old > installation before doing a make install When you build, we don’t automatically purge the prefix location of any prior libraries. Thus, the old install of the internal PMIx library was still present. It has a higher priority than the external components, and so it was being picked up and used. Starting clean removed it, leaving the external component to be selected. > > when i recompiled openmpi it seems to have figured itself out > > i think things are still a little wonky, but at least that issue is gone > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] openmpi/slurm/pmix
On Mon, Apr 23, 2018 at 6:07 PM, r...@open-mpi.org wrote: > Looks like the problem is that you didn’t wind up with the external PMIx. The > component listed in your error is the internal PMIx one which shouldn’t have > built given that configure line. > > Check your config.out and see what happened. Also, ensure that your > LD_LIBRARY_PATH is properly pointing to the installation, and that you built > into a “clean” prefix. the "clean prefix" part seemed to fix my issue. i'm not exactly sure i understand why/how though. i recompiled pmix and removed the old installation before doing a make install when i recompiled openmpi it seems to have figured itself out i think things are still a little wonky, but at least that issue is gone ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] openmpi/slurm/pmix
Hi Gilles, Yes, I did. It was ignored AFAICT.I did not look for the reason - only so many hours in the day. Regards, Charlie > On Apr 24, 2018, at 8:07 AM, wrote: > > Charles, > > have you tried to configure --with-pmix-libdir=/.../lib64 ? > > Cheers, > > Gilles > > - Original Message - >> I´ll add that when building OpenMPI 3.0.0 with an external PMIx, I > found that the OpenMPI configure script only looks in “lib” for the the > pmix library but the pmix configure/build uses “lib64” (as it should on > a 64-bit system) so the configure script falls back to the internal PMIx. > As Robert suggested, check your config.log for “not found” messages. >> >> In my case, I simply added a “lib -> lib64” symlink in the PMIx > installation directory rather than alter the configure script and that > did the trick. >> >> Good luck, >> >> Charlie >> >>> On Apr 23, 2018, at 6:07 PM, r...@open-mpi.org wrote: >>> >>> Hi Michael >>> >>> Looks like the problem is that you didn´t wind up with the external > PMIx. The component listed in your error is the internal PMIx one which > shouldn´t have built given that configure line. >>> >>> Check your config.out and see what happened. Also, ensure that your > LD_LIBRARY_PATH is properly pointing to the installation, and that you > built into a “clean” prefix. >>> >>> On Apr 23, 2018, at 12:01 PM, Michael Di Domenico gmail.com> wrote: i'm trying to get slurm 17.11.5 and openmpi 3.0.1 working with pmix. everything compiled, but when i run something it get : symbol lookup error: /openmpi/mca_pmix_pmix2x.so: undefined > symbol: opal_libevent2022_evthread_use_pthreads i more then sure i did something wrong, but i'm not sure what, here > 's what i did compile libevent 2.1.8 ./configure --prefix=/libevent-2.1.8 compile pmix 2.1.0 ./configure --prefix=/pmix-2.1.0 --with-psm2 --with-munge=/munge-0.5.13 --with-libevent=/libevent-2.1.8 compile openmpi ./configure --prefix=/openmpi-3.0.1 --with-slurm=/slurm-17.11.5 --with-hwloc=external --with-mxm=/opt/mellanox/mxm --with-cuda=/usr/local/cuda --with-pmix=/pmix-2.1.0 --with-libevent=/libevent-2.1.8 when i look at the symbols in the mca_pmix_pmix2x.so library the function is indeed undefined (U) in the output, but checking ldd against the library doesn't show any missing any thoughts? ___ users mailing list users@lists.open-mpi.org https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.open-2Dmpi.org_mailman_listinfo_users&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=HOtXciFqK5GlgIgLAxthUQ&m=XE6hInyZVJ5VMrO5vdTEKEw3pZBBVnLE7U8Nm67zj2M&s=_sgJVrkRzlv7dIYMvtMfj26AJdbH-fcOOarmN7PyJCI&e= > >>> >>> ___ >>> users mailing list >>> users@lists.open-mpi.org >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.open-2Dmpi.org_mailman_listinfo_users&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=HOtXciFqK5GlgIgLAxthUQ&m=XE6hInyZVJ5VMrO5vdTEKEw3pZBBVnLE7U8Nm67zj2M&s=_sgJVrkRzlv7dIYMvtMfj26AJdbH-fcOOarmN7PyJCI&e= > >> >> ___ >> users mailing list >> users@lists.open-mpi.org >> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.open-2Dmpi.org_mailman_listinfo_users&d=DwIFag&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=HOtXciFqK5GlgIgLAxthUQ&m=0XUVnlQfzGhlRDSBAm8nGvZt27jITo3r1oX9_vg639w&s=ErD6RckR-Uvdpj4CTtNvT9iZck285Vdf6sgYskQ_Z-k&e= > ___ > users mailing list > users@lists.open-mpi.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.open-2Dmpi.org_mailman_listinfo_users&d=DwIFag&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=HOtXciFqK5GlgIgLAxthUQ&m=0XUVnlQfzGhlRDSBAm8nGvZt27jITo3r1oX9_vg639w&s=ErD6RckR-Uvdpj4CTtNvT9iZck285Vdf6sgYskQ_Z-k&e= ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] openmpi/slurm/pmix
Charles, have you tried to configure --with-pmix-libdir=/.../lib64 ? Cheers, Gilles - Original Message - > I’ll add that when building OpenMPI 3.0.0 with an external PMIx, I found that the OpenMPI configure script only looks in “lib” for the the pmix library but the pmix configure/build uses “lib64” (as it should on a 64-bit system) so the configure script falls back to the internal PMIx. As Robert suggested, check your config.log for “not found” messages. > > In my case, I simply added a “lib -> lib64” symlink in the PMIx installation directory rather than alter the configure script and that did the trick. > > Good luck, > > Charlie > > > On Apr 23, 2018, at 6:07 PM, r...@open-mpi.org wrote: > > > > Hi Michael > > > > Looks like the problem is that you didn’t wind up with the external PMIx. The component listed in your error is the internal PMIx one which shouldn’t have built given that configure line. > > > > Check your config.out and see what happened. Also, ensure that your LD_LIBRARY_PATH is properly pointing to the installation, and that you built into a “clean” prefix. > > > > > >> On Apr 23, 2018, at 12:01 PM, Michael Di Domenico wrote: > >> > >> i'm trying to get slurm 17.11.5 and openmpi 3.0.1 working with pmix. > >> > >> everything compiled, but when i run something it get > >> > >> : symbol lookup error: /openmpi/mca_pmix_pmix2x.so: undefined symbol: > >> opal_libevent2022_evthread_use_pthreads > >> > >> i more then sure i did something wrong, but i'm not sure what, here 's what i did > >> > >> compile libevent 2.1.8 > >> > >> ./configure --prefix=/libevent-2.1.8 > >> > >> compile pmix 2.1.0 > >> > >> ./configure --prefix=/pmix-2.1.0 --with-psm2 > >> --with-munge=/munge-0.5.13 --with-libevent=/libevent-2.1.8 > >> > >> compile openmpi > >> > >> ./configure --prefix=/openmpi-3.0.1 --with-slurm=/slurm-17.11.5 > >> --with-hwloc=external --with-mxm=/opt/mellanox/mxm > >> --with-cuda=/usr/local/cuda --with-pmix=/pmix-2.1.0 > >> --with-libevent=/libevent-2.1.8 > >> > >> when i look at the symbols in the mca_pmix_pmix2x.so library the > >> function is indeed undefined (U) in the output, but checking ldd > >> against the library doesn't show any missing > >> > >> any thoughts? > >> ___ > >> users mailing list > >> users@lists.open-mpi.org > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.open-2Dmpi.org_mailman_listinfo_users&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=HOtXciFqK5GlgIgLAxthUQ&m=XE6hInyZVJ5VMrO5vdTEKEw3pZBBVnLE7U8Nm67zj2M&s=_sgJVrkRzlv7dIYMvtMfj26AJdbH-fcOOarmN7PyJCI&e= > > > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.open-2Dmpi.org_mailman_listinfo_users&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=HOtXciFqK5GlgIgLAxthUQ&m=XE6hInyZVJ5VMrO5vdTEKEw3pZBBVnLE7U8Nm67zj2M&s=_sgJVrkRzlv7dIYMvtMfj26AJdbH-fcOOarmN7PyJCI&e= > > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] openmpi/slurm/pmix
I’ll add that when building OpenMPI 3.0.0 with an external PMIx, I found that the OpenMPI configure script only looks in “lib” for the the pmix library but the pmix configure/build uses “lib64” (as it should on a 64-bit system) so the configure script falls back to the internal PMIx. As Robert suggested, check your config.log for “not found” messages. In my case, I simply added a “lib -> lib64” symlink in the PMIx installation directory rather than alter the configure script and that did the trick. Good luck, Charlie > On Apr 23, 2018, at 6:07 PM, r...@open-mpi.org wrote: > > Hi Michael > > Looks like the problem is that you didn’t wind up with the external PMIx. The > component listed in your error is the internal PMIx one which shouldn’t have > built given that configure line. > > Check your config.out and see what happened. Also, ensure that your > LD_LIBRARY_PATH is properly pointing to the installation, and that you built > into a “clean” prefix. > > >> On Apr 23, 2018, at 12:01 PM, Michael Di Domenico >> wrote: >> >> i'm trying to get slurm 17.11.5 and openmpi 3.0.1 working with pmix. >> >> everything compiled, but when i run something it get >> >> : symbol lookup error: /openmpi/mca_pmix_pmix2x.so: undefined symbol: >> opal_libevent2022_evthread_use_pthreads >> >> i more then sure i did something wrong, but i'm not sure what, here's what i >> did >> >> compile libevent 2.1.8 >> >> ./configure --prefix=/libevent-2.1.8 >> >> compile pmix 2.1.0 >> >> ./configure --prefix=/pmix-2.1.0 --with-psm2 >> --with-munge=/munge-0.5.13 --with-libevent=/libevent-2.1.8 >> >> compile openmpi >> >> ./configure --prefix=/openmpi-3.0.1 --with-slurm=/slurm-17.11.5 >> --with-hwloc=external --with-mxm=/opt/mellanox/mxm >> --with-cuda=/usr/local/cuda --with-pmix=/pmix-2.1.0 >> --with-libevent=/libevent-2.1.8 >> >> when i look at the symbols in the mca_pmix_pmix2x.so library the >> function is indeed undefined (U) in the output, but checking ldd >> against the library doesn't show any missing >> >> any thoughts? >> ___ >> users mailing list >> users@lists.open-mpi.org >> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.open-2Dmpi.org_mailman_listinfo_users&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=HOtXciFqK5GlgIgLAxthUQ&m=XE6hInyZVJ5VMrO5vdTEKEw3pZBBVnLE7U8Nm67zj2M&s=_sgJVrkRzlv7dIYMvtMfj26AJdbH-fcOOarmN7PyJCI&e= > > ___ > users mailing list > users@lists.open-mpi.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.open-2Dmpi.org_mailman_listinfo_users&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=HOtXciFqK5GlgIgLAxthUQ&m=XE6hInyZVJ5VMrO5vdTEKEw3pZBBVnLE7U8Nm67zj2M&s=_sgJVrkRzlv7dIYMvtMfj26AJdbH-fcOOarmN7PyJCI&e= ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] openmpi/slurm/pmix
Hi Michael Looks like the problem is that you didn’t wind up with the external PMIx. The component listed in your error is the internal PMIx one which shouldn’t have built given that configure line. Check your config.out and see what happened. Also, ensure that your LD_LIBRARY_PATH is properly pointing to the installation, and that you built into a “clean” prefix. > On Apr 23, 2018, at 12:01 PM, Michael Di Domenico > wrote: > > i'm trying to get slurm 17.11.5 and openmpi 3.0.1 working with pmix. > > everything compiled, but when i run something it get > > : symbol lookup error: /openmpi/mca_pmix_pmix2x.so: undefined symbol: > opal_libevent2022_evthread_use_pthreads > > i more then sure i did something wrong, but i'm not sure what, here's what i > did > > compile libevent 2.1.8 > > ./configure --prefix=/libevent-2.1.8 > > compile pmix 2.1.0 > > ./configure --prefix=/pmix-2.1.0 --with-psm2 > --with-munge=/munge-0.5.13 --with-libevent=/libevent-2.1.8 > > compile openmpi > > ./configure --prefix=/openmpi-3.0.1 --with-slurm=/slurm-17.11.5 > --with-hwloc=external --with-mxm=/opt/mellanox/mxm > --with-cuda=/usr/local/cuda --with-pmix=/pmix-2.1.0 > --with-libevent=/libevent-2.1.8 > > when i look at the symbols in the mca_pmix_pmix2x.so library the > function is indeed undefined (U) in the output, but checking ldd > against the library doesn't show any missing > > any thoughts? > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
[OMPI users] openmpi/slurm/pmix
i'm trying to get slurm 17.11.5 and openmpi 3.0.1 working with pmix. everything compiled, but when i run something it get : symbol lookup error: /openmpi/mca_pmix_pmix2x.so: undefined symbol: opal_libevent2022_evthread_use_pthreads i more then sure i did something wrong, but i'm not sure what, here's what i did compile libevent 2.1.8 ./configure --prefix=/libevent-2.1.8 compile pmix 2.1.0 ./configure --prefix=/pmix-2.1.0 --with-psm2 --with-munge=/munge-0.5.13 --with-libevent=/libevent-2.1.8 compile openmpi ./configure --prefix=/openmpi-3.0.1 --with-slurm=/slurm-17.11.5 --with-hwloc=external --with-mxm=/opt/mellanox/mxm --with-cuda=/usr/local/cuda --with-pmix=/pmix-2.1.0 --with-libevent=/libevent-2.1.8 when i look at the symbols in the mca_pmix_pmix2x.so library the function is indeed undefined (U) in the output, but checking ldd against the library doesn't show any missing any thoughts? ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
