Orion,
PMIx 4.2.3 should be avoided since there is a bad interaction with Open MPI
4.1.x
(as reported in https://github.com/open-mpi/ompi/issues/10416 and
https://github.com/open-mpi/ompi/issues/11478)
The issue occurs with this specific version: 4.2.2 works fine and the bug
is expected to be fixed in 4.2.4
Cheers,
Gilles
On Sun, Mar 19, 2023 at 8:26 AM Orion Poplawski <or...@nwra.com> wrote:
> What are the current recommendations for compatible combinations of
> pmix/prrte/openmpi?
>
> I'm looking into updating each of these in Fedora and running into a
> couple of issues. Currently in Fedora Rawhide we have:
>
> - pmix 4.1.2
> - prrte 2.0.2
> - openmpi 4.1.5
>
> After updating to pmix 4.2.3 I see the following:
>
> - openmpi programs fail to run with:
>
> mca_base_component_repository_open: unable to open mca_pmix_ext3x:
> /usr/lib64/openmpi/lib/openmpi/mca_pmix_ext3x.so: undefined symbol:
> pmix_value_load (ignored)
> [[3260,0],0] ORTE_ERROR_LOG: Not found in file ess_hnp_module.c at line 320
> --------------------------------------------------------------------------
> It looks like orte_init failed for some reason; your parallel process is
> likely to abort. There are many reasons that a parallel process can
> fail during orte_init; some of which are due to configuration or
> environment problems. This failure appears to be an internal failure;
> here's some additional information (which may only be relevant to an
> Open MPI developer):
>
> opal_pmix_base_select failed
> --> Returned value Not found (-13) instead of ORTE_SUCCESS
>
>
> - prrte 2.0.2 fails to build with:
>
> make[2]: Entering directory
> '/builddir/build/BUILD/prrte-2.0.2/src/tools/prted'
> /bin/sh ../../../libtool --tag=CC --mode=link gcc
> -DPRTE_CONFIGURE_USER="\"mockbuild\""
> -DPRTE_CONFIGURE_HOST="\"2188cae5486f485888615d01f56cd6c9\""
> -DPRTE_CONFIGURE_DATE="\"Fri Jan 20 00:00:00 UTC 2023\""
> -DPRTE_BUILD_USER="\"$USER\""
> -DPRTE_BUILD_HOST="\"${HOSTNAME:-`(hostname || uname -n) | sed 1q`}\""
> -DPRTE_BUILD_DATE="\"`../../../config/getdate.sh`\""
> -DPRTE_BUILD_CFLAGS="\"-DNDEBUG -O2 -flto=auto -ffat-lto-objects
> -fexceptions -g -grecord-gcc-switches -pipe
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
> -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -finline-functions
> -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3
> -Wp,-D_GLIBCXX_ASSERTIONS\""
> -DPRTE_BUILD_CPPFLAGS="\"-iquote/builddir/build/BUILD/prrte-2.0.2
> -iquote/builddir/build/BUILD/prrte-2.0.2/src/include\""
> -DPRTE_BUILD_LDFLAGS="\"-Wl,-z,relro -Wl,--as-needed -Wl,-z,now
> -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1
> -specs=/usr/lib/rpm/redhat/redhat-package-notes\""
> -DPRTE_BUILD_LIBS="\"-lm -levent_core -levent_pthreads -lpmix -lhwloc\""
> -DPRTE_CC_ABSOLUTE="\"/usr/bin/gcc\"" -DPRTE_GREEK_VERSION="\"\""
> -DPRTE_REPO_REV="\"v2.0.1-8-gaa57929\"" -DPRTE_RELEASE_DATE="\"Feb 10,
> 2022\"" -DNDEBUG -O2 -flto=auto -ffat-lto-objects -fexceptions -g
> -grecord-gcc-switches -pipe
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
> -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -finline-functions
> -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3
> -Wp,-D_GLIBCXX_ASSERTIONS -Wl,-z,relro -Wl,--as-needed -Wl,-z,now
> -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1
> -specs=/usr/lib/rpm/redhat/redhat-package-notes -o prted prted.o
> ../../../src/libprrte.la -lm -levent_core -levent_pthreads -lpmix -lhwloc
> libtool: link: gcc -DPRTE_CONFIGURE_USER=\"mockbuild\"
> -DPRTE_CONFIGURE_HOST=\"2188cae5486f485888615d01f56cd6c9\"
> "-DPRTE_CONFIGURE_DATE=\"Fri Jan 20 00:00:00 UTC 2023\""
> -DPRTE_BUILD_USER=\"mockbuild\"
> -DPRTE_BUILD_HOST=\"2188cae5486f485888615d01f56cd6c9\"
> "-DPRTE_BUILD_DATE=\"Fri Jan 20 00:00:00 UTC 2023\""
> "-DPRTE_BUILD_CFLAGS=\"-DNDEBUG -O2 -flto=auto -ffat-lto-objects
> -fexceptions -g -grecord-gcc-switches -pipe
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
> -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -finline-functions
> -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3
> -Wp,-D_GLIBCXX_ASSERTIONS\""
> "-DPRTE_BUILD_CPPFLAGS=\"-iquote/builddir/build/BUILD/prrte-2.0.2
> -iquote/builddir/build/BUILD/prrte-2.0.2/src/include\""
> "-DPRTE_BUILD_LDFLAGS=\"-Wl,-z,relro -Wl,--as-needed -Wl,-z,now
> -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1
> -specs=/usr/lib/rpm/redhat/redhat-package-notes\""
> "-DPRTE_BUILD_LIBS=\"-lm -levent_core -levent_pthreads -lpmix -lhwloc\""
> -DPRTE_CC_ABSOLUTE=\"/usr/bin/gcc\" -DPRTE_GREEK_VERSION=\"\"
> -DPRTE_REPO_REV=\"v2.0.1-8-gaa57929\" "-DPRTE_RELEASE_DATE=\"Feb 10,
> 2022\"" -DNDEBUG -O2 -flto=auto -ffat-lto-objects -fexceptions -g
> -grecord-gcc-switches -pipe
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
> -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -finline-functions
> -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3
> -Wp,-D_GLIBCXX_ASSERTIONS -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z
> -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1
> -specs=/usr/lib/rpm/redhat/redhat-package-notes -o .libs/prted prted.o
> ../../../src/.libs/libprrte.so -lm -levent_core -levent_pthreads -lpmix
> -lhwloc
> make[2]: Leaving directory
> '/builddir/build/BUILD/prrte-2.0.2/src/tools/prted'
> /usr/bin/ld: ../../../src/.libs/libprrte.so: undefined reference to
> `pmix_argv_append_unique_nosize'
> /usr/bin/ld: ../../../src/.libs/libprrte.so: undefined reference to
> `PMIx_Byte_object_load'
> /usr/bin/ld: ../../../src/.libs/libprrte.so: undefined reference to
> `pmix_argv_join'
> collect2: error: ld returned 1 exit status
> make[2]: *** [Makefile:792: prted] Error 1
>
> So I'm guessing an update to prrte 3.0.0 is in order here. But will
> openmpi 4.1.5 work with prrte 3.0.0?
>
> I've also reported and discussed these issues here:
> https://github.com/openpmix/openpmix/issues/3022
>
> but figured it would be of interest to other openmpi packagers.
>
>
> --
> Orion Poplawski
> he/him/his - surely the least important thing about me
> IT Systems Manager 720-772-5637
> NWRA, Boulder/CoRA Office FAX: 303-415-9702
> 3380 Mitchell Lane or...@nwra.com
> Boulder, CO 80301 https://www.nwra.com/
>
> _______________________________________________
> ompi-packagers mailing list
> ompi-packagers@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/ompi-packagers
>
_______________________________________________
ompi-packagers mailing list
ompi-packagers@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/ompi-packagers
