cc: Antonio On Tue, 14 Sep 2021, Matthew Knepley wrote:
> Okay, we have to send this to OpenMPI. Volunteers? > > Maybe we should note this in the FAQ, or installation, so we remember how > to fix it if someone else asks? > > Thanks, > > Matt > > On Tue, Sep 14, 2021 at 2:35 AM Stefano Zampini <stefano.zamp...@gmail.com> > wrote: > > > I can reproduce it even with OpenMPI 4.1.1 on a different machine (Ubuntu > > 18 + AMD milan + clang from AOCC) and it is definitely an OpenMPI bug in > > the vader BTL If I use tcp, everything runs smooth > > > > zampins@kanary:~/Devel/petsc$ cat > > /home/zampins/local/etc/openmpi-mca-params.conf | grep btl > > btl = tcp,self > > zampins@kanary:~/Devel/petsc$ make -f gmakefile.test > > vec_is_sf_tutorials-ex1_4 > > Using MAKEFLAGS: > > TEST arch-debug/tests/counts/vec_is_sf_tutorials-ex1_4.counts > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-create > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-create > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-dynamic > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-dynamic > > ok > > vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-allocate > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-allocate > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-create > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-create > > ok > > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-dynamic > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-dynamic > > ok > > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-allocate > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-allocate > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-create > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-create > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-dynamic > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-dynamic > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-allocate > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-allocate > > > > > > zampins@kanary:~/Devel/petsc$ cat > > /home/zampins/local/etc/openmpi-mca-params.conf | grep btl > > btl = vader,tcp,self > > zampins@kanary:~/Devel/petsc$ make -f gmakefile.test > > vec_is_sf_tutorials-ex1_4 > > Using MAKEFLAGS: > > TEST arch-debug/tests/counts/vec_is_sf_tutorials-ex1_4.counts > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-create > > not ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-create > > # Error code: 1 > > # 43,46c43,46 > > # < [0] 0: 4001 2000 2002 3002 4002 > > # < [1] 0: 1001 3000 > > # < [2] 0: 2001 4000 > > # < [3] 0: 3001 1000 > > # --- > > # > [0] 0: 2002 2146435072 2 2146435072 38736240 > > # > [1] 0: 3000 2146435072 > > # > [2] 0: 2001 2146435072 > > # > [3] 0: 3001 2146435072 > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-dynamic > > not ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-dynamic > > # Error code: 1 > > # 43,46c43,46 > > # < [0] 0: 4001 2000 2002 3002 4002 > > # < [1] 0: 1001 3000 > > # < [2] 0: 2001 4000 > > # < [3] 0: 3001 1000 > > # --- > > # > [0] 0: 2002 2146435072 2 2146435072 0 > > # > [1] 0: 3000 2146435072 > > # > [2] 0: 2001 2146435072 > > # > [3] 0: 3001 2146435072 > > ok > > vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-allocate > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-allocate > > # retrying > > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-create > > not ok > > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-create # > > Error code: 98 > > # [1]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > # [1]PETSC ERROR: General MPI error > > # [1]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank > > # [1]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > > # [1]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b > > GIT Date: 2021-09-13 14:01:22 +0000 > > # [1]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by > > zampins Tue Sep 14 09:31:42 2021 > > # [1]PETSC ERROR: [2]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > # [2]PETSC ERROR: General MPI error > > # [2]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank > > # [2]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > > # [2]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b > > GIT Date: 2021-09-13 14:01:22 +0000 > > # [2]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by > > zampins Tue Sep 14 09:31:42 2021 > > # [2]PETSC ERROR: Configure options > > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14 > > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1 > > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0 > > --with-hip-dir=/opt/rocm --with-hip=1 > > --with-hypre-dir=/home/zampins/local-petsc > > --with-kokkos-dir=/home/zampins/local-petsc > > --with-kokkos-kernels-dir=/home/zampins/local-petsc > > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include > > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]" > > HIPFLAGS=--amdgpu-target=gfx908 > > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug > > # [2]PETSC ERROR: #1 PetscSFGetGroups() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166 > > # [2]PETSC ERROR: #2 PetscSFSetUp_Window() at > > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708 > > # [2]PETSC ERROR: #3 PetscSFSetUp() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318 > > # [2]PETSC ERROR: #4 main() at > > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172 > > # [2]PETSC ERROR: PETSc Option Table entries: > > # [2]PETSC ERROR: -sf_type window > > # [2]PETSC ERROR: -sf_window_flavor create > > # [2]PETSC ERROR: -sf_window_sync active > > # [2]PETSC ERROR: -test_gather > > # [2]PETSC ERROR: ----------------End of Error Message -------send entire > > error message to petsc-ma...@mcs.anl.gov---------- > > # [3]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > # [3]PETSC ERROR: General MPI error > > # [3]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank > > # [3]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > > # [3]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b > > GIT Date: 2021-09-13 14:01:22 +0000 > > # [3]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by > > zampins Tue Sep 14 09:31:42 2021 > > # [3]PETSC ERROR: Configure options > > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14 > > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1 > > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0 > > --with-hip-dir=/opt/rocm --with-hip=1 > > --with-hypre-dir=/home/zampins/local-petsc > > --with-kokkos-dir=/home/zampins/local-petsc > > --with-kokkos-kernels-dir=/home/zampins/local-petsc > > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include > > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]" > > HIPFLAGS=--amdgpu-target=gfx908 > > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug > > # [3]PETSC ERROR: #1 PetscSFGetGroups() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166 > > # [3]PETSC ERROR: #2 PetscSFSetUp_Window() at > > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708 > > # [3]PETSC ERROR: #3 PetscSFSetUp() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318 > > # [3]PETSC ERROR: #4 main() at > > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172 > > # [3]PETSC ERROR: PETSc Option Table entries: > > # [3]PETSC ERROR: -sf_type window > > # [3]PETSC ERROR: -sf_window_flavor create > > # [3]PETSC ERROR: Configure options > > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14 > > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1 > > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0 > > --with-hip-dir=/opt/rocm --with-hip=1 > > --with-hypre-dir=/home/zampins/local-petsc > > --with-kokkos-dir=/home/zampins/local-petsc > > --with-kokkos-kernels-dir=/home/zampins/local-petsc > > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include > > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]" > > HIPFLAGS=--amdgpu-target=gfx908 > > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug > > # [1]PETSC ERROR: #1 PetscSFGetGroups() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166 > > # [1]PETSC ERROR: #2 PetscSFSetUp_Window() at > > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708 > > # [1]PETSC ERROR: #3 PetscSFSetUp() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318 > > # [1]PETSC ERROR: #4 main() at > > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172 > > # [1]PETSC ERROR: PETSc Option Table entries: > > # [1]PETSC ERROR: -sf_type window > > # [1]PETSC ERROR: -sf_window_flavor create > > # [1]PETSC ERROR: -sf_window_sync active > > # [1]PETSC ERROR: -test_gather > > # [1]PETSC ERROR: ----------------End of Error Message -------send entire > > error message to petsc-ma...@mcs.anl.gov---------- > > # -sf_window_sync active > > # [3]PETSC ERROR: -test_gather > > # [3]PETSC ERROR: ----------------End of Error Message -------send entire > > error message to petsc-ma...@mcs.anl.gov---------- > > # > > -------------------------------------------------------------------------- > > # MPI_ABORT was invoked on rank 2 in communicator MPI_COMM_WORLD > > # with errorcode 98. > > # > > # NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > > # You may or may not see output from other processes, depending on > > # exactly when Open MPI kills them. > > # > > -------------------------------------------------------------------------- > > # [kanary.kaust.edu.sa:115527] 2 more processes have sent help message > > help-mpi-api.txt / mpi-abort > > # [kanary.kaust.edu.sa:115527] Set MCA parameter > > "orte_base_help_aggregate" to 0 to see all help / error messages > > ok vec_is_sf_tutorials-ex1_4 # SKIP Command failed so no diff > > # retrying > > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-dynamic > > not ok > > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-dynamic # > > Error code: 98 > > # [1]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > # [1]PETSC ERROR: General MPI error > > # [1]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank > > # [1]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > > # [1]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b > > GIT Date: 2021-09-13 14:01:22 +0000 > > # [1]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by > > zampins Tue Sep 14 09:31:45 2021 > > # [1]PETSC ERROR: Configure options > > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14 > > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1 > > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0 > > --with-hip-dir=/opt/rocm --with-hip=1 > > --with-hypre-dir=/home/zampins/local-petsc > > --with-kokkos-dir=/home/zampins/local-petsc > > --with-kokkos-kernels-dir=/home/zampins/local-petsc > > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include > > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]" > > HIPFLAGS=--amdgpu-target=gfx908 > > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug > > # [1]PETSC ERROR: #1 PetscSFGetGroups() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166 > > # [1]PETSC ERROR: #2 PetscSFSetUp_Window() at > > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708 > > # [1]PETSC ERROR: #3 PetscSFSetUp() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318 > > # [1]PETSC ERROR: #4 main() at > > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172 > > # [1]PETSC ERROR: PETSc Option Table entries: > > # [1]PETSC ERROR: -sf_type window > > # [1]PETSC ERROR: [2]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > # [2]PETSC ERROR: General MPI error > > # [2]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank > > # [2]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > > # [2]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b > > GIT Date: 2021-09-13 14:01:22 +0000 > > # [2]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by > > zampins Tue Sep 14 09:31:45 2021 > > # [2]PETSC ERROR: Configure options > > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14 > > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1 > > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0 > > --with-hip-dir=/opt/rocm --with-hip=1 > > --with-hypre-dir=/home/zampins/local-petsc > > --with-kokkos-dir=/home/zampins/local-petsc > > --with-kokkos-kernels-dir=/home/zampins/local-petsc > > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include > > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]" > > HIPFLAGS=--amdgpu-target=gfx908 > > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug > > # [2]PETSC ERROR: #1 PetscSFGetGroups() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166 > > # [2]PETSC ERROR: #2 PetscSFSetUp_Window() at > > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708 > > # [2]PETSC ERROR: #3 PetscSFSetUp() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318 > > # [2]PETSC ERROR: #4 main() at > > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172 > > # [2]PETSC ERROR: PETSc Option Table entries: > > # [2]PETSC ERROR: -sf_type window > > # [2]PETSC ERROR: -sf_window_flavor dynamic > > # [2]PETSC ERROR: -sf_window_sync active > > # [2]PETSC ERROR: -test_gather > > # [2]PETSC ERROR: ----------------End of Error Message -------send entire > > error message to petsc-ma...@mcs.anl.gov---------- > > # [3]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > # [3]PETSC ERROR: General MPI error > > # [3]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank > > # [3]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > > # [3]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b > > GIT Date: 2021-09-13 14:01:22 +0000 > > # [3]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by > > zampins Tue Sep 14 09:31:45 2021 > > # [3]PETSC ERROR: Configure options > > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14 > > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1 > > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0 > > --with-hip-dir=/opt/rocm --with-hip=1 > > --with-hypre-dir=/home/zampins/local-petsc > > --with-kokkos-dir=/home/zampins/local-petsc > > --with-kokkos-kernels-dir=/home/zampins/local-petsc > > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include > > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]" > > HIPFLAGS=--amdgpu-target=gfx908 > > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug > > # [3]PETSC ERROR: #1 PetscSFGetGroups() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166 > > # [3]PETSC ERROR: #2 PetscSFSetUp_Window() at > > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708 > > # [3]PETSC ERROR: #3 PetscSFSetUp() at > > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318 > > # [3]PETSC ERROR: #4 main() at > > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172 > > # [3]PETSC ERROR: PETSc Option Table entries: > > # [3]PETSC ERROR: -sf_type window > > # [3]PETSC ERROR: -sf_window_flavor dynamic > > # [3]PETSC ERROR: -sf_window_sync active > > # [3]PETSC ERROR: -test_gather > > # [3]PETSC ERROR: ----------------End of Error Message -------send entire > > error message to petsc-ma...@mcs.anl.gov---------- > > # -sf_window_flavor dynamic > > # [1]PETSC ERROR: -sf_window_sync active > > # [1]PETSC ERROR: -test_gather > > # [1]PETSC ERROR: ----------------End of Error Message -------send entire > > error message to petsc-ma...@mcs.anl.gov---------- > > # > > -------------------------------------------------------------------------- > > # MPI_ABORT was invoked on rank 3 in communicator MPI_COMM_WORLD > > # with errorcode 98. > > # > > # NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > > # You may or may not see output from other processes, depending on > > # exactly when Open MPI kills them. > > # > > -------------------------------------------------------------------------- > > # [kanary.kaust.edu.sa:115572] 2 more processes have sent help message > > help-mpi-api.txt / mpi-abort > > # [kanary.kaust.edu.sa:115572] Set MCA parameter > > "orte_base_help_aggregate" to 0 to see all help / error messages > > ok vec_is_sf_tutorials-ex1_4 # SKIP Command failed so no diff > > ok > > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-allocate > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-allocate > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-create > > not ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-create > > # Error code: 1 > > # 43,46c43,46 > > # < [0] 0: 4001 2000 2002 3002 4002 > > # < [1] 0: 1001 3000 > > # < [2] 0: 2001 4000 > > # < [3] 0: 3001 1000 > > # --- > > # > [0] 0: 4002 2146435072 2 2146435072 34619728 > > # > [1] 0: 3000 2146435072 > > # > [2] 0: 4000 2146435072 > > # > [3] 0: 3001 2146435072 > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-dynamic > > not ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-dynamic > > # Error code: 1 > > # 43,46c43,46 > > # < [0] 0: 4001 2000 2002 3002 4002 > > # < [1] 0: 1001 3000 > > # < [2] 0: 2001 4000 > > # < [3] 0: 3001 1000 > > # --- > > # > [0] 0: 4002 2146435072 2 2146435072 0 > > # > [1] 0: 3000 2146435072 > > # > [2] 0: 4000 2146435072 > > # > [3] 0: 3001 2146435072 > > ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-allocate > > ok > > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-allocate > > > > > > > > Il giorno mar 14 set 2021 alle ore 07:44 Stefano Zampini < > > stefano.zamp...@gmail.com> ha scritto: > > > >> I'll see if I can reproduce > >> > >> Il Mar 14 Set 2021, 06:58 Junchao Zhang <junchao.zh...@gmail.com> ha > >> scritto: > >> > >>> Hi, Stefano, > >>> Ping you again to see if you want to resolve this problem before > >>> petsc-3.16 > >>> > >>> --Junchao Zhang > >>> > >>> > >>> On Sun, Sep 12, 2021 at 3:06 PM Antonio T. sagitter < > >>> sagit...@fedoraproject.org> wrote: > >>> > >>>> Unfortunately, it's not possible. I must use the OpenMPI provided by > >>>> Fedora build-system (these rpm builds of PETSc are for Fedora's > >>>> repositories), downloading external software is not permitted. > >>>> > >>>> On 9/12/21 21:10, Pierre Jolivet wrote: > >>>> > > >>>> >> On 12 Sep 2021, at 8:56 PM, Matthew Knepley <knep...@gmail.com > >>>> >> <mailto:knep...@gmail.com>> wrote: > >>>> >> > >>>> >> On Sun, Sep 12, 2021 at 2:49 PM Antonio T. sagitter > >>>> >> <sagit...@fedoraproject.org <mailto:sagit...@fedoraproject.org>> > >>>> wrote: > >>>> >> > >>>> >> Those attached are configure.log/make.log from a MPI build in > >>>> >> Fedora 34 > >>>> >> x86_64 where the error below occurred. > >>>> >> > >>>> >> > >>>> >> This is OpenMPI 4.1.0. Is that the only MPI you build? My first > >>>> >> inclination is that this is an MPI implementation bug. > >>>> >> > >>>> >> Junchao, do we have an OpenMPI build in the CI? > >>>> > > >>>> > config/examples/arch-ci-linux-cuda-double-64idx.py: > >>>> > '--download-openmpi=1', > >>>> > config/examples/arch-ci-linux-pkgs-dbg-ftn-interfaces.py: > >>>> > '--download-openmpi=1', > >>>> > config/examples/arch-ci-linux-pkgs-opt.py: '--download-openmpi=1', > >>>> > > >>>> > config/BuildSystem/config/packages/OpenMPI.py uses version 4.1.0 as > >>>> well. > >>>> > I’m not sure PETSc is to blame here Antonio. You may want to try to > >>>> > ditch the OpenMPI shipped by your packet manager and try > >>>> > --download-openmpi as well, just for a quick sanity check. > >>>> > > >>>> > Thanks, > >>>> > Pierre > >>>> > > >>>> > >>>> -- > >>>> --- > >>>> Antonio Trande > >>>> Fedora Project > >>>> mailto: sagit...@fedoraproject.org > >>>> GPG key: 0x29FBC85D7A51CC2F > >>>> GPG key server: https://keyserver1.pgp.com/ > >>>> > >>> > > > > -- > > Stefano > > > > >
