On Tue, 19 Apr 2011 09:18:44 -0600, Ethan Coon <ecoon at lanl.gov> wrote: > Can you shoot me your example? I run 1-node z direction both with and > without periodic BCs, and with and without ghost nodes, without a > valgrind complaint in DMDACreate3D.
See attached very simple test case. It only crashes for stencil width greater than 1 (the original support request needs stencil width of 2 for TVD FV methods), mpiexec -n 1 ./a.out # stencil width 2 in code [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Argument out of range! [0]PETSC ERROR: Negative index -100 at 0 location! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: 335e836c7798747e6242df554ae58fe4db9f803f HG Date: Fri Apr 15 16:04:07 2011 -0500 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./a.out on a ompi named kunyang by jed Tue Apr 19 21:22:07 2011 [0]PETSC ERROR: Libraries linked from /home/jed/petsc/ompi/lib [0]PETSC ERROR: Configure run at Sat Apr 16 13:36:39 2011 [0]PETSC ERROR: Configure options --download-blacs --download-chaco --download-hypre=externalpackages/hypre-2.7.0b.tar.gz --download-ml --download-mum ps --download-parms --download-scalapack --download-scotch --download-spai --download-spooles --download-sundials --download-superlu --download-superl u_dist --with-blas-lapack-dir=/usr --with-c2html --with-cholmod-dir=/usr --with-clib-autodetect=0 --with-cxxlib-autodetect=0 --with-fortranlib-autodet ect=0 --with-hdf5-dir=/usr --with-java-dir=/usr --with-lgrind --with-mpi-dir=/usr --with-parmetis-dir=/usr --with-shared-libraries --with-single-libra ry=0 --with-sowing --with-umfpack-dir=/usr -LIBS="-L/usr/lib/openmpi -lmpi_f90 -lmpi_f77 -lmpi_cxx -lstdc++ -lgfortran" PETSC_ARCH=ompi [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: VecScatterCheckIndices_Private() line 26 in /home/jed/petsc/src/vec/vec/utils/vscat.c [0]PETSC ERROR: VecScatterCreate() line 1106 in /home/jed/petsc/src/vec/vec/utils/vscat.c [0]PETSC ERROR: DMSetUp_DA_3D() line 953 in /home/jed/petsc/src/dm/impls/da/da3.c [0]PETSC ERROR: DMSetUp_DA() line 22 in /home/jed/petsc/src/dm/impls/da/dareg.c [0]PETSC ERROR: DMSetUp() line 189 in /home/jed/petsc/src/dm/interface/dm.c [0]PETSC ERROR: DMDACreate3d() line 1366 in /home/jed/petsc/src/dm/impls/da/da3.c [0]PETSC ERROR: main() line 14 in "unknowndirectory/"da3d.c -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 63. I still see Valgrind noise for stencil width of 1: mpiexec -n 2 valgind ./a.out # stencil width 1 in code ==26044== Invalid read of size 2 ==26044== at 0x4C26C2C: memcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==26044== by 0x71D8D01: opal_convertor_pack (in /usr/lib/openmpi/libmpi.so.1.0.1) ==26044== by 0x117600C8: ??? (in /usr/lib/openmpi/openmpi/mca_btl_sm.so) ==26044== by 0x10F421B7: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) ==26044== by 0x10F30C0C: ??? (in /usr/lib/openmpi/openmpi/mca_pml_ob1.so) ==26044== by 0x7127650: PMPI_Isend (in /usr/lib/openmpi/libmpi.so.1.0.1) ==26044== by 0x6C5A103: PetscGatherMessageLengths (mpimesg.c:127) ==26044== by 0x697A90A: VecScatterCreate_PtoS (vpscat.c:1623) ==26044== by 0x69564FD: VecScatterCreate (vscat.c:1334) ==26044== by 0x7CB339A: DMSetUp_DA_3D (da3.c:953) ==26044== by 0x7D044BF: DMSetUp_DA (dareg.c:22) ==26044== by 0x7C76A6C: DMSetUp (dm.c:189) ==26044== Address 0xdfeda02 is 2 bytes inside a block of size 8 alloc'd ==26044== at 0x4C23918: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==26044== by 0x6CC9DEF: PetscMallocAlign (mal.c:30) ==26044== by 0x697A5EA: VecScatterCreate_PtoS (vpscat.c:1603) ==26044== by 0x69564FD: VecScatterCreate (vscat.c:1334) ==26044== by 0x7CB339A: DMSetUp_DA_3D (da3.c:953) ==26044== by 0x7D044BF: DMSetUp_DA (dareg.c:22) ==26044== by 0x7C76A6C: DMSetUp (dm.c:189) ==26044== by 0x7CB6D8D: DMDACreate3d (da3.c:1366) ==26044== by 0x400938: main (in /home/jed/lang/c/petsc/a.out) -------------- next part -------------- A non-text attachment was scrubbed... Name: da3d.c Type: text/x-csrc Size: 600 bytes Desc: not available URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20110419/37a4a0b3/attachment.c>
