Re: [petsc-dev] Bug in DMPlexVecGetClosure/DMPlexVecRestoreClosure in Fortran

Tue, 06 Sep 2022 13:34:17 -0700

Hi,

Adding up to my previous email: looking at all calls to DMGetWorkArray / DMRestoreArray in DMPlexVecGetClosure/DMPlexVecRestoreClosure , I don’t understand why they don’t match. Here is a cleaned up lldb output from ex96. See how the mem argument in the second calls to DMGetWorkArray and DMRestoreArray (highlighted in black) don’t match while those in red and orange do.

Blaise



* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.2
frame #0: 0x000000010494f698 ex96`main(argc=5, argv=0x000000016b4b2788) at ex96.c:49:3
46
47 PetscCall(PetscPrintf(PETSC_COMM_WORLD, "Point %" PetscInt_FMT "\n", pStart + 1));
48 cval = NULL;
-> 49 PetscCall(DMPlexVecGetClosure(dm, section, v, pStart + 1, &clSize, &cval));
50 PetscCall(PetscPrintf(PETSC_COMM_WORLD, "clSize %" PetscInt_FMT "\n", clSize));
51 if (clSize > 0) PetscCall(PetscRealView(clSize, cval, PETSC_VIEWER_STDOUT_SELF));
52 PetscCall(DMPlexVecRestoreClosure(dm, section, v, pStart + 1, &clSize, &cval));
Target 0: (ex96) stopped.
(lldb) c

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 2.1
* frame #0: 0x00000001097e3c9c libpetsc.3.017.4.dylib`DMGetWorkArray(dm=0x000000015c0f0250, count=129, dtype=1275069445, mem=0x000000016b4b1cf8) at dm.c:1569:3
frame #1: 0x0000000109ccee28 libpetsc.3.017.4.dylib`DMPlexGetTransitiveClosure_Internal(dm=0x000000015c0f0250, p=1, ornt=0, useCone=PETSC_TRUE, numPoints=0x000000016b4b1eac, points=0x000000016b4b1ea0) at plex.c:3560:3
frame #2: 0x0000000109cd2f20 libpetsc.3.017.4.dylib`DMPlexGetTransitiveClosure(dm=0x000000015c0f0250, p=1, useCone=PETSC_TRUE, numPoints=0x000000016b4b1eac, points=0x000000016b4b1ea0) at plex.c:3650:3
frame #3: 0x0000000109ceb028 libpetsc.3.017.4.dylib`DMPlexGetCompressedClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, point=1, numPoints=0x000000016b4b1ff4, points=0x000000016b4b2010, clSec=0x000000016b4b2020, clPoints=0x000000016b4b2018, clp=0x000000016b4b2008) at plex.c:5555:5
frame #4: 0x0000000109cec6d0 libpetsc.3.017.4.dylib`DMPlexVecGetClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, v=0x000000015c09f050, point=1, csize=0x000000016b4b21c4, values=0x000000016b4b21c8) at plex.c:5741:3
frame #5: 0x000000010494f70c ex96`main(argc=5, argv=0x000000016b4b2788) at ex96.c:49:3
frame #6: 0x0000000104b9508c dyld`start + 520
(lldb) c
Process 64051 resuming
Command #2 'c' continued the target.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 2.1
* frame #0: 0x00000001097e3c9c libpetsc.3.017.4.dylib`DMGetWorkArray(dm=0x000000015c0f0250, count=86, dtype=1275069445, mem=0x000000016b4b1d00) at dm.c:1569:3
frame #1: 0x0000000109ccef20 libpetsc.3.017.4.dylib`DMPlexGetTransitiveClosure_Internal(dm=0x000000015c0f0250, p=1, ornt=0, useCone=PETSC_TRUE, numPoints=0x000000016b4b1eac, points=0x000000016b4b1ea0) at plex.c:3563:10
frame #2: 0x0000000109cd2f20 libpetsc.3.017.4.dylib`DMPlexGetTransitiveClosure(dm=0x000000015c0f0250, p=1, useCone=PETSC_TRUE, numPoints=0x000000016b4b1eac, points=0x000000016b4b1ea0) at plex.c:3650:3
frame #3: 0x0000000109ceb028 libpetsc.3.017.4.dylib`DMPlexGetCompressedClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, point=1, numPoints=0x000000016b4b1ff4, points=0x000000016b4b2010, clSec=0x000000016b4b2020, clPoints=0x000000016b4b2018, clp=0x000000016b4b2008) at plex.c:5555:5
frame #4: 0x0000000109cec6d0 libpetsc.3.017.4.dylib`DMPlexVecGetClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, v=0x000000015c09f050, point=1, csize=0x000000016b4b21c4, values=0x000000016b4b21c8) at plex.c:5741:3
frame #5: 0x000000010494f70c ex96`main(argc=5, argv=0x000000016b4b2788) at ex96.c:49:3
frame #6: 0x0000000104b9508c dyld`start + 520
(lldb) c
Process 64051 resuming
Command #2 'c' continued the target.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 3.1
* frame #0: 0x00000001097e4af0 libpetsc.3.017.4.dylib`DMRestoreWorkArray(dm=0x000000015c0f0250, count=129, dtype=1275069445, mem=0x000000016b4b1cf8) at dm.c:1617:3
frame #1: 0x0000000109ccf8b8 libpetsc.3.017.4.dylib`DMPlexGetTransitiveClosure_Internal(dm=0x000000015c0f0250, p=1, ornt=0, useCone=PETSC_TRUE, numPoints=0x000000016b4b1eac, points=0x000000016b4b1ea0) at plex.c:3612:3
frame #2: 0x0000000109cd2f20 libpetsc.3.017.4.dylib`DMPlexGetTransitiveClosure(dm=0x000000015c0f0250, p=1, useCone=PETSC_TRUE, numPoints=0x000000016b4b1eac, points=0x000000016b4b1ea0) at plex.c:3650:3
frame #3: 0x0000000109ceb028 libpetsc.3.017.4.dylib`DMPlexGetCompressedClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, point=1, numPoints=0x000000016b4b1ff4, points=0x000000016b4b2010, clSec=0x000000016b4b2020, clPoints=0x000000016b4b2018, clp=0x000000016b4b2008) at plex.c:5555:5
frame #4: 0x0000000109cec6d0 libpetsc.3.017.4.dylib`DMPlexVecGetClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, v=0x000000015c09f050, point=1, csize=0x000000016b4b21c4, values=0x000000016b4b21c8) at plex.c:5741:3
frame #5: 0x000000010494f70c ex96`main(argc=5, argv=0x000000016b4b2788) at ex96.c:49:3
frame #6: 0x0000000104b9508c dyld`start + 520
(lldb) c
Process 64051 resuming
Command #2 'c' continued the target.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 2.1
* frame #0: 0x00000001097e3c9c libpetsc.3.017.4.dylib`DMGetWorkArray(dm=0x000000015c0f0250, count=0, dtype=1275070475, mem=0x000000016b4b21c8) at dm.c:1569:3
frame #1: 0x0000000109cec938 libpetsc.3.017.4.dylib`DMPlexVecGetClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, v=0x000000015c09f050, point=1, csize=0x000000016b4b21c4, values=0x000000016b4b21c8) at plex.c:5755:12
frame #2: 0x000000010494f70c ex96`main(argc=5, argv=0x000000016b4b2788) at ex96.c:49:3
frame #3: 0x0000000104b9508c dyld`start + 520
(lldb) c
Process 64051 resuming
Command #2 'c' continued the target.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 3.1
* frame #0: 0x00000001097e4af0 libpetsc.3.017.4.dylib`DMRestoreWorkArray(dm=0x000000015c0f0250, count=0, dtype=1275069445, mem=0x000000016b4b2010) at dm.c:1617:3
frame #1: 0x0000000109cd3500 libpetsc.3.017.4.dylib`DMPlexRestoreTransitiveClosure(dm=0x000000015c0f0250, p=1, useCone=PETSC_TRUE, numPoints=0x000000016b4b1ff4, points=0x000000016b4b2010) at plex.c:3677:3
frame #2: 0x0000000109ceb728 libpetsc.3.017.4.dylib`DMPlexRestoreCompressedClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, point=1, numPoints=0x000000016b4b1ff4, points=0x000000016b4b2010, clSec=0x000000016b4b2020, clPoints=0x000000016b4b2018, clp=0x000000016b4b2008) at plex.c:5567:5
frame #3: 0x0000000109cecea0 libpetsc.3.017.4.dylib`DMPlexVecGetClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, v=0x000000015c09f050, point=1, csize=0x000000016b4b21c4, values=0x000000016b4b21c8) at plex.c:5767:3
frame #4: 0x000000010494f70c ex96`main(argc=5, argv=0x000000016b4b2788) at ex96.c:49:3
frame #5: 0x0000000104b9508c dyld`start + 520
(lldb) c
Process 64051 resuming
Command #2 'c' continued the target.
(lldb) clSize 0
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 3.1
* frame #0: 0x00000001097e4af0 libpetsc.3.017.4.dylib`DMRestoreWorkArray(dm=0x000000015c0f0250, count=0, dtype=1275070475, mem=0x000000016b4b21c8) at dm.c:1617:3
frame #1: 0x0000000109cf17f8 libpetsc.3.017.4.dylib`DMPlexVecRestoreClosure(dm=0x000000015c0f0250, section=0x000000015c09e650, v=0x000000015c09f050, point=1, csize=0x000000016b4b21c4, values=0x000000016b4b21c8) at plex.c:5876:3
frame #2: 0x000000010494f99c ex96`main(argc=5, argv=0x000000016b4b2788) at ex96.c:52:3
frame #3: 0x0000000104b9508c dyld`start + 520
(lldb) c




On Sep 5, 2022, at 11:34 AM, Blaise Bourdin <bour...@mcmaster.ca> wrote:

All,

I am running in a very strange issue with DMPlexVecGetClosure/DMPlexVecRestoreClosure in Fortran when passing a point such that the returned array should be of 0 size. The typical use case is a vector associated with cell-based unknowns, and requesting the values on the closure of a vertex.

I am attaching C and fortran90 examples. They define a Section with a single value at the first cell of a mesh, an associated Vec, then call DMPlexVecGetClosure/DMPlexVecRestoreClosure on the second element. The C example is fine, but the fortran one crashes when calling DMPlexVecRestoreClosure.

SiMini:tests (main)$ ./ex96f90 -i ${PETSC_DIR}/share/petsc/datafiles/meshes/SquareFaceSet.exo a
Point: 0
size(cval): 1
 0:  -1.0000e+00
Point: 1
size(cval): 0
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: Array was not checked out
[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.17.4-1175-g59be91c8676  GIT Date: 2022-09-01 12:22:09 -0400
[0]PETSC ERROR: ./ex96f90 on a monterey-gcc12.2-arm64-basic-g named SiMini.local by blaise Mon Sep  5 11:24:40 2022
[0]PETSC ERROR: Configure options --CFLAGS="-Wimplicit-function-declaration -Wunused -Wuninitialized" --FFLAGS="-ffree-line-length-none -fallow-argument-mismatch -Wunused -Wuninitialized" --download-chaco-1 --download-exodusii=1 --download-hdf5=1 --download-metis=1 --download-netcdf=1 --download-parmetis=1 --download-pnetcdf=1 --download-zlib=1 --with-debugging=1 --with-exodusii-fortran-bindings --with-shared-libraries=1 --with-x11=1 --download-ctetgen=1 --download-triangle=1 --download-p4est=1
[0]PETSC ERROR: #1 DMRestoreWorkArray() at /opt/HPC/petsc-main/src/dm/interface/dm.c:1628
[0]PETSC ERROR: #2 DMPlexVecRestoreClosure() at /opt/HPC/petsc-main/src/dm/impls/plex/plex.c:5876
[0]PETSC ERROR: #3 ex96f90.F90:62
Abort(73) on node 0 (rank 0 in comm 16): application called MPI_Abort(MPI_COMM_SELF, 73) - process 0

I get the same behaviour on multiple platforms / compilers.

I followed the whole calling tree for DMGetWorkArray / DMRestoreArray in C and Fortran and didn’t see anything of interest.
I tried to modify DMGetWorkArray / DMRestoreWorkArray to handle the case of a work array of length 0 differently, but only managed to break other plex examples.

A possible fix at eth application code level would be to first compute the size of the closure in the section and only call DMPlexVecGetClosure/DMPlexVecRestoreClosure when it is >0. This is a bit silly however, as this is precisely the first thing does in DMPlexVecGetClosure.

Any help would be MUCH appreciated.

Regards,
Blaise



—
Tier 1 Canada Research Chair in Mathematical and Computational Aspects of Solid Mechanics
Professor, Department of Mathematics & Statistics
Hamilton Hall room 409A, McMaster University
1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
https://www.math.mcmaster.ca/bourdin | +1 (905) 525 9140 ext. 27243

<ex96.c><ex96f90.F90>

— 
Tier 1 Canada Research Chair in Mathematical and Computational Aspects of Solid Mechanics
Professor, Department of Mathematics & Statistics
Hamilton Hall room 409A, McMaster University
1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada 
https://www.math.mcmaster.ca/bourdin | +1 (905) 525 9140 ext. 27243

Reply via email to