Thanks a lot, Matt. I assemble a matrix using MatSetValuesLocal(), where the cell-to-local mapping comes from PetscSectionGetOffset().
With 2 MPI ranks, Valgrind (rank 0) reports uninitialized data being passed to MPI_Isend during matrix assembly (inside matstash.c). Two observations: - If I hard-code the local mapping to always be 1, the Valgrind warning disappears. - The warning occurs with PETSc *3.23.3*, but disappears when switching to *3.24.3* under the same setup. This suggests either uninitialized local indices/values on my side, or a change in PETSc between these versions affecting stash packing or index handling. Does this type of Valgrind warning usually point to user-provided local indices being uninitialized, and were there relevant changes in this area between 3.23.3 and 3.24.x? Thanks, Xiaodong ==683913== Syscall param write(buf) points to uninitialised byte(s) ==683913== at 0xCAD15A8: write (in /usr/lib64/libc-2.28.so) ==683913== by 0xBB9B0A9: MPIDI_CH3I_Sock_write (sock.c:2622) ==683913== by 0xBBA12DF: MPIDI_CH3_iStartMsg (ch3_istartmsg.c:68) ==683913== by 0xBB55F82: MPIDI_CH3_RndvSend (ch3u_rndv.c:48) ==683913== by 0xBB70905: MPID_Isend (mpid_isend.c:159) ==683913== by 0xB83EDB7: internal_Isend_c (isend.c:273) ==683913== by 0xB83EFE9: PMPI_Isend_c (isend.c:363) ==683913== by 0x82CAC1A: MatStashBTSSend_Private (matstash.c:793) ==683913== by 0x6AA2BF8: PetscCommBuildTwoSidedFReq_Reference (mpits.c:311) ==683913== by 0x6AA9DAD: PetscCommBuildTwoSidedFReq (mpits.c:515) ==683913== by 0x82CEB46: MatStashScatterBegin_BTS (matstash.c:920) ==683913== by 0x82C009D: MatStashScatterBegin_Private (matstash.c:440) ==683913== by 0x73E77BF: MatAssemblyBegin_MPIAIJ (mpiaij.c:768) ==683913== by 0x81F0777: MatAssemblyBegin (matrix.c:5807) On Tue, Jan 27, 2026 at 6:13 PM Matthew Knepley <[email protected]> wrote: > On Tue, Jan 27, 2026 at 3:34 PM neil liu <[email protected]> wrote: > >> Dear Petsc users and developers, >> >> I am exploring the mapping between local, partition and global dofs. >> The following is my pseudo code, dof2Partitionmapping denotes the mapping >> between the local dof (20 local dofs each tetrahedra) and partition dof. >> > > We usually say cell, local, and global dofs. > > >> Is this mapping determined by Petsc itself under the hood >> (PetscSectionGetOffset)? >> > > Plex just iterates over the points in the canonical numbering (cells, > vertices, faces, edges). You can change the iteration order using > > > https://urldefense.us/v3/__https://petsc.org/main/manualpages/PetscSection/PetscSectionSetPermutation/__;!!G_uCfscf7eWS!Z7m5oJFaTt_KCIUQJxJx-4SEoHIv5k1rWuDb3ClOur8MISuVMaOzs6WTo6lqZ2OWUBvBAKwK4hyKszJ1HUAjOw$ > > > You can use that, for example, to place all ghost dofs at the end. > > Thanks, > > Matt > > >> For now, I am coding this mapping (local to partition) myself just based >> on the edge and face number in the partition. It seems the results are >> reasonable. But with this kind of self-defined mapping, the owned dofs and >> ghost dofs are interleaved. Will this bring bad results for the >> communication of MatStash? >> >> Thanks, >> Xiaodong >> >> *1. set 2 dofs for each edge and 2 dofs for each edge face respectively* >> PetscSectionSetChart(s, faceStart, edgeEnd); >> PetscSectionSetDof(s, faceIndex, 2); >> PetscSectionSetFieldDof(s, faceIndex, 0, 1); >> PetscSectionSetDof(s, edgeIndex, 2); >> PetscSectionSetFieldDof(s, edgeIndex, 0, 1); >> PetscSectionsetup(s) >> >> *2. Create matrix based on DMPlex* >> DMSetMatType(dm, MATAIJ); >> DMCreateMatrix(dm, &A); >> >> *3. loop over elements to set local values for Matrix* >> MatSetValuesLocal(A, dof2Partitionmapping.size(), >> dof2Partitionmapping.data(), dof2Partitionmapping.size(), >> dof2Partitionmapping.data(), Matrix_Local.data(), ADD_VALUES); >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Z7m5oJFaTt_KCIUQJxJx-4SEoHIv5k1rWuDb3ClOur8MISuVMaOzs6WTo6lqZ2OWUBvBAKwK4hyKszJveN_t0Q$ > > <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Z7m5oJFaTt_KCIUQJxJx-4SEoHIv5k1rWuDb3ClOur8MISuVMaOzs6WTo6lqZ2OWUBvBAKwK4hyKszK-GUW0Jw$ > > >
