I think you want https://petsc.org/main/docs/manualpages/DMPLEX/DMPlexConstructGhostCells
Mark On Sat, Jul 2, 2022 at 6:20 PM Nicholas Arnold-Medabalimi < [email protected]> wrote: > Dear PETSc users, > > I have some general novice questions regarding how to properly access a > distributed DMPLEX vector that has ghost cells for a finite volume use case. > > My process for setup > 1) generated a simple box mesh using DMPlexCreateBoxMesh > 2) distribute the mesh using DMPlexDistribute (at this point everything > looks fine when I View (as paraview file) and I can see the partitions) > 3) I generate a section with 5 variables located at the cells. (I don't > think it matters if I use 5 fields or 1 field with 5 components?) I can > alternatively do this using the PetscFV variable with 5 components. > 4) I then have the dm setup properly (with no ghost cells at this point) > and I use DMCreateGlobalVector and DMCreateLocalVector to get the Vectors > to work with > 5) I'm just setting a quadratic IC. The overall loop over the cells is > constructed via getting the cell stratum which has the Cell point list and > then using the SectionOffset to assign each of the components at each cell > using DMPlexComputeCellGeometryFVM to get the cell centroid. > > As far as I can tell this is an adequate solution but now I'm moving into > learning how to access neighboring cells for flux calculations which isn't > just the local cell but requires access to neighboring cells which at some > point will be located on neighboring processors and synched to be > accessible. When I did this for a DMDA it was pretty straightforward > insofar as I just needed to call DMDAGetCorners and the LocalVector > indexing would allow the stencil to extend gracefully into the ghost > values. > > My general question is how do I achieve a similar loop structure for > DMPlex as far as the indexing over the owned cells while being able to > access the neighbor halo cells. I have been looking at example 52 and 11 in > the ts tutorial examples but I'm struggling to extract exactly what I need. > > To boil it down into 3 starting questions > 1) What is the preferred way to setup the cells and then access the > internal cells for say N cell halos for the boundaries between processors > in a DMPlex? > 2) What is the distinction between the overlap argument in > DMPlexDistribute and DMPlexConstructGhostCells (which I see used in the > examples)? (at least based on my tests it seems GhostCells is generating > cells at the overall boundaries not the internal partition boundaries?) > 3) Once I do have a DMPlex object that has the appropriate halos, how > would I get the access range for those cells? Based on my current > assumption the Stratum range for the Cells would include the ghost cells as > well which I would like to avoid? (Ultimately in the examples I see cases > where there is the synch step for Local to Global and vice versa but I'm > trying to just get to the point where I have the DM setup properly to > handle the internal mesh boundaries and have the correct indexing from the > StratumGet commands.) > > Any advice and corrections would be welcome. > > Sincerely > Nicholas > > -- > Nicholas Arnold-Medabalimi > >
