Ok! Thanks for the reply. I'll take a look when I get a chance. -Colton
On Wed, Dec 13, 2023 at 10:48 AM Matthew Knepley <[email protected]> wrote: > On Wed, Dec 13, 2023 at 11:22 AM Colton Bryant < > [email protected]> wrote: > >> Hi, >> >> Thanks for the help last week. The suggestions made the implementation I >> had much cleaner. I had one follow up question. Is there a way to sort of >> undo this operation? I know the vec scatter can be done backwards to >> distribute the arrays but I didn't see an easy way to migrate the DMDA >> vectors back into the DMStag object. >> > > It is not there. However, writing it would be straightforward. I would > > 1) Expose DMStagMigrateVecDMDA(), which is not currently public > > 2) Add a ScatterMode argument > > 3) Put in code that calls DMStagSetValuesStencil(), rather than > GetValuesStencil() > > We would be happy to take on MR on this, and could help in the review. > > Thanks, > > MAtt > > >> Thanks for any advice. >> >> -Colton >> >> On Wed, Dec 6, 2023 at 8:18 PM Barry Smith <[email protected]> wrote: >> >>> >>> >>> On Dec 6, 2023, at 8:35 PM, Matthew Knepley <[email protected]> wrote: >>> >>> On Wed, Dec 6, 2023 at 8:10 PM Barry Smith <[email protected]> wrote: >>> >>>> >>>> Depending on the serial library you may not need to split the vector >>>> into DMDA vectors with DMStagVecSplitToDMDA() for each component. Just >>>> global to natural and scatter to zero on the full vector, now the full >>>> vector is on the first rank and you can access what you need in that one >>>> vector if possible. >>>> >>> >>> Does DMStag have a GlobalToNatural? >>> >>> >>> Good point, it does not appear to have such a thing, though it could. >>> >>> Also, the serial code would have to have identical interleaving. >>> >>> Thanks, >>> >>> Matt >>> >>>> On Dec 6, 2023, at 6:37 PM, Colton Bryant < >>>> [email protected]> wrote: >>>> >>>> Ah excellent! I was not aware of the ability to preallocate the objects >>>> and migrate them each time. >>>> >>>> Thanks! >>>> -Colton >>>> >>>> On Wed, Dec 6, 2023 at 5:18 PM Matthew Knepley <[email protected]> >>>> wrote: >>>> >>>>> On Wed, Dec 6, 2023 at 5:54 PM Colton Bryant < >>>>> [email protected]> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> I am working on a code in which a DMSTAG object is used to solve a >>>>>> fluid flow problem and I need to gather this flow data on a single >>>>>> process >>>>>> to interact with an existing (serial) library at each timestep of my >>>>>> simulation. After looking around the solution I've tried is: >>>>>> >>>>>> -use DMStagVecSplitToDMDA to extract vectors of each component of the >>>>>> flow >>>>>> -use DMDACreateNaturalVector and DMDAGlobalToNatural to get the >>>>>> components naturally ordered >>>>>> -use VecScatterCreateToZero to set up and then do the scatter to >>>>>> gather on the single process >>>>>> >>>>>> Unless I'm misunderstanding something this method results in a lot of >>>>>> memory allocation/freeing happening at each step of the evolution and I >>>>>> was >>>>>> wondering if there is a way to directly perform such a scatter from the >>>>>> DMSTAG object without splitting as I'm doing here. >>>>>> >>>>> >>>>> 1) You can see here: >>>>> >>>>> >>>>> https://petsc.org/main/src/dm/impls/stag/stagda.c.html#DMStagVecSplitToDMDA >>>>> >>>>> that this function is small. You can do the DMDA creation manually, >>>>> and then just call DMStagMigrateVecDMDA() each time, which will not >>>>> create anything. >>>>> >>>>> 2) You can create the natural vector upfront, and just scatter each >>>>> time. >>>>> >>>>> 3) You can create the serial vector upfront, and just scatter each >>>>> time. >>>>> >>>>> This is some data movement. You can compress the g2n and 2zero >>>>> scatters using >>>>> >>>>> https://petsc.org/main/manualpages/PetscSF/PetscSFCompose/ >>>>> >>>>> as an optimization. >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> >>>>>> Any advice would be much appreciated! >>>>>> >>>>>> Best, >>>>>> Colton Bryant >>>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> https://www.cse.buffalo.edu/~knepley/ >>>>> <http://www.cse.buffalo.edu/~knepley/> >>>>> >>>> >>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> <http://www.cse.buffalo.edu/~knepley/> >>> >>> >>> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> >
