On Mon, Jan 22, 2024 at 2:26 PM Berend van Wachem <berend.vanwac...@ovgu.de> wrote:
> Dear Matt, > > The problem is that I haven't figured out how to write a polyhedral DMplex > in parallel. So, currently, I can write the Vec data > in parallel, but the cones for the cells/faces/edges/nodes for the mesh > from just one process to a file (after gathering the > DMplex to a single process). > Ah shoot. Can you send me a polyhedral mesh (or code to generate one) so I can fix the parallel write problem? Or maybe it is already an issue and I forgot? > From the restart, I can then read the cone information from one process > from the file, recreate the DMPlex, and then > redistribute it. In this scenario, the Vec data I read in (in parallel) > will not match the correct cells of the DMplex. Hence, I > need to put it in the right place afterwards. > Yes, then searching makes sense. You could call DMLocatePoints(), but maybe you are doing that. Thanks, Matt > Best, Berend. > > On 1/22/24 20:03, Matthew Knepley wrote: > > On Mon, Jan 22, 2024 at 1:57 PM Berend van Wachem < > berend.vanwac...@ovgu.de <mailto:berend.vanwac...@ovgu.de>> wrote: > > > > Dear Matt, > > > > Thanks for your quick response. > > I have a DMPlex with a polyhedral mesh, and have defined a number of > vectors with data at the cell center. I have generated > > data > > for a number of timesteps, and I write the data for each point to a > file together with the (x,y,z) co-ordinate of the cell > > center. > > > > When I want to do a restart from the DMPlex, I recreate the DMplex > with the polyhedral mesh, redistribute it, and for each cell > > center find the corresponding (x,y,z) co-ordinate and insert the > data that corresponds to it. This is quite expensive, as it > > means I need to compare doubles very often. > > > > But reading your response, this may not be a bad way of doing it? > > > > > > It always seems to be a game of "what do you want to assume?". I tend to > assume that I wrote the DM and Vec in the same order, > > so when I load them they match. This is how Firedrake I/O works, so that > you can load up on a different number of processes > > (https://arxiv.org/abs/2401.05868 <https://arxiv.org/abs/2401.05868>). > > > > So, are you writing a Vec, and then redistributing and writing another > Vec? In the scheme above, you would have to write both > > DMs. Are you trying to avoid this? > > > > Thanks, > > > > Matt > > > > Thanks, > > > > Berend. > > > > On 1/22/24 18:58, Matthew Knepley wrote: > > > On Mon, Jan 22, 2024 at 10:49 AM Berend van Wachem < > berend.vanwac...@ovgu.de <mailto:berend.vanwac...@ovgu.de> > > <mailto:berend.vanwac...@ovgu.de <mailto:berend.vanwac...@ovgu.de>>> > wrote: > > > > > > Dear Petsc-Team, > > > > > > Is there a good way to define a unique integer number in each > element > > > (e.g. a cell) of a DMPlex mesh, which is in the same location, > > > regardless of the number of processors or the distribution of > the mesh > > > over the processors? > > > > > > So, for instance, if I have a DMPlex box mesh, the > top-right-front > > > corner element (e.g. cell) will always have the same unique > number, > > > regardless of the number of processors the mesh is > distributed over? > > > > > > I want to be able to link the results I have achieved with a > mesh from > > > DMPlex on a certain number of cores to the same mesh from a > DMPlex on a > > > different number of cores. > > > > > > Of course, I could make a tree based on the distance of each > element to > > > a certain point (based on the X,Y,Z co-ordinates of the > element), and go > > > through this tree in the same way and define an integer based > on this, > > > but that seems rather cumbersome. > > > > > > > > > I think this is harder than it sounds. The distance will not work > because it can be very degenerate. > > > You could lexicographically sort the coordinates, but this is > hard in parallel. It is fine if you are willing > > > to gather everything on one process. You could put down a p4est, > use the Morton order to number them since this is stable > > for a > > > given refinement. And then within each box lexicographically sort > the centroids. This is definitely cumbersome, but I cannot > > > think of anything else. This also might have parallel problems > since you need to know how much overlap you need to fill > > each box. > > > > > > Thanks, > > > > > > Matt > > > > > > Thanks and best regards, Berend. > > > > > > -- > > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any > > results to > > > which their experiments lead. > > > -- Norbert Wiener > > > > > > https://www.cse.buffalo.edu/~knepley/ < > https://www.cse.buffalo.edu/~knepley/> < > http://www.cse.buffalo.edu/~knepley/ > > <http://www.cse.buffalo.edu/~knepley/>> > > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to > > which their experiments lead. > > -- Norbert Wiener > > > > https://www.cse.buffalo.edu/~knepley/ < > http://www.cse.buffalo.edu/~knepley/> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>