Thanks a lot, Pierre. It works now. Another question is, with the present strategy, after the adapting, we will get a pseudo DM object, which has all information on rank 0 and nothing on all other ranks. Then I tried to use DMPlexdistribute to partition it and the partitioned DMs seem correct. Is it safe to do things like this?
Thanks, Xiaodong On Wed, Apr 23, 2025 at 4:33 PM Pierre Jolivet <pie...@joliv.et> wrote: > > > On 23 Apr 2025, at 7:28 PM, neil liu <liufi...@gmail.com> wrote: > > > > *MMG only supports serial execution, whereas ParMMG supports parallel mode > (although ParMMG is not as robust or mature as MMG).* > Given this, could you please provide some guidance on how to handle this > in the code? > > Here are my current thoughts; please let know whether it could work as > a temporary solution. > > That could work, > Pierre > > We may only need to make minor modifications in the > DMAdaptMetric_Mmg_Plex() subroutine. Specifically: > > - > > Allow all *collective PETSc functions* to run across all ranks as > usual. > - > > Restrict the *MMG-specific logic* to run *only on rank 0*, since MMG > is serial-only. > - > > Add a check before MMG is called to ensure that *only rank 0 holds > mesh cells*, i.e., validate that cEnd - cStart > 0 only on rank 0. If > more than one rank holds cells, raise a clear warning or error. > > > On Wed, Apr 23, 2025 at 1:11 PM Stefano Zampini <stefano.zamp...@gmail.com> > wrote: > >> If mmg does not support parallel communicators, we should handle it >> internally in the code, always use commself, and raise an error if there >> are two or more processes in the comm that have cEnd - cStart > 0 >> >> Il giorno mer 23 apr 2025 alle ore 20:05 neil liu <liufi...@gmail.com> >> ha scritto: >> >>> Thanks a lot. Pierre. >>> Do you have any suggestions to build a real serial DM from this >>> gatherDM? >>> I tried several ways, which don't work. >>> DMClone? >>> >>> Thanks, >>> >>> On Wed, Apr 23, 2025 at 11:39 AM Pierre Jolivet <pie...@joliv.et> wrote: >>> >>>> >>>> >>>> On 23 Apr 2025, at 5:31 PM, neil liu <liufi...@gmail.com> wrote: >>>> >>>> Thanks a lot, Stefano. >>>> I tried DMPlexGetGatherDM and DMPlexDistributeField. It can give what >>>> we expected. >>>> The final gatherDM is listed as follows, rank 0 has all information >>>> (which is right) while rank 1 has nothing. >>>> Then I tried to feed this gatherDM into adaptMMG on rank 0 only (it >>>> seems MMG works better than ParMMG, that is why I want MMG to be tried >>>> first). But it was stuck at collective petsc functions >>>> in DMAdaptMetric_Mmg_Plex(). By the way, the present work can work well >>>> with 1 rank. >>>> >>>> Do you have any suggestions ? Build a real serial DM? >>>> >>>> >>>> Yes, you need to change the underlying MPI_Comm as well, but I’m not >>>> sure if there is any user-facing API for doing this with a one-liner. >>>> >>>> Thanks, >>>> Pierre >>>> >>>> Thanks a lot. >>>> Xiaodong >>>> >>>> DM Object: Parallel Mesh 2 MPI processes >>>> type: plex >>>> Parallel Mesh in 3 dimensions: >>>> Number of 0-cells per rank: 56 0 >>>> Number of 1-cells per rank: 289 0 >>>> Number of 2-cells per rank: 452 0 >>>> Number of 3-cells per rank: 216 0 >>>> Labels: >>>> depth: 4 strata with value/size (0 (56), 1 (289), 2 (452), 3 (216)) >>>> celltype: 4 strata with value/size (0 (56), 1 (289), 3 (452), 6 (216)) >>>> Cell Sets: 2 strata with value/size (29 (152), 30 (64)) >>>> Face Sets: 3 strata with value/size (27 (8), 28 (40), 101 (20)) >>>> Edge Sets: 1 strata with value/size (10 (10)) >>>> Vertex Sets: 5 strata with value/size (27 (2), 28 (6), 29 (2), 101 >>>> (4), 106 (4)) >>>> Field Field_0: >>>> adjacency FEM >>>> >>>> >>>> >>>> On Fri, Apr 18, 2025 at 10:09 AM Stefano Zampini < >>>> stefano.zamp...@gmail.com> wrote: >>>> >>>>> If you have a vector distributed on the original mesh, then you can >>>>> use the SF returned by DMPlexGetGatherDM and use that in a call to >>>>> DMPlexDistributeField >>>>> >>>>> Il giorno ven 18 apr 2025 alle ore 17:02 neil liu <liufi...@gmail.com> >>>>> ha scritto: >>>>> >>>>>> Dear PETSc developers and users, >>>>>> >>>>>> I am currently exploring the integration of MMG3D with PETSc. Since >>>>>> MMG3D supports only serial execution, I am planning to combine parallel >>>>>> and >>>>>> serial computing in my workflow. Specifically, after solving the linear >>>>>> systems in parallel using PETSc: >>>>>> >>>>>> 1. >>>>>> >>>>>> I intend to use DMPlexGetGatherDM to collect the entire mesh on >>>>>> the root process for input to MMG3D. >>>>>> 2. >>>>>> >>>>>> Additionally, I plan to gather the error field onto the root >>>>>> process using VecScatter. >>>>>> >>>>>> However, I am concerned that the nth value in the gathered error >>>>>> vector (step 2) may not correspond to the nth element in the gathered >>>>>> mesh >>>>>> (step 1). Is this a valid concern? >>>>>> >>>>>> Do you have any suggestions or recommended practices for ensuring >>>>>> correct correspondence between the solution fields and the mesh when >>>>>> switching from parallel to serial mode? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Xiaodong >>>>>> >>>>> >>>>> >>>>> -- >>>>> Stefano >>>>> >>>> >>>> >> >> -- >> Stefano >> >
