Now it works, thanks a lot! Miguel
On Wed, Feb 25, 2015 at 10:12 AM, Matthew Knepley <[email protected]> wrote: > On Wed, Feb 25, 2015 at 9:59 AM, Abhyankar, Shrirang G. < > [email protected]> wrote: > >> Miguel, >> I'm a bit tied up today. I'll try to debug this issue tomorrow and get >> back to you. >> > > The problem is the way that DMNetwork is using the default section. When > DMCreateGlobalVec() is called, > it uses the default section for its included network->plex, but > DMSetDefaultSection() sets the section associated > with network. This is inconsistent. > > I am sending the code which I hacked to show the correct result by using > the private header. > > I think that > > 1) The underlying Plex should be exposed by DMNetworkGetPlex() > > 2) The default section business should be made consistent > > Thanks, > > Matt > > >> Thanks, >> Shri >> >> From: Miguel Angel Salazar de Troya <[email protected]> >> Date: Wed, 25 Feb 2015 08:48:10 -0600 >> To: Matthew Knepley <[email protected]> >> Cc: Shri <[email protected]>, "[email protected]" < >> [email protected]> >> >> Subject: Re: [petsc-users] DMNetworkGetEdgeRange() in parallel >> >> I modified the DMNetwork example to include the new DM with the >> modified section. It has the same problems. Please find attached the code >> to this email. >> >> Thanks >> >> On Tue, Feb 24, 2015 at 6:49 PM, Matthew Knepley <[email protected]> >> wrote: >> >>> On Tue, Feb 24, 2015 at 6:42 PM, Miguel Angel Salazar de Troya < >>> [email protected]> wrote: >>> >>>> I implemented the code as agreed, but I don't get the results I >>>> expected. When I create the vector with DMCreateGlobalVector(), I obtain a >>>> vector with a layout similar to the original DMNetwork, instead of the >>>> cloned network with the new PetscSection. The code is as follows: >>>> >>>> DMClone(dm, &dmEdge); >>>> >>>> PetscSectionCreate(PETSC_COMM_WORLD, &s); >>>> PetscSectionSetNumFields(s, 1); >>>> PetscSectionSetFieldComponents(s, 0, 1); >>>> >>>> // Now to set the chart, I pick the edge range >>>> >>>> DMNetworkGetEdgeRange(dmEdge, & eStart, & eEnd) >>>> >>>> PetscSectionSetChart(s, eStart, eEnd); >>>> >>>> for(PetscInt e = eStart; c < eEnd; ++e) { >>>> PetscSectionSetDof(s, e, 1); >>>> PetscSectionSetFieldDof(s, e, 0, 1); >>>> } >>>> PetscSectionSetUp(s); >>>> >>>> DMSetDefaultSection(dmEdge s); >>>> DMCreateGlobalVector(dmEdge, &globalVec); >>>> >>>> When I get into DMCreateGlobalVector(dmEdge, &globalVec) in the >>>> debugger, in the function DMCreateSubDM_Section_Private() I call >>>> PetscSectionView() on the section >>>> >>> >>> I have no idea why you would be in DMCreateSubDM(). >>> >>> Just view globalVec. If the code is as above, it will give you a >>> vector with that layout. If not >>> it should be trivial to make a small code and send it. I do this >>> everywhere is PETSc, so the >>> basic mechanism certainly works. >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> obtained by DMGetDefaultGlobalSection(dm, §ionGlobal), and I >>>> obtain a PetscSection nothing like the one I see when I call >>>> PetscSectionView() >>>> on the PetscSection I created above. Does this have anything to do? I >>>> tried to compare this strange PetscSection with the one from the original >>>> DMNetwork, I call DMGetDefaultGlobalSection(dm, §ionGlobal) before >>>> the first line of the snippet above and I get this error message. >>>> >>>> 0]PETSC ERROR: --------------------- Error Message >>>> -------------------------------------------------------------- >>>> [0]PETSC ERROR: Object is in wrong state >>>> [0]PETSC ERROR: DM must have a default PetscSection in order to create >>>> a global PetscSection >>>> >>>> Thanks in advance >>>> Miguel >>>> >>>> >>>> On Mon, Feb 23, 2015 at 3:24 PM, Matthew Knepley <[email protected]> >>>> wrote: >>>> >>>>> On Mon, Feb 23, 2015 at 2:15 PM, Miguel Angel Salazar de Troya < >>>>> [email protected]> wrote: >>>>> >>>>>> Thanks a lot, the partition should be done before setting up the >>>>>> section, right? >>>>>> >>>>> >>>>> The partition will be automatic. All you have to do is make the >>>>> local section. The DM is already partitioned, >>>>> and the Section will inherit that. >>>>> >>>>> Matt >>>>> >>>>> >>>>>> Miguel >>>>>> >>>>>> On Mon, Feb 23, 2015 at 2:05 PM, Matthew Knepley <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> On Mon, Feb 23, 2015 at 1:40 PM, Miguel Angel Salazar de Troya < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Wouldn't including the edge variables in the global vector make the >>>>>>>> code slower? I'm using the global vector in a TS, using one of the >>>>>>>> explicit >>>>>>>> RK schemes. The edge variables would not be updated in the RHSFunction >>>>>>>> evaluation. I only change the edge variables in the TSUpdate. If the >>>>>>>> global >>>>>>>> vector had the edge variables, it would be a much larger vector, and >>>>>>>> all >>>>>>>> the vector operations performed by the TS would be slower. Although the >>>>>>>> vector F returned by the RHSFunction would be zero in the edge variable >>>>>>>> components. I guess that being the vector sparse that would not be a >>>>>>>> problem. >>>>>>>> >>>>>>>> I think I'm more interested in the PetscSection approach because >>>>>>>> it might require less modifications in my code. However, I don't know >>>>>>>> how I >>>>>>>> could do this. Maybe something like this? >>>>>>>> >>>>>>>> PetscSectionCreate(PETSC_COMM_WORLD, &s); >>>>>>>> PetscSectionSetNumFields(s, 1); >>>>>>>> PetscSectionSetFieldComponents(s, 0, 1); >>>>>>>> >>>>>>>> // Now to set the chart, I pick the edge range >>>>>>>> >>>>>>>> DMNetworkGetEdgeRange(dm, & eStart, & eEnd >>>>>>>> >>>>>>>> PetscSectionSetChart(s, eStart, eEnd); >>>>>>>> >>>>>>>> for(PetscInt e = eStart; c < eEnd; ++e) { >>>>>>>> PetscSectionSetDof(s, e, 1); >>>>>>>> PetscSectionSetFieldDof(s, e, 1, 1); >>>>>>>> >>>>>>> >>>>>>> It should be PetscSectionSetFieldDof(s, e, 0, 1); >>>>>>> >>>>>>> >>>>>>>> } >>>>>>>> PetscSectionSetUp(s); >>>>>>>> >>>>>>>> Now in the manual I see this: >>>>>>>> >>>>>>> >>>>>>> First you want to do: >>>>>>> >>>>>>> DMClone(dm, &dmEdge); >>>>>>> >>>>>>> and then use dmEdge below. >>>>>>> >>>>>>> >>>>>>>> DMSetDefaultSection(dm, s); >>>>>>>> DMGetLocalVector(dm, &localVec); >>>>>>>> DMGetGlobalVector(dm, &globalVec); >>>>>>>> >>>>>>>> Setting up the default section in the DM would interfere with the >>>>>>>> section already set up with the variables in the vertices? >>>>>>>> >>>>>>> >>>>>>> Yep, thats why you would use a clone. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Matt >>>>>>> >>>>>>> >>>>>>>> Thanks a lot for your responses. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 23, 2015 at 11:37 AM, Matthew Knepley < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> On Mon, Feb 23, 2015 at 9:27 AM, Miguel Angel Salazar de Troya < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> I'm iterating through local edges given in DMNetworkGetEdgeRange(). >>>>>>>>>> For each edge, I extract or modify its corresponding value in a >>>>>>>>>> global >>>>>>>>>> petsc vector. Therefore that vector must have as many components as >>>>>>>>>> edges >>>>>>>>>> there are in the network. To extract the value in the vector, I use >>>>>>>>>> VecGetArray() and a variable counter that is incremented in each >>>>>>>>>> iteration. >>>>>>>>>> The array that I obtain in VecGetArray() has to be the same size >>>>>>>>>> than the edge range. That variable counter starts as 0, so if the >>>>>>>>>> array >>>>>>>>>> that I obtained in VecGetArray() is x_array, x_array[0] must be >>>>>>>>>> the component in the global vector that corresponds with the start >>>>>>>>>> edge >>>>>>>>>> given in DMNetworkGetEdgeRange() >>>>>>>>>> >>>>>>>>>> I need that global petsc vector because I will use it in other >>>>>>>>>> operations, it's not just data. Sorry for the confusion. Thanks in >>>>>>>>>> advance. >>>>>>>>>> >>>>>>>>> >>>>>>>>> This sounds like an assembly operation. The usual paradigm is to >>>>>>>>> compute in the local space, and then communicate to get to the global >>>>>>>>> space. So you would make a PetscSection that had 1 (or some) unknowns >>>>>>>>> on >>>>>>>>> each cell (edge) and then you can use DMCreateGlobal/LocalVector() and >>>>>>>>> DMLocalToGlobal() to do this. >>>>>>>>> >>>>>>>>> Does that make sense? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Matt >>>>>>>>> >>>>>>>>> >>>>>>>>>> Miguel >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Feb 23, 2015 at 9:09 AM, Matthew Knepley < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> On Mon, Feb 23, 2015 at 8:42 AM, Miguel Angel Salazar de Troya >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks, that will help me. Now what I would like to have is the >>>>>>>>>>>> following: if I have two processors and ten edges, the >>>>>>>>>>>> partitioning results >>>>>>>>>>>> in the first processor having the edges 0-4 and the second >>>>>>>>>>>> processor, the >>>>>>>>>>>> edges 5-9. I also have a global vector with as many components as >>>>>>>>>>>> edges, >>>>>>>>>>>> 10. How can I partition it so the first processor also has the 0-4 >>>>>>>>>>>> components and the second, the 5-9 components of the vector? >>>>>>>>>>>> >>>>>>>>>>> I think it would help to know what you want to accomplish. >>>>>>>>>>> This is how you are proposing to do it.' >>>>>>>>>>> >>>>>>>>>>> If you just want to put data on edges, DMNetwork has a >>>>>>>>>>> facility for that already. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Matt >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Miguel >>>>>>>>>>>> On Feb 23, 2015 8:08 AM, "Abhyankar, Shrirang G." < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Miguel, >>>>>>>>>>>>> One possible way is to store the global numbering of any >>>>>>>>>>>>> edge/vertex in the "component" attached to it. Once the mesh gets >>>>>>>>>>>>> partitioned, the components are also distributed so you can >>>>>>>>>>>>> easily retrieve >>>>>>>>>>>>> the global number of any edge/vertex by accessing its component. >>>>>>>>>>>>> This is >>>>>>>>>>>>> what is done in the DMNetwork example pf.c although the global >>>>>>>>>>>>> numbering is >>>>>>>>>>>>> not used for anything. >>>>>>>>>>>>> >>>>>>>>>>>>> Shri >>>>>>>>>>>>> From: Matthew Knepley <[email protected]> >>>>>>>>>>>>> Date: Mon, 23 Feb 2015 07:54:34 -0600 >>>>>>>>>>>>> To: Miguel Angel Salazar de Troya <[email protected]> >>>>>>>>>>>>> Cc: "[email protected]" <[email protected]> >>>>>>>>>>>>> Subject: Re: [petsc-users] DMNetworkGetEdgeRange() in parallel >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Feb 22, 2015 at 3:59 PM, Miguel Angel Salazar de >>>>>>>>>>>>> Troya <[email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks. Once I obtain that Index Set with the routine >>>>>>>>>>>>>> DMPlexCreateCellNumbering() >>>>>>>>>>>>>> (I assume that the edges in DMNetwork correspond to cells in >>>>>>>>>>>>>> DMPlex) can I >>>>>>>>>>>>>> use it to partition a vector with as many components as edges I >>>>>>>>>>>>>> have in my >>>>>>>>>>>>>> network? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I do not completely understand the question. >>>>>>>>>>>>> >>>>>>>>>>>>> If you want a partition of the edges, you can use >>>>>>>>>>>>> DMPlexCreatePartition() and its friend DMPlexDistribute(). What >>>>>>>>>>>>> are you trying to do? >>>>>>>>>>>>> >>>>>>>>>>>>> Matt >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> Miguel >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sun, Feb 22, 2015 at 12:15 PM, Matthew Knepley < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sun, Feb 22, 2015 at 11:01 AM, Miguel Angel Salazar de >>>>>>>>>>>>>>> Troya <[email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I noticed that the routine DMNetworkGetEdgeRange() >>>>>>>>>>>>>>>> returns the local indices for the edge range. Is there any way >>>>>>>>>>>>>>>> to obtain >>>>>>>>>>>>>>>> the global indices? So if my network has 10 edges, the >>>>>>>>>>>>>>>> processor 1 has the >>>>>>>>>>>>>>>> 0-4 edges and the processor 2, the 5-9 edges, how can I obtain >>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>> information? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> One of the points of DMPlex is we do not require a global >>>>>>>>>>>>>>> numbering. Everything is numbered >>>>>>>>>>>>>>> locally, and the PetscSF maps local numbers to local numbers >>>>>>>>>>>>>>> in order to determine ownership. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If you want to create a global numbering for some reason, >>>>>>>>>>>>>>> you can using DMPlexCreatePointNumbering(). >>>>>>>>>>>>>>> There are also cell and vertex versions that we use for >>>>>>>>>>>>>>> output, so you could do it just for edges as well. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Matt >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>> Miguel >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> *Miguel Angel Salazar de Troya* >>>>>>>>>>>>>>>> Graduate Research Assistant >>>>>>>>>>>>>>>> Department of Mechanical Science and Engineering >>>>>>>>>>>>>>>> University of Illinois at Urbana-Champaign >>>>>>>>>>>>>>>> (217) 550-2360 >>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> What most experimenters take for granted before they begin >>>>>>>>>>>>>>> their experiments is infinitely more interesting than any >>>>>>>>>>>>>>> results to which >>>>>>>>>>>>>>> their experiments lead. >>>>>>>>>>>>>>> -- Norbert Wiener >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> *Miguel Angel Salazar de Troya* >>>>>>>>>>>>>> Graduate Research Assistant >>>>>>>>>>>>>> Department of Mechanical Science and Engineering >>>>>>>>>>>>>> University of Illinois at Urbana-Champaign >>>>>>>>>>>>>> (217) 550-2360 >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> What most experimenters take for granted before they begin >>>>>>>>>>>>> their experiments is infinitely more interesting than any results >>>>>>>>>>>>> to which >>>>>>>>>>>>> their experiments lead. >>>>>>>>>>>>> -- Norbert Wiener >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> What most experimenters take for granted before they begin their >>>>>>>>>>> experiments is infinitely more interesting than any results to >>>>>>>>>>> which their >>>>>>>>>>> experiments lead. >>>>>>>>>>> -- Norbert Wiener >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Miguel Angel Salazar de Troya* >>>>>>>>>> Graduate Research Assistant >>>>>>>>>> Department of Mechanical Science and Engineering >>>>>>>>>> University of Illinois at Urbana-Champaign >>>>>>>>>> (217) 550-2360 >>>>>>>>>> [email protected] >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> What most experimenters take for granted before they begin their >>>>>>>>> experiments is infinitely more interesting than any results to which >>>>>>>>> their >>>>>>>>> experiments lead. >>>>>>>>> -- Norbert Wiener >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Miguel Angel Salazar de Troya* >>>>>>>> Graduate Research Assistant >>>>>>>> Department of Mechanical Science and Engineering >>>>>>>> University of Illinois at Urbana-Champaign >>>>>>>> (217) 550-2360 >>>>>>>> [email protected] >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> What most experimenters take for granted before they begin their >>>>>>> experiments is infinitely more interesting than any results to which >>>>>>> their >>>>>>> experiments lead. >>>>>>> -- Norbert Wiener >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Miguel Angel Salazar de Troya* >>>>>> Graduate Research Assistant >>>>>> Department of Mechanical Science and Engineering >>>>>> University of Illinois at Urbana-Champaign >>>>>> (217) 550-2360 >>>>>> [email protected] >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>> >>>> >>>> >>>> -- >>>> *Miguel Angel Salazar de Troya* >>>> Graduate Research Assistant >>>> Department of Mechanical Science and Engineering >>>> University of Illinois at Urbana-Champaign >>>> (217) 550-2360 >>>> [email protected] >>>> >>>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> >> >> -- >> *Miguel Angel Salazar de Troya* >> Graduate Research Assistant >> Department of Mechanical Science and Engineering >> University of Illinois at Urbana-Champaign >> (217) 550-2360 >> [email protected] >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 [email protected]
