Anders Logg wrote:
> On Mon, Aug 18, 2008 at 11:05:30AM +0200, Niclas Jansson wrote:
>
>> Anders Logg wrote:
>>
>>>>> I think it looks good.
>>>>>
>>>>> As far as I understand, you build a global numbering of all mesh
>>>>> entities (which may be different from the local numbering on each
>>>>> processor), and then the (global parallel) local-to-global mapping
>>>>> follows from tabulate_dofs just as usual.
>>>>>
>>>>> So, the difference is that you build a global numbering of the mesh
>>>>> entities, and we wanted to build a global numbering of the dofs. The
>>>>> only advantage I can see with our approach is that it may use less
>>>>> memory, since we don't need to store an extra numbering scheme for all
>>>>> mesh entities but this not a big deal.
>>>>>
>>>>> A few questions:
>>>>>
>>>>> 1. Is the above interpretation correct?
>>>>>
>>>>>
>>>>>
>>>> Yes.
>>>>
>>>> Another disadvantage with the global numbering scheme is the mesh
>>>> connectivity calculations (mesh.init in MeshRenumber).
>>>>
>>>>
>>> Why is this a problem? As far as I understand, there are always two
>>> different numberings of mesh entities, one local (same as we have
>>> now) and one global. The local can be computed as usual and then the
>>> global can be reconstructed from the local + the overlap.
>>>
>>> (overlap = how the local pieces fit together)
>>>
>>>
>>>
>>>
>> Iterating over the local + overlap requires some mesh connectivity,
>> which are costly to generate.
>>
>
> What's your point? Are you arguing against a global numbering scheme
> for the mesh entities? I thought this is what you have implemented.
>
>
I'm not sure if the global numbering scheme is the best approach. It
worked well for simple dof_maps / elements, with a low renumbering time.
But for a more general implementation, renumbering starts to take too
much time.
>>>>> 6. Does it work for assembly over interior facets (like in DG
>>>>> methods)?
>>>>>
>>>>>
>>>>>
>>>> I'm not sure, haven't tried any DG type problems.
>>>>
>>>>
>>> I expect it to be problematic. For interior facet integrals, we need
>>> to iterate over all interior facets in the mesh and for each facet
>>> access its two adjacent cells. One of these may be on another
>>> processor.
>>>
>>>
>>>
>> But shouldn't a set of ghosted cells for the interior boundary solve
>> this problem?
>>
>
> Do you mean an overlap consisting of cells, not facets?
>
>
Yes, cells. Otherwise, how should a processor access a cell on a
adjacent processor?
>>>
>>>>> Then some suggestions:
>>>>>
>>>>> 1. I'd like to move the implementation of DofMap::build() to
>>>>> DofMapBuilder (to simplify DofMap.cpp).
>>>>>
>>>>> 2. Function names should be fooBar(), not foo_bar().
>>>>>
>>>>>
>>>>>
>>>> I recently found doc/misc/policy :)
>>>>
>>>>
>>>>> 3. There needs to be some #ifdef HAS_PARMETIS so it may be built
>>>>> without Parmetis.
>>>>>
>>>>> 4. I'd like to add a new class GlobalNumbering (to replace
>>>>> MeshDistributedData) that holds the global numbering scheme.
>>>>>
>>>>> The Mesh class can have a pointer to a GlobalNumbering object which is
>>>>> 0 by default so no extra data (or at least not more than 4 bytes) is
>>>>> stored when not running in parallel.
>>>>>
>>>>> Then we can add a function MeshEntity::number() which returns the same
>>>>> as index() if GlobalNumbering is 0. Otherwise, it returns what is
>>>>> stored in GlobalNumbering.
>>>>>
>>>>> GlobalNumbering can have an array of MeshFunctions, one for each
>>>>> topological dimension, that maps the local entity indices to their
>>>>> global numbers.
>>>>>
>>>>> Thus, a MeshEntity will have two functions index() and number().
>>>>> These will return the same value in sequential and possibly different
>>>>> values in parallel.
>>>>>
>>>>> Let's await some more comments and then get started. It would be nice
>>>>> to get it in small patches to give us an opportunity to comment/edit.
>>>>> Adding GlobalNumbering and MeshEntity::number() would be a good start.
>>>>>
>>>>>
>>>>>
>>>> Sounds good, However I think the {T,S,F,V,O} design mentioned by Johan
>>>> is more efficient.
>>>>
>>>>
>>> These are different issues. One issue is how to represent the overlap,
>>> however we choose to do that, either {T,S,F,V,O} or just {T, S, F}.
>>> The other issue is how we store either the global mesh numbering or
>>> the global dof map. We need to discuss both.
>>>
>>> I'd like to continue this discussion for some time before we jump on
>>> to either option.
>>>
>>> 1. Do we want to compute a global mesh numbering and then let the
>>> global dof map follow as usual, or do we want to have just a local
>>> mesh numbering and then compute a global dof map from the local dof
>>> maps + the overlap?
>>>
>>> In either case, the resulting global dof map can be renumbered to
>>> reduce bandwidth/communication.
>>>
>>> 2. How do we want to represent the overlap? Is it enough to store it
>>> as {T, S, F} or do we need more? Do we need a class Overlap that
>>> stores the overlap?
>>>
>>>
>> I think the {V,O} information is needed in order to redistribute
>> (reconstruct with the MeshEditor) a mesh during computation (due to load
>> balancing). A Global numbering is (probably) also needed for
>> refinement/coarsening.
>>
>
> Again, aren't these separate issues? One issue is how we compute the
> global dof map (which is needed for assembly). The other is how we
> store information about the overlap which we need both for computing a
> global numbering or global dof map, and for load balancing,
> refinement, coarsening etc.
>
> Other opinions?
>
> Do we want to (i) build a global numbering of mesh entities or (ii)
> have local numbering of mesh entities and build a global dof map?
>
> --
> Anders
>
True, they are different issues, but I think the choice of dof map
method should consider how expensive the overlap information are to
compute, and if it could be done with an already distributed mesh.
Niclas
_______________________________________________
DOLFIN-dev mailing list
[email protected]
http://www.fenics.org/mailman/listinfo/dolfin-dev
