On Tue, 10 Jan 2017, Barna Becsek wrote:
What we DON’T HAVE:
- A clear understanding of how ghost elements in LibMesh work.
If you need complicated ghosting (multiple layers of ghost elements,
ghost elements defined by contact problems, ghosted DoFs on distant
elements, etc) then the best writeup of the problem is probably the
one in include/base/ghosting_functor.h
If you have a simple problem, then "ghost elements" are just
"neighbors of local elements which are not themselves local".
- Knowledge about the difference of ghost and semilocal elements.
"semilocal" is just shorthand for "either local or ghost"
- A clear outline of operations to perform in order to set up such a
parallel mesh with the availability of the above mentioned functions
for numbering.
The operations you listed looked correct at first glance, but if
you're throwing assertion failures then there must be some subtle bug.
A note on this: The above mentioned functions create elements
across the entire FD grid also such element that span across
processor boundaries.
I do not understand this sentence.
If I read them in like that will Libmesh be able to find out
about which of them are ghost elements just by knowing that
their vertices are not all on the same processor (e.g. after
calling prepare_for_use).
Some ghost elements *do* have vertices all on the same processor!
For instance, in 1D, you might have elements and nodes:
0 1 2 3 4
*---*---*---*---*
0 1 2 3
Suppose that elements 0 and 1 belong to processor 0; libMesh will
typically also assign all their nodes, 0, 1, and 2, to processor 0.
The rest of each set belong to processor 1.
Then, on processor 1, element 1 is a ghost element (because it belongs
to processor 0, but it neighbors element 2 on processor 1 and so must
be retained on processor 1), but both vertices of element 1 belong to
the same processor, processor 0.
Suppose instead that elements 0 and 2 belong to processor 0, but 1 and
3 belong to processor 1. Processor 0 might end up with nodes 0, 1, 2,
and 3. In this case, element 1 is a ghost element on processor 0,
even though every one of its nodes is a local node!
If this is not the case how exactly would I go about telling
this kind of information to Libmesh?
You tell libMesh that an element is local or not local by setting its
processor_id().
libMesh will automatically collect ghost elements (in the simple case
of one layer of side neighbors!) if you call the
gather_neighboring_elements() method; however this method is very
general and requires tricky all-to-all communications of data
regarding all elements at processor boundaries; If you can add ghost
elements manually then doing so may be much faster at large scales.
If you add ghost elements manually then there is a trick that wasn't
in your list of operations above: you need to set their neighbor links
to remote_elem on all sides where their neighbor is a remote (i.e.
neither local nor ghosted) element. gather_neighboring_elements()
does that for you, and everything in prepare_for_use() simply assumes
that remote_elem links are already correct.
---
Roy
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Libmesh-users mailing list
Libmesh-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libmesh-users
