On Fri, Jan 26, 2018 at 11:06 AM, Roy Stogner <[email protected]> wrote:
> > On Thu, 25 Jan 2018, David Knezevic wrote: > > Thanks for that info. I'm only using ReplicatedMesh for now. Regarding >> this: >> >> IIRC: on ReplicatedMesh we try to avoid communication as much >> as possible, so you have to add user constraints identically on >> every processor. There might be ways to get around that (add >> local constraints only, then re-add after any repartitioning?) but >> I definitely wouldn't suggest trying to depend on them. >> >> hmm, the reason I asked about this was because I had a case that hit >> a "cyclic loop" assertion in parallel when I added the constraints >> identically on every processor, >> >> and then when I changed it so that I only added the constraint on >> the processor that owned the constrained dof, it worked fine. I >> assumed that I must have been violating an assumption by adding the >> constraints identically on all processors, but your comment would >> indicate that isn't the case... >> > > Looking back at dof_map_constraints.C, it looks like my comment was > wrong - I did parallelize the constraint calculations, even on > ReplicatedMesh. We're looping only over local elements when we > compute hanging node and Dirichlet constraints. That means we *must* > communicate (e.g. to handle cases where a local constraint only gets > computed on a ghost element) later. OK, thanks for confirming that. Yes, I see that we only loop over local elements in DofMap::create_dof_constraints. > I'd like to understand this better. Can you point me to the relevant >> code in the library? I've been looking through >> allgather_recursive_constraints and similar, and I figured they send >> and receive constraint rows as needed, which is why I assumed we >> didn't need to add constraint rows identically on all processors. >> > > The first section of allgather_recursive_constraints() is what's > relevant here - it does the initial push of constraints that we've > calculated to their owners. > > And based on that code, it looks like the rule is "we don't need to > add constraint rows on all processors, but if we *do* then we need to > add them identically". > OK, that makes sense. Yeah, I see where we have the comment "If we don't already have a constraint for this dof, add the one we were sent" > It might be worth adding some error checking there to make sure that > happens. Right now if a processor has a constrained row, and gets > pushed a constraint for that row, it assumes the remote constraint is > equivalent to the local and it ignores it. We should probably > libmesh_assert equivalency (within TOLERANCE*TOLERANCE ought to be > safe; these are nondimensional numbers) instead. > Agreed, that seems like a good idea. I guess that check could be added near the comment that I pasted above. Thanks for your help! That clarifies things a lot! Regards, David ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Libmesh-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-users
