On Wed, Mar 10, 2021 at 8:01 AM Pilhwa Lee <jpilp...@gmail.com> wrote:
> Dear John, > > I didn’t point out the actual reason of the problem in the previous > correspondence. > > The issue was “distributed mesh”. In TACC, I need to use “allgather” to > fetch all elements of the mesh. That wasn’t the case in my local > workstation with the same code. > > Could you clarify how the usage of “distributed mesh” is determined in > libMesh? Thanks. Based on the error message you reported above, it looks like you are using a libmesh build provided by TACC: > /admin/build/admin/rpms/stampede2/BUILD/libmesh-1.6.0/ src/partitioning/mapped_subdomain_partitioner.C It is possible that TACC configured libmesh with the --enable-parmesh flag, which causes all instances of "Mesh" in your code to be treated as DistributedMesh. You can check whether this is the case by looking at the libmesh_config.h file of the libmesh installation you are using and checking whether LIBMESH_ENABLE_PARMESH is defined. The default is for "Mesh" to be treated as ReplicatedMesh, which may be how you compiled it on your local workstation. Unfortunately, as you found, you can't generally switch a code that was developed with ReplicatedMesh to use DistributedMesh without finding and fixing a few bugs first. Also, if you find that you have to use allgather() then this may defeat the purpose of using a DistributedMesh, as you have now "serialized" the Mesh, but with a less efficient data structure than would have been employed by ReplicatedMesh. A possible fix is to update your code so that it explicitly uses ReplicatedMesh and avoids "Mesh". This will make it more portable between your workstation and systems like TACC. -- John _______________________________________________ Libmesh-users mailing list Libmesh-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libmesh-users
