Hi, I am considering implementation of parallel read of a large exodusII mesh file using nemesis API. I am sharing my line of thinking to seek comments from those more experienced.
Motivation: I have a mesh of ~40M elements in an exodusII file that I am reading using the exodusII_IO API. Reading and initialization of this mesh before analysis is taking ~30 minutes of time and ~22 GB of data before construction of matrices/vectors, etc. I am hoping to cut down on memory utilization and initialization time. Approach: For various reasons, the use of SEACAS to prepartition the file into multiple subdomains for use in nemesis is proving to be a bit cumbersome, and I am hoping for a more automated solution, described below: Step 1: The basic approach is to first divide the mesh into Nprocessor partitions uniformly, so that each processor reads in its portion of the element connectivity. (needs expansion of NemesisIO and helper classes) Step 2: Feed the connectivity data into Parmetis to create the partitioning. Step 3: With the partition data available, ask each processor to update its portion of required node IDs. Step 4: Each processor reads the details of its elements and relevant nodal data into the ParallelMesh. Step 5: Each processor now reads its portion of the boundary condition data. Step 6: Prepare the dofmaps. I understand that I am loosely using the notion of "read in its portion of the data" for each processor. Following the Parmetis partitioning, the data might not be contiguous in the file, in which case the master node might do the reading and send the data to processors. I am curious to hear the thoughts of developers in the merit of this approach, or some obvious pitfalls. If this seems of interest, I can take a dig at it and share the patch once it is working. Thanks, Manav ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j _______________________________________________ Libmesh-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-users
