Copying to libmesh-users ---------- Forwarded message ---------- From: Cody Permann <[email protected]> Date: Tue, Oct 29, 2013 at 9:32 AM Subject: Re: [Libmesh-users] Memory usage in 3D mesh generation To: ernestol <[email protected]>
libMesh doesn't generate meshes "in parallel", or to be more specific, it doesn't spread the work out when running in parallel. Why? Well, with SerialMesh, you need a copy of the whole mesh on each processor anyway so you might as well generate the same mesh on every processor rather than generate pieces and then end up doing an all-gather anyway. So, if it runs on 3 and not 4 (I'm assuming you mean procs / node), it's not a logic problem, but instead a memory problem. Have you considered using libMesh to generate a mesh offline, save it and then read it back in on your large parallel run? I'm running a large generated 3D cube right now with a few uniform_refinements on over a thousand processors. Memory is certainly an issue, but not with the mesh, instead with the equation systems. Cody On Tue, Oct 29, 2013 at 5:54 AM, ernestol <[email protected]> wrote: > I am using an cluster with 23 node for a total of 184 cores, and each node > additionally has 16GB of RAM. I was thinking that the problem maybe is in > the code. Because if I run at up to 3 processors I dont have any problens > but when I try with 4 or more I get this problem. > I was wondering that libmesh had other ways to generate the mesh in > parallel. > > Em 2013-10-25 20:23, Cody Permann escreveu: > >> 200*200*200 = 8 Million elements with roughly as many nodes. The >> error message says that its requesting ~95 MB which is well within >> >> reason when you consider that each piece of data needs to occupy >> several bytes, and each node and element have several pieces of data >> each. Youre going to need a lot more memory than that once your >> >> equation systems and PETSc vectors are allocated... >> >> Cody >> >> On Fri, Oct 25, 2013 at 9:34 AM, ernestol <[email protected] [4]> >> wrote: >> >> Hi all, >>> >>> I am changing my code from 2D to 3D. But I am having problens with >>> the >>> memory usage. >>> I mannage to create an example that reproduce the error. As we can >>> I >>> just generate the mesh and get all the memory. >>> The interesting part is that when I run the code (for Nelx,Nely and >>> Nelz>150) in 1 ,2 or 3 processors the code run completely. But in 4 >>> or >>> more I have some memory issues. >>> I saw the parallel_mesh but the code didnt work even with it. Note >>> sure >>> if I can use because the warning "Dont use this class unless youre >>> >>> developing or debugging it." >>> I wonder how can I do with my code to solve bigger problens. >>> >>> Thanks! >>> >>> Ernesto >>> >>> CODE: >>> >>> #include <iostream> >>> #include <stdio.h> >>> >>> #include "libmesh/libmesh.h" >>> #include "libmesh/mesh.h" >>> #include "libmesh/mesh_generation.h" >>> #include "libmesh/equation_systems.h" >>> >>> using namespace libMesh; >>> using namespace std; >>> >>> int main (int argc, char** argv){ >>> LibMeshInit init (argc, argv); >>> Mesh mesh; >>> EquationSystems eq_sys(mesh); >>> int Nelx,Nely,Nelz; >>> Nelx = Nely = Nelz = 200; >>> cout << "Generating Mesh" << endl; >>> >>> >>> >> MeshTools::Generation::build_**cube(mesh,Nelx,Nely,Nelz,0.,1.** >> ,0.,1.,0.,1.,HEX8); >> >>> cout << "Print Mesh Info" << endl; >>> mesh.print_info(); >>> return 0; >>> } >>> >>> The output is: >>> Generating Mesh >>> >>> >>> >> ==============================**==============================** >> ========================= >> >>> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >>> = EXIT CODE: 9 >>> = CLEANING UP REMAINING PROCESSES >>> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES >>> >>> >> ==============================**==============================** >> ========================= >> >>> >>> And the following error >>> >>> Current memory used: 906046008 bytes >>> Maximum memory used: 985036960 bytes >>> ***Memory allocation failed for SetupCoarseGraph: adjncy. Requested >>> size: 95218200 bytes >>> >>> >>> >> ------------------------------**------------------------------** >> ------------------ >> >>> October Webinars: Code for Performance >>> Free Intel webinars can help you accelerate application >>> performance. >>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the >>> most from >>> the latest Intel processors and coprocessors. See abstracts and >>> register > >>> >>> >> http://pubads.g.doubleclick.**net/gampad/clk?id=60135991&iu=** >> /4140/ostg.clktrk<http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk> >> >>> [1] >>> ______________________________**_________________ >>> Libmesh-users mailing list >>> Libmesh-users@lists.**sourceforge.net<[email protected]>[2] >>> https://lists.sourceforge.net/**lists/listinfo/libmesh-users<https://lists.sourceforge.net/lists/listinfo/libmesh-users>[3] >>> >> >> >> >> Links: >> ------ >> [1] >> >> http://pubads.g.doubleclick.**net/gampad/clk?id=60135991&** >> amp;iu=/4140/ostg.clktrk<http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk> >> [2] >> mailto:Libmesh-users@lists.**sourceforge.net<[email protected]> >> [3] >> https://lists.sourceforge.net/**lists/listinfo/libmesh-users<https://lists.sourceforge.net/lists/listinfo/libmesh-users> >> [4] mailto:[email protected] >> > ------------------------------------------------------------------------------ Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk _______________________________________________ Libmesh-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-users
