On Tue, Aug 30, 2011 at 10:36 AM, robert <[email protected]> wrote: > Am Dienstag, den 30.08.2011, 10:08 -0600 schrieb John Peterson: >> On Tue, Aug 30, 2011 at 9:56 AM, robert <[email protected]> wrote: >> > >> > The mesh.uniformly_refine(1) takes some minutes when I do it in serial >> > on my PC. However, on the BlueGene/P it doesn't come to an end (I >> > aborted after 15 min). Do I have to prepare my mesh in a special way for >> > use in parallel? One thing I have tried was to call the >> > prepare_for_use() before the refinement. I thought this makes sense >> > because the mesh is partitioned before. But, the problem remains. >> >> Your mesh is serial unless you specifically compiled libmesh with >> --enable-parmesh. > > Ok, but nevertheless it shouldn't be slower than on my PC, or am I > wrong?
It will be extremely slow if it goes into swap. > Would this option just work with the existing code or do I have to > change to ParallelMesh? Should work with existing code. Hopefully. >> What this means is that the mesh data structure (and any refinements, >> etc.) is duplicated on all processors -- convenient for writing code, >> not great for conserving memory. >> >> And it's particularly bad on multicore shared memory architectures: >> you will have one copy of the mesh per MPI process, typically that >> means one per core. >> >> > Mesh Information: >> > Mesh Information: >> > mesh_dimension()=3 >> > spatial_dimension()=3 >> > n_nodes()=447273 >> > n_local_nodes()=447273 >> > n_elem()=2994336 >> > n_local_elem()=2994336 >> > n_active_elem()=2661632 >> > n_subdomains()=2 >> > n_processors()=1 >> > processor_id()=0 >> >> You may be going into swap depending on how little memory is available >> per core on this machine, but 3M elements isn't particularly huge.... >> so >> >> .) How much memory do you have per core? > > – SMP mode : 4 GB of physical memory available to the MPI task > running on each node (4 cores/node) I can't tell if you mean 4GB/node or 4GB/core. If it's the latter (16 GB/node) then I don't see how you can possibly be running into swap. If it's the former then there could be an issue. > For testing and learning I only use a partition of 32 nodes. 32 nodes or 32 cores? I don't know the details of your cluster so it may be obvious, but make sure you aren't accidentally running too many MPI processes on a given node. >> .) What is going on in 'top' or the equivalent on BGL when your code is >> running? > What do you mean by this? Excuse my question - I am just not the most > experienced person with computers. On Linux/UNIX systems, top is a program that prints what processes are currently running, the relative amount of CPU they are using, and their memory consumption. If you can access the compute node where your code is running and see how much memory/CPU each of its processes is consuming, you can get some idea if the code is running into swap (there will be a very low CPU utilization and a very high memory consumption). -- John ------------------------------------------------------------------------------ Special Offer -- Download ArcSight Logger for FREE! Finally, a world-class log management solution at an even better price-free! And you'll get a free "Love Thy Logs" t-shirt when you download Logger. Secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsisghtdev2dev _______________________________________________ Libmesh-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-users
