Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-03-13 Thread Roy Stogner
On Mon, 13 Mar 2017, John Peterson wrote: On Mon, Mar 13, 2017 at 11:06 AM, Barna Becsek wrote: I was able to overcome the problem with a little help from a friend. Basically what I did is replace “this->qrule->get_weights()” with “const_cast(this->qrule)->get_weights(

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-03-13 Thread John Peterson
On Mon, Mar 13, 2017 at 11:06 AM, Barna Becsek wrote: > Dear Roy, > > I was able to overcome the problem with a little help from a friend. > Basically what I did is replace “this->qrule->get_weights()” with > “const_cast(this->qrule)->get_weights()” and everything > worked fine! > I, however, sti

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-03-13 Thread Barna Becsek
Dear Roy, I was able to overcome the problem with a little help from a friend. Basically what I did is replace “this->qrule->get_weights()” with “const_cast(this->qrule)->get_weights()” and everything worked fine! I, however, still don’t understand why this would be necessary. In theory qrule a

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-03-09 Thread Barna Becsek
Dear Roy, I was finally able to install in a location where I can both read from compute nodes and edit the files. In the attachment is what I found. Basically, it seems like get_weights() returns an incorrect address. –Barna > On 7 Mar 2017, at 11:19, Barna Becsek wrote: > >> >> On 3 Mar

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-03-07 Thread Barna Becsek
> On 3 Mar 2017, at 22:30, Roy Stogner wrote: > > > On Fri, 3 Mar 2017, Barna Becsek wrote: > >>> On 3 Mar 2017, at 17:47, Roy Stogner wrote: >>> >>> On Fri, 3 Mar 2017, Barna Becsek wrote: >>> > On 3 Mar 2017, at 17:28, Roy Stogner wrote: > > What is &qw and how does it compa

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-03-01 Thread Roy Stogner
On Wed, 1 Mar 2017, Barna Becsek wrote: I found the place the assertion is tripped. In the attachment you can see the stack and location of the casting. Haven’t dug into why this happens yet… Well, at least we know the cast is a red herring now! That's not a 64-bit invalid_something_or_other

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-03-01 Thread Barna Becsek
Dear Roy, I found the place the assertion is tripped. In the attachment you can see the stack and location of the casting. Haven’t dug into why this happens yet… –Barna > On 1 Mar 2017, at 14:50, Barna Becsek wrote: > > Alright, thank you for giving it a look. As far as I can see from libm

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-03-01 Thread Barna Becsek
Alright, thank you for giving it a look. As far as I can see from libmesh’s config.log the CXXFLAGS_DBG do include the option -O0. For the others I cannot really do much at the moment because I am in a somewhat unfortunate situation, where: I have a filesystem on the cluster that I can compile s

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-02-28 Thread Roy Stogner
On Tue, 28 Feb 2017, Barna Becsek wrote: > I actually attached the stack to the previous email but I am not > sure whether I can actually send attachments to this mailing list. Depends on attachment size, but I simply overlooked it. I use a text email reader 90% of the time and my eyes glaze ov

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-02-28 Thread Barna Becsek
Thank you very much! I actually attached the stack to the previous email but I am not sure whether I can actually send attachments to this mailing list. So this is the output: Assertion `oldvar == static_cast(static_cast(oldvar))' failed. oldvar = [4] /apps/daint/UES/6.0.UP02/sandbox-ws/moose

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-02-28 Thread Roy Stogner
On Tue, 28 Feb 2017, Barna Becsek wrote: > To remedy this I called prepare_for_use once additionally before > gather_neigboring_elements (to make sure that the mesh is set up as > a distributed one) and then once after. > > This actually made me overcome the original problem, i.e. the above > loo

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-02-28 Thread Barna Becsek
Thank you so much for your help. I still have some issues/questions: I have verified that the problem is exactly what you said by checking how far the loop over i (till mesh.max_elem_id()) goes before that process (rank 0) gets ahead of the other processes. It exactly only goes as far as the loc

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-02-27 Thread Roy Stogner
On Mon, 27 Feb 2017, Barna Becsek wrote: > I hope this does not come too late. That depends entirely on your deadlines, I fear. ;-) > But I finally got the debugger to work with some help of support on > the Supercomputer that we are using. This is the complete stack: > > To recap the problem:

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-02-27 Thread Barna Becsek
Dear Roy, I hope this does not come too late. But I finally got the debugger to work with some help of support on the Supercomputer that we are using. This is the complete stack: To recap the problem: We were reading in a mesh that was created outside of libmesh (rectilinear) and wanted to use

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-13 Thread Roy Stogner
On Fri, 13 Jan 2017, Barna Becsek wrote: > Hmm, you are right but I cannot find a process that would correspond to the > children of any of these. Is this a cluster, by any chance? Where slurm is running on the login node but the actual applications are running on separate compute nodes? If I

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-13 Thread Barna Becsek
Hmm, you are right but I cannot find a process that would correspond to the children of any of these. These are all the processes for my username: ps -elf | grep becsekba 4 S root 12129 17897 0 80 0 - 24867 SyS_po 14:35 ?00:00:00 sshd: becsekba [priv] 4 S becsekba 12131 1

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-13 Thread Roy Stogner
On Fri, 13 Jan 2017, Barna Becsek wrote: > I think these also belong to the execution: I think you've posted backtraces for every process *except* the important ones. :-D Shouldn't there be child processes of those srun processes? The children would be the ones running *your* code; that's whe

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-13 Thread Barna Becsek
I think these also belong to the execution: 0 S becsekba 54421 52395 0 80 0 - 4438 wait 12:39 pts/92 00:00:00 /bin/bash /apps/daint/UES/xalt/0.7.6/bin/srun -n 8 whale-dbg -i IMP/RunImpact2D.i (gdb) bt #0 0x2b641c273cec in waitpid () from /lib64/libc.so.6 #1 0x004297aa in

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-13 Thread Barna Becsek
Ok, this is the backtrace of the running processes. There are two processes running: 0 S becsekba 54451 54421 0 80 0 - 76108 futex_ 12:39 pts/92 00:00:00 /opt/slurm/16.05.8/bin/srun -n 8 whale-dbg -i IMP/RunImpact2D.i 1 S becsekba 54477 54451 0 80 0 - 24908 pipe_w 12:39 pts/92 00

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-12 Thread Roy Stogner
On Thu, 12 Jan 2017, Barna Becsek wrote: > What I meant was the program will not exit gather_neighboring_elements. I > think the processes are still running. Right. But you can e.g. attach gdb to a running process to get a stack trace. If there's an infinite loop then we can at least find out

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-12 Thread Barna Becsek
What I meant was the program will not exit gather_neighboring_elements. I think the processes are still running. > On Jan 12, 2017, at 4:26 PM, Roy Stogner wrote: > > >> On Thu, 12 Jan 2017, Barna Becsek wrote: >> >> thank you very much for these thorough elaborations. Judging by >> these I

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-12 Thread Roy Stogner
On Thu, 12 Jan 2017, Barna Becsek wrote: thank you very much for these thorough elaborations. Judging by these I think we are doing the setup correctly. However, it seems our code gets stuck somewhere in the get_neighboring_elements() method in debug mode. It halts and does not seem to advan

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-12 Thread Barna Becsek
Dear Roy, dear Cody, thank you very much for these thorough elaborations. Judging by these I think we are doing the setup correctly. However, it seems our code gets stuck somewhere in the get_neighboring_elements() method in debug mode. It halts and does not seem to advance even with very coars

Re: [Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-11 Thread Roy Stogner
On Tue, 10 Jan 2017, Barna Becsek wrote: What we DON’T HAVE: - A clear understanding of how ghost elements in LibMesh work. If you need complicated ghosting (multiple layers of ghost elements, ghost elements defined by contact problems, ghosted DoFs on distant elements, etc) then the best wri

[Libmesh-users] Parallel Mesh Generation in Libmesh

2017-01-10 Thread Barna Becsek
Dear Users, We are writing a multiphysics tool where we need to replicate a FD mesh in a FE way. We have been almost able to translate one into the other but are facing some minor difficulties (hopefully). I would greatly appreciate if someone could help us find the missing steps for having a c

Re: [Libmesh-users] Parallel mesh

2013-04-04 Thread Manav Bhatia
Sure, I will try to probe a bit more. Will keep you posted. Manav On Apr 4, 2013, at 6:03 PM, Roy Stogner wrote: > > On Thu, 4 Apr 2013, Manav Bhatia wrote: > >> Attached are outputs (with profiling data at the end) from a run >> with 288 processors. The case with Parallel mesh is taking

Re: [Libmesh-users] Parallel mesh

2013-04-04 Thread Roy Stogner
On Thu, 4 Apr 2013, Manav Bhatia wrote: Attached are outputs (with profiling data at the end) from a run with 288 processors. The case with Parallel mesh is taking about 10 times as long to run. Thanks for this! Two things jump out at me: Ben, why is MeshCommunication::find_global_indices()

Re: [Libmesh-users] Parallel mesh

2013-04-04 Thread Roy Stogner
On Thu, 4 Apr 2013, Manav Bhatia wrote: > At this point, I do not have a need for off-processor element > data. So, the current status of ParallelMesh could be a good > thing. > > I did give it a go for my application, and so far it seems to be > working well. The memory footprint of ea

Re: [Libmesh-users] Parallel mesh

2013-04-04 Thread Manav Bhatia
Yes, this memory usage is for the program on each core, including the mesh, linear system, etc. I had my fingers crossed for the other tweaks that I had done in my code (and some other in libMesh), but was very please to see it work in the first go. I have run this only for my real solver, but

Re: [Libmesh-users] Parallel mesh

2013-04-04 Thread Kirk, Benjamin (JSC-EG311)
On Apr 4, 2013, at 8:08 AM, Manav Bhatia wrote: > His was some 450 cores. > > All that I did was replace serialmesh with Parallelmesh in my diver routine. > I will look into the output in some more detail. Ahh excellent - so you were reporting the total memory usage by the entire program?

Re: [Libmesh-users] Parallel mesh

2013-04-04 Thread Manav Bhatia
Hi Ben, His was some 450 cores. All that I did was replace serialmesh with Parallelmesh in my diver routine. I will look into the output in some more detail. Manav On Apr 4, 2013, at 8:00 AM, "Kirk, Benjamin (JSC-EG311)" wrote: > That sounds like a good savings - how many cores? > >

Re: [Libmesh-users] Parallel mesh

2013-04-04 Thread Kirk, Benjamin (JSC-EG311)
That sounds like a good savings - how many cores? ParallelMesh should be capable of writing pieces to many files, or streaming into one file. In the latter case the mesh should be completely compatible with SerialMesh. -Ben On Apr 4, 2013, at 12:23 AM, "Manav Bhatia" wrote: > Hi Roy, > >

Re: [Libmesh-users] Parallel mesh

2013-04-03 Thread Manav Bhatia
Hi Roy, At this point, I do not have a need for off-processor element data. So, the current status of ParallelMesh could be a good thing. I did give it a go for my application, and so far it seems to be working well. The memory footprint of each process has also come down significantly

Re: [Libmesh-users] Parallel mesh

2013-04-02 Thread Roy Stogner
On Wed, 3 Apr 2013, Manav Bhatia wrote: > As a related question, if my code is running on a multicore machine, > then can I use --n-threads to parallelize both the matrix assembly > and the Petsc linear solvers? Or do I have to use mpi for Petsc? PETSc isn't multithreaded, but I'm told it can b

Re: [Libmesh-users] Parallel mesh

2013-04-02 Thread Manav Bhatia
As a related question, if my code is running on a multicore machine, then can I use --n-threads to parallelize both the matrix assembly and the Petsc linear solvers? Or do I have to use mpi for Petsc? I am running problems with over a million elements, and using mpi on my multicore machine make

[Libmesh-users] Parallel mesh

2013-04-02 Thread Manav Bhatia
Hi, I am curious if the parallel mesh is now suitable for general use. Thanks Manav -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the mo

Re: [Libmesh-users] parallel mesh/mesh partitioners

2011-11-07 Thread Derek Gaston
By default if you just use the examples exactly the way they are it will just work the way you expect it to. Don't worry about partitioners and mesh classes until you _know_ you need to. Basically, with the default configuration the mesh will be read and partitioned automatically. Derek Sent fr

[Libmesh-users] parallel mesh/mesh partitioners

2011-11-07 Thread Ataollah Mesgarnejad
Dear all, I'm somewhat confused by parallel mesh and mesh partitioner class (and it's sub classes: ParmetisPartirioner, MetisPartitioner,…). I wonder if someone can explain steps you need to take after reading a mesh file on head node (or constructing mesh) to partiion it (say using parmetis)

Re: [Libmesh-users] parallel mesh

2011-03-31 Thread Roy Stogner
On Wed, 23 Mar 2011, Roy Stogner wrote: > On Fri, 18 Mar 2011, fdkong wrote: > >> I am developing a program to solve physical problems. I want to >> use parallel mesh of libmesh, but I do not know whether parallel >> mesh support large-scale cluster; that is, can I use 10 cpus >> wi

Re: [Libmesh-users] parallel mesh

2011-03-23 Thread fdkong
- Original -- From: "Roy Stogner"; Date: Wed, Mar 23, 2011 01:27 PM To: "fdkong"; Cc: "libmesh-users"; Subject: Re: [Libmesh-users] parallel mesh On Fri, 18 Mar 2011, fdkong wrote: > I am developing a program to solve physical problems.

Re: [Libmesh-users] parallel mesh

2011-03-22 Thread Roy Stogner
On Fri, 18 Mar 2011, fdkong wrote: > I am developing a program to solve physical problems. I want to > use parallel mesh of libmesh, but I do not know whether parallel > mesh support large-scale cluster; that is, can I use 10 cpus > with using prarallel mesh? In theory, yes. In pr

[Libmesh-users] parallel mesh

2011-03-18 Thread fdkong
Dear everyone I am developing a program to solve physical problems. I want to use parallel mesh of libmesh, but I do not know whether parallel mesh support large-scale cluster; that is, can I use 10 cpus with using prarallel mesh? In addition, where can I find more detail manual abou

Re: [Libmesh-users] Parallel Mesh Write

2010-06-03 Thread Andrea Hawkins
Got it. Thanks! Andrea On Thu, Jun 3, 2010 at 2:50 PM, Roy Stogner wrote: > > On Thu, 3 Jun 2010, Andrea Hawkins wrote: > ParallelMesh::allgather() >> >> I inserted mesh.allgather() prior to every mesh.write, but this didn't >> fix the issue. Is there anything else (easy) I should worry abou

Re: [Libmesh-users] Parallel Mesh Write

2010-06-03 Thread Roy Stogner
On Thu, 3 Jun 2010, Andrea Hawkins wrote: >>> ParallelMesh::allgather() > > I inserted mesh.allgather() prior to every mesh.write, but this didn't > fix the issue. Is there anything else (easy) I should worry about? I'm afraid not. > Is anyone actively looking at the libHilbert bug? Or is this

Re: [Libmesh-users] Parallel Mesh Write

2010-06-03 Thread Andrea Hawkins
>> >> ParallelMesh::allgather() > I inserted mesh.allgather() prior to every mesh.write, but this didn't fix the issue. Is there anything else (easy) I should worry about? Is anyone actively looking at the libHilbert bug? Or is this something that will probably not get fixed for awhile and I would

Re: [Libmesh-users] Parallel Mesh Write

2010-06-03 Thread Andrea Hawkins
>> I'm experimenting with ParallelMesh hoping it will in scaling. > > I guess one way to avoid running into a possible AMR bug with the > PeriodicBoundary stuff is to use a class that has a *definite* AMR > bug. ;-) Well, you know... =) > >> So, is there a way to serialize a parallel mesh. In lo

Re: [Libmesh-users] Parallel Mesh Write

2010-06-03 Thread Roy Stogner
On Thu, 3 Jun 2010, Andrea Hawkins wrote: > I'm experimenting with ParallelMesh hoping it will in scaling. I guess one way to avoid running into a possible AMR bug with the PeriodicBoundary stuff is to use a class that has a *definite* AMR bug. ;-) > When running in debug mode I get the follow

[Libmesh-users] Parallel Mesh Write

2010-06-03 Thread Andrea Hawkins
Hello- I'm experimenting with ParallelMesh hoping it will in scaling. When running in debug mode I get the following warning: Warning: This MeshInput subclass only support meshes which have been serialized! [0] /h1/ahawkins/LIBRARIES/LIBMESH/libmesh-r3882/include/mesh/mesh_input.h, line 134, com

Re: [Libmesh-users] Parallel mesh and mesh/solution IO

2009-10-21 Thread Lorenzo Botti
No prob, it's just that sometime I forget to add the list in the recipients. Back to the main issue, when you say that Exodus meshes can be split in Nemesis meshes you mean that Nemesis can do it? Il giorno 21/ott/2009, alle ore 22.08, Derek Gaston ha scritto: > Is it ok if I send my response

Re: [Libmesh-users] Parallel mesh and mesh/solution IO

2009-10-21 Thread Derek Gaston
On Oct 21, 2009, at 11:29 AM, Lorenzo Botti wrote: > Thanks for the hints. > Just a question... It seems that you prefer the Nemesis format > compared to the native xdr xda, is there any reason for that? The reason is that Nemesis is a format that other things read / write... where xdr/a is o

Re: [Libmesh-users] Parallel mesh and mesh/solution IO

2009-10-21 Thread Lorenzo Botti
Thanks for the hints. Just a question... It seems that you prefer the Nemesis format compared to the native xdr xda, is there any reason for that? I don't know if I will have time to do it in the near future because the end of my PhD is approaching very fast. Anyway I think that parallel IO for

Re: [Libmesh-users] Parallel mesh and mesh/solution IO

2009-10-21 Thread Derek Gaston
1) In certain circumstances but probably not huge. 2) Yes 3) You can read and write parallel xda/r files although I haven't done it myself. BUT... it looks like that is only possible when writing the entire equation system not just the mesh. I don't know if there is a way around that. M

[Libmesh-users] Parallel mesh and mesh/solution IO

2009-10-19 Thread Lorenzo Botti
I'm planning to perform a lot of parallel runs (without AMR) on a big cluster and I would like to know a couple of details about the parallel mesh implementation. 1) Is it faster than serial mesh? 2) Does it use less memory per process? 3) Is it possible to read and/or write parallel .xdr/.xda dat