Re: [petsc-users] GAMG scaling

On Tue, Dec 25, 2018 at 12:10 AM Jed Brown wrote: > Mark Adams writes: > > > On Mon, Dec 24, 2018 at 4:56 PM Jed Brown wrote: > > > >> Mark Adams via petsc-users writes: > >> > >> > Anyway, my data for this is in my SC 2004 paper (MakeNextMat_private > in > >> > attached, NB, this is code

Re: [petsc-users] GAMG scaling

Mark Adams writes: > On Mon, Dec 24, 2018 at 4:56 PM Jed Brown wrote: > >> Mark Adams via petsc-users writes: >> >> > Anyway, my data for this is in my SC 2004 paper (MakeNextMat_private in >> > attached, NB, this is code that I wrote in grad school). It is memory >> > efficient and simple,

Re: [petsc-users] GAMG scaling

On Mon, Dec 24, 2018 at 4:56 PM Jed Brown wrote: > Mark Adams via petsc-users writes: > > > Anyway, my data for this is in my SC 2004 paper (MakeNextMat_private in > > attached, NB, this is code that I wrote in grad school). It is memory > > efficient and simple, just four nested loops i,j,I,J:

Re: [petsc-users] GAMG scaling

Mark Adams via petsc-users writes: > Anyway, my data for this is in my SC 2004 paper (MakeNextMat_private in > attached, NB, this is code that I wrote in grad school). It is memory > efficient and simple, just four nested loops i,j,I,J: C(I,J) = > P(i,I)*A(i,j)*P(j,J). In eyeballing the numbers

Re: [petsc-users] GAMG scaling

Wow, this is an old thread. Sorry if I sound like an old fart talking about the good old days but I originally did RAP. in Prometheus, in a non work optimal way that might be of interest. Not hard to implement. I bring this up because we continue to struggle with this damn thing. I think this

Re: [petsc-users] GAMG scaling

OK, so this thread has drifted, see title :) On Fri, Dec 21, 2018 at 10:01 PM Fande Kong wrote: > Sorry, hit the wrong button. > > > > On Fri, Dec 21, 2018 at 7:56 PM Fande Kong wrote: > >> >> >> On Fri, Dec 21, 2018 at 9:44 AM Mark Adams wrote: >> >>> Also, you mentioned that you are using

Re: [petsc-users] GAMG scaling

Sorry, hit the wrong button. On Fri, Dec 21, 2018 at 7:56 PM Fande Kong wrote: > > > On Fri, Dec 21, 2018 at 9:44 AM Mark Adams wrote: > >> Also, you mentioned that you are using 10 levels. This is very strange >> with GAMG. You can run with -info and grep on GAMG to see the sizes and the >>

Re: [petsc-users] GAMG scaling

Thanks so much, Hong, If any new finding, please let me know. On Fri, Dec 21, 2018 at 9:36 AM Zhang, Hong wrote: > Fande: > I will explore it and get back to you. > Does anyone know how to profile memory usage? > We are using gperftools

Re: [petsc-users] GAMG scaling

On Fri, Dec 21, 2018 at 12:55 PM Zhang, Hong wrote: > Matt: > >> Does anyone know how to profile memory usage? >>> >> >> The best serial way is to use Massif, which is part of valgrind. I think >> it might work in parallel if you >> only look at one process at a time. >> > > Can you give an

Re: [petsc-users] GAMG scaling

Matt: Does anyone know how to profile memory usage? The best serial way is to use Massif, which is part of valgrind. I think it might work in parallel if you only look at one process at a time. Can you give an example of using Massif? For example, how to use it on

Re: [petsc-users] GAMG scaling

On Fri, Dec 21, 2018 at 11:36 AM Zhang, Hong via petsc-users < petsc-users@mcs.anl.gov> wrote: > Fande: > I will explore it and get back to you. > Does anyone know how to profile memory usage? > The best serial way is to use Massif, which is part of valgrind. I think it might work in parallel if

Re: [petsc-users] GAMG scaling

Also, you mentioned that you are using 10 levels. This is very strange with GAMG. You can run with -info and grep on GAMG to see the sizes and the number of non-zeros per level. You should coarsen at a rate of about 2^D to 3^D with GAMG (with 10 levels this would imply a very large fine grid

Re: [petsc-users] GAMG scaling

Fande: I will explore it and get back to you. Does anyone know how to profile memory usage? Hong Thanks, Hong, I just briefly went through the code. I was wondering if it is possible to destroy "c->ptap" (that caches a lot of intermediate data) to release the memory after the coarse matrix is

Re: [petsc-users] GAMG scaling

Thanks, Hong, I just briefly went through the code. I was wondering if it is possible to destroy "c->ptap" (that caches a lot of intermediate data) to release the memory after the coarse matrix is assembled. I understand you may still want to reuse these data structures by default but for my

Re: [petsc-users] GAMG scaling

We use nonscalable implementation as default, and switch to scalable for matrices over finer grids. You may use option '-matptap_via scalable' to force scalable PtAP implementation for all PtAP. Let me know if it works. Hong On Thu, Dec 20, 2018 at 8:16 PM Smith, Barry F.

Re: [petsc-users] GAMG scaling

See MatPtAP_MPIAIJ_MPIAIJ(). It switches to scalable automatically for "large" problems, which is determined by some heuristic. Barry > On Dec 20, 2018, at 6:46 PM, Fande Kong via petsc-users > wrote: > > > > On Thu, Dec 20, 2018 at 4:43 PM Zhang, Hong wrote: > Fande: > Hong, >

Re: [petsc-users] GAMG scaling

> On Dec 20, 2018, at 5:51 PM, Zhang, Hong via petsc-users > wrote: > > Fande: > Hong, > Thanks for your improvements on PtAP that is critical for MG-type algorithms. > > On Wed, May 3, 2017 at 10:17 AM Hong wrote: > Mark, > Below is the copy of my email sent to you on Feb 27: > > I

Re: [petsc-users] GAMG scaling

Fande: Hong, Thanks for your improvements on PtAP that is critical for MG-type algorithms. On Wed, May 3, 2017 at 10:17 AM Hong mailto:hzh...@mcs.anl.gov>> wrote: Mark, Below is the copy of my email sent to you on Feb 27: I implemented scalable MatPtAP and did comparisons of three

Re: [petsc-users] GAMG scaling

Mark, Fixed https://bitbucket.org/petsc/petsc/commits/68eacb73b84ae7f3fd7363217d47f23a8f967155 Run ex56 gives mpiexec -n 8 ./ex56 -ne 13 ... -h |grep via -mattransposematmult_via Algorithmic approach (choose one of) scalable nonscalable matmatmult (MatTransposeMatMult) -matmatmult_via

Re: [petsc-users] GAMG scaling

Thanks Hong, I am not seeing these options with -help ... On Wed, May 3, 2017 at 10:05 PM, Hong wrote: > I basically used 'runex56' and set '-ne' be compatible with np. > Then I used option > '-matptap_via scalable' > '-matptap_via hypre' > '-matptap_via nonscalable' > > I

Re: [petsc-users] GAMG scaling

I basically used 'runex56' and set '-ne' be compatible with np. Then I used option '-matptap_via scalable' '-matptap_via hypre' '-matptap_via nonscalable' I attached a job script below. In master branch, I set default as 'nonscalable' for small - medium size matrices, and automatically switch to

Re: [petsc-users] GAMG scaling

Hong,the input files do not seem to be accessible. What are the command line option? (I don't see a "rap" or "scale" in the source). On Wed, May 3, 2017 at 12:17 PM, Hong wrote: > Mark, > Below is the copy of my email sent to you on Feb 27: > > I implemented scalable

Re: [petsc-users] GAMG scaling

Mark, Below is the copy of my email sent to you on Feb 27: I implemented scalable MatPtAP and did comparisons of three implementations using ex56.c on alcf cetus machine (this machine has small memory, 1GB/core): - nonscalable PtAP: use an array of length PN to do dense axpy - scalable PtAP:

[petsc-users] GAMG scaling

(Hong), what is the current state of optimizing RAP for scaling? Nate, is driving 3D elasticity problems at scaling with GAMG and we are working out performance problems. They are hitting problems at ~1.5B dof problems on a basic Cray (XC30 I think). Thanks, Mark