Re: [petsc-users] SNESComputeJacobianDefaultColor use too much memory

Hmm, Matt, this is nuts. This if (size > 1) { /* create a sequential iscoloring on all processors */ ierr = MatGetSeqNonzeroStructure(mat,_seq);CHKERRQ(ierr); } It sequentializes the graph. It looks like the only parallel coloring for Jacobians is MATCOLORINGGREEDY? Try

Re: [petsc-users] memory scalable AO

The attached test program demonstrates the problem. When I run it, I get the following output: [0]PETSC ERROR: [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range

Re: [petsc-users] memory scalable AO

Ahh, yes. This is our fault. Because this function takes a string it needs a custom Fortran stub which it does not have. We'll get you a patch soon. Barry > On Feb 17, 2016, at 3:46 PM, Randall Mackie wrote: > > The attached test program demonstrates the problem.

Re: [petsc-users] memory scalable AO

Here is patch. fix-fortran-dmdasetaotype.patch Description: Binary data If it works for you I'll put it in maint and master tomorrow. Barry > On Feb 17, 2016, at 3:46 PM, Randall Mackie wrote: > > The attached test program demonstrates the problem. When I run

[petsc-users] Slow MatAssembly and MatMat Mult

Hi, I have two small (2500x2500) matrices parallelized across 480 processors. One of them is an MPIAIJ matrix while the other is an MPIDENSE matrix. I perform a MatMatMult involving these two matrices. I tried these operations on two machines, one is the local cluster at University of Michigan

Re: [petsc-users] SNESComputeJacobianDefaultColor use too much memory

The greedy coloring works. Thanks Barry. Best, Rongliang On 02/18/2016 04:11 AM, Barry Smith wrote: Hmm, Matt, this is nuts. This if (size > 1) { /* create a sequential iscoloring on all processors */ ierr = MatGetSeqNonzeroStructure(mat,_seq);CHKERRQ(ierr); } It

Re: [petsc-users] memory scalable AO

It is probably my fault. Edit src/dm/impls/da/ftn-auto/daindexf.c and remove the function function from that file and then run make on the library again. If that does not work then run in the debugger and put a break point in dmdasetaotype_ once it is there in the debugger you can make sure

Re: [petsc-users] Slow MatAssembly and MatMat Mult

This is an absolutely tiny problem for 480 processors I am not surprised by the terrible performance. You should run this sub calculation on a small subset of the processes. Barry > On Feb 17, 2016, at 7:03 PM, Bikash Kanungo wrote: > > Hi, > > I have two small

Re: [petsc-users] memory scalable AO

If I remove lines 47-50 in daindexf.c (the PETSC_EXTERN …for dmdasetaotype_), then when I try to compile the test code, it complains of an undefined reference to dmdasetaotype_. So I put the lines back in, and with your patch applied, and then run in the debugger, with a breakpoint at

Re: [petsc-users] memory scalable AO

> On Feb 17, 2016, at 8:58 PM, Randall Mackie wrote: > > If I remove lines 47-50 in daindexf.c (the PETSC_EXTERN …for dmdasetaotype_), > then when I try to compile the test code, it complains of an undefined > reference to dmdasetaotype_. > > So I put the lines back

Re: [petsc-users] memory scalable AO

Randall Mackie writes: > So it seems to be ignoring the dmdasetaotype_ in > /src/dm/impls/da/ftn-custom/zdaindexf.c in favor of ../ftn-auto/daindexf.c Sounds like you need to generate your Fortran stubs so that the version in ftn-auto/daindexf.c is fixed. make

Re: [petsc-users] Slow MatAssembly and MatMat Mult

Hi Barry, I had thought of using sub-communicator for these operations. But the matrix entries have contributions from all the processors. Moreover, after the MatMatMult operation is done, I need to retrieve certain values from the resultant matrix through non-local calls (MatGetSubMatrices) on

Re: [petsc-users] Slow MatAssembly and MatMat Mult

Thanks a lot Barry. I guess it's time to bite the bullet. Regards, Bikash On Wed, Feb 17, 2016 at 10:30 PM, Barry Smith wrote: > > You need to bite the bullet and do the communication needed to get the > info on the sub communicator and then get the result back out to

Re: [petsc-users] memory scalable AO

this leads to the error ‘bin/maint/generatefortranstubs.py’ …No such file. there is no maint directory under bin. Randy > On Feb 17, 2016, at 7:05 PM, Jed Brown wrote: > > Randall Mackie writes: >> So it seems to be ignoring the dmdasetaotype_ in

Re: [petsc-users] memory scalable AO

Randall Mackie writes: > this leads to the error ‘bin/maint/generatefortranstubs.py’ …No such file. > > there is no maint directory under bin. Ah, I missed that you have a tarball, not a clone of the repository. Then you'll have to fix the file manually. I would start

Re: [petsc-users] memory scalable AO

Attached are the modified src/dm/impls/da/ftn-auto/daindexf.c and src/dm/impls/da/ftn-custom/zdaindexf.c files. Satish On Wed, 17 Feb 2016, Jed Brown wrote: > Randall Mackie writes: > > > this leads to the error ‘bin/maint/generatefortranstubs.py’ …No such file. > > > >

Re: [petsc-users] memory scalable AO

Satish, It is unclear from this email exchange if I am still suppose to do make allfortranstubs after updating these two files, or if these are the files that are suppose to result from that make operation. Although these files looked the same as what I tried before with Barry’s suggestions,

Re: [petsc-users] memory scalable AO

Randy, It is not suppose to be this hard! When you ran the make gnumake after you applied the patch did it recompile the file zdaindexf.c (in fact that file and daindexf.c should be the only files that changed and hence the only files that got recompiled). Barry > On Feb 17,

Re: [petsc-users] memory scalable AO

Look like zdaindexf.c is not listed in makefile - so it never gets compiled. Also it has code for dmdagetglobalindices_() dmdarestoreglobalindices_(). But then src/docs/website/documentation/changes/35.html has: DMDAGetGlobalIndices(DM,PetscInt*,const PetscInt*[]) and