Re: [petsc-dev] DMDAGlobalToNatural errors with Ubuntu:latest; gcc 7 & Open MPI 2.1.1

2019-07-30 Thread Smith, Barry F. via petsc-dev
Note in init.c that, by default, PETSc does not use PetscTrMallocDefault() when valgrind is running; because it doesn't necessarily make sense to put one memory checker on top of another memory checker. So, at a glance, I'm puzzled how it can be in the routine PetscTrMallocDefault(). Do you

Re: [petsc-dev] [petsc-users] MatMultTranspose memory usage

2019-07-30 Thread Jed Brown via petsc-dev
"Smith, Barry F. via petsc-users" writes: >The reason this worked for 4 processes is that the largest count in that > case was roughly 6,653,750,976/4 which does fit into an int. PETSc only needs > to know the number of nonzeros on each process, it doesn't need to know the > amount across

Re: [petsc-dev] DMDAGlobalToNatural errors with Ubuntu:latest; gcc 7 & Open MPI 2.1.1

2019-07-30 Thread Zhang, Junchao via petsc-dev
Fabian, I happen have a Ubuntu virtual machine and I could reproduce the error with your mini-test, even with two processes. It is horrible to see wrong results in such a simple test. We'd better figure out whether it is a PETSc bug or an OpenMPI bug. If it is latter, which MPI call is at

Re: [petsc-dev] DMDAGlobalToNatural errors with Ubuntu:latest; gcc 7 & Open MPI 2.1.1

2019-07-30 Thread Smith, Barry F. via petsc-dev
Satish, Can you please add to MPI.py a check for this and simply reject it telling the user there are bugs in that version of OpenMP/ubuntu? It is not debuggable, and hence not fixable and wastes everyones time and could even lead to wrong results (which is worse than crashing). We've

Re: [petsc-dev] MatPinToCPU

2019-07-30 Thread Smith, Barry F. via petsc-dev
Sorry, I meant 24 CPU only > On Jul 30, 2019, at 9:19 AM, Mark Adams wrote: > > > > On Mon, Jul 29, 2019 at 11:27 PM Smith, Barry F. wrote: > > Thanks. Could you please send the 24 processors with the GPU? > > That is in out_cuda_24 > > >Note the final column of the

Re: [petsc-dev] DMDAGlobalToNatural errors with Ubuntu:latest; gcc 7 & Open MPI 2.1.1

2019-07-30 Thread Balay, Satish via petsc-dev
We've seen such behavior with ubuntu default OpenMPI - but have no idea why this happens or if we can work around it. Last I checked - the same version of openmpi - when installed separately did not exhibit such issues.. Satish On Tue, 30 Jul 2019, Fabian.Jakub via petsc-dev wrote: > Dear

[petsc-dev] DMDAGlobalToNatural errors with Ubuntu:latest; gcc 7 & Open MPI 2.1.1

2019-07-30 Thread Fabian.Jakub via petsc-dev
Dear Petsc Team, Our cluster recently switched to Ubuntu 18.04 which has gcc 7.4 and (Open MPI) 2.1.1 - with this I ended up with segfault and valgrind errors in DMDAGlobalToNatural. This is evident in a minimal fortran example such as the attached example petsc_ex.F90 with the following error:

Re: [petsc-dev] MatPinToCPU

2019-07-30 Thread Mark Adams via petsc-dev
On Mon, Jul 29, 2019 at 11:27 PM Smith, Barry F. wrote: > > Thanks. Could you please send the 24 processors with the GPU? > That is in out_cuda_24 >Note the final column of the table gives you the percentage of flops > (not rates, actual operations) on the GPU. For you biggest