Re: [petsc-users] memory use of a DMDA

[Jed Brown]

Juha Jäykkä <ju...@iki.fi> writes:

>> I will test with 3.3 later today if I have the time, but I'm pretty sure
>> things were "better" there.
>
> I did happen to have one system still having 3.2 on it, so I tested that 
> instead of 3.3 and at least 3.2 seems to be a bit more conservative in 
> numbers. For the same $CMD as before, I get
>
> for x in 1 2 4 ; do mpirun -np $x python -c "$CMD"; done
> 21 MiB / 21868 kB
> 21 MiB / 21888 kB
> 218 MiB / 224028 kB
> 22 MiB / 22852 kB
> 22 MiB / 22600 kB
> 22 MiB / 22620 kB
> 22 MiB / 22872 kB
> 79 MiB / 81640 kB
> 79 MiB / 81372 kB
> 22 MiB / 22628 kB
> 22 MiB / 22648 kB
> 22 MiB / 22632 kB
> 22 MiB / 22652 kB
> 22 MiB / 22880 kB
> 22 MiB / 22900 kB
> 22 MiB / 22908 kB
> 22 MiB / 22928 kB
> 51 MiB / 52944 kB
> 51 MiB / 52936 kB
> 51 MiB / 53188 kB
> 51 MiB / 53216 kB
>
> For comparison, 3.4.2 gives (from the previous email): 354 MiB for 1
> rank, 141 for 2 ranks and 81 for 4 ranks, which is a LOT less. I
> suspect this might have something to do with the DA -> DMDA change?

Hmm, I wonder where you're getting your numbers from.  

CMD='import psutil,os; rss = lambda: 
psutil.Process(os.getpid()).get_memory_info().rss; print rss()/2**20, "MiB 
initial"; from petsc4py import PETSc; print rss()/2**20, "MiB loaded petsc4py"; 
da = PETSc.DA().create(sizes=[100,100,100], 
proc_sizes=[PETSc.DECIDE,PETSc.DECIDE,PETSc.DECIDE], boundary_type=[3,0,0], 
stencil_type=PETSc.DA.StencilType.BOX, dof=7, stencil_width=1, 
comm=PETSc.COMM_WORLD); print rss()/2**20, "MiB DMDA"'

Here is what I have with v3.4.2, which is almost exactly the same as you
show above for v3.2.  In particular, it is much better than your email
claimed for v3.4.2.

  $ mpirun.hydra -np 1 python2 -c "$CMD"
  8 MiB /
  29 MiB
  226 MiB
  $ mpirun.hydra -np 2 python2 -c "$CMD"
  8 MiB initial
  8 MiB initial
  27 MiB loaded petsc4py
  27 MiB loaded petsc4py
  77 MiB DMDA
  77 MiB DMDA
  $ mpirun.hydra -np 4 python2 -c "$CMD"
  8 MiB initial
  8 MiB initial
  8 MiB initial
  8 MiB initial
  27 MiB loaded petsc4py
  27 MiB loaded petsc4py
  27 MiB loaded petsc4py
  27 MiB loaded petsc4py
  53 MiB DMDA
  53 MiB DMDA
  53 MiB DMDA
  53 MiB DMDA

and with Barry's patch:

  $ mpirun.hydra -np 1 python2 -c "$CMD"
  8 MiB initial
  29 MiB loaded petsc4py
  172 MiB DMDA
  $ mpirun.hydra -np 2 python2 -c "$CMD"
  8 MiB initial
  8 MiB initial
  27 MiB loaded petsc4py
  27 MiB loaded petsc4py
  51 MiB DMDA
  51 MiB DMDA
  $ mpirun.hydra -np 4 python2 -c "$CMD"
  8 MiB initial
  8 MiB initial
  8 MiB initial
  8 MiB initial
  27 MiB loaded petsc4py
  27 MiB loaded petsc4py
  27 MiB loaded petsc4py
  27 MiB loaded petsc4py
  39 MiB DMDA
  39 MiB DMDA
  38 MiB DMDA
  39 MiB DMDA


The psutil package is in PyPI and most distributions.

pgp6nmyTvvT9A.pgp
Description: PGP signature