Please send also the -log_summary for 1 process. Note that in the run you provided the time spent in PETSc is about 25 percent of the total run. So how the "other" portion of the code scales will make a large difference for speedup, hence we need to see with a different number of processes.
Barry On Jun 20, 2013, at 3:24 PM, HOUSSEN Franck <[email protected]> wrote: > Hello, > > I am new to PETSc. > > I have written a (MPI) PETSc code to solve an AX=B system : how to know the > bandwidth peak for a given run ? The code does not scale as I would expect > (doubling the number of MPI processes does not half the elapsed time) : I > would like to understand if this behavior is related to a bad MPI > parallelization (that I may be able to improve), or, to the fact that the > bandwidth limit as been reached (and in this case, my understanding is that I > can not do anything to improve neither the performance, nor the scaling). I > would like to know what's going on and why ! > > Concerning the computer, I have tried to estimate the bandwidth peak with the > "stream benchmark" : www.streambench.org/index.html. I get this : > ~>./stream_c.exe > ... > Function Best Rate MB/s Avg time Min time Max time > Copy: 11473.8 0.014110 0.013945 0.015064 > Scale: 11421.2 0.014070 0.014009 0.014096 > Add: 12974.4 0.018537 0.018498 0.018590 > Triad: 12964.6 0.018683 0.018512 0.019277 > As a conclusion, my understanding is that the bandwidth peak of my computer > is about 12 200 MB / s (= average between 11 400 and 12 900) which is about > 12200 / 1024 = 11.9 GB/s. > > Concerning PETSc, I tried to find (without success) a figure to compare to > 11.9 GB/s. First, I tried to run "make streams" but it doesn't compile (I run > petsc3.4.1. on Ubuntu 12.04). Then, I looked into the PETSc log (using > -log_summary and -ksp_view) but I was not able to find out an estimate of the > bandwidth out of it (I get information about time and flops but not about the > amount of data transfered between MPI processes in MB). How can I get > bandwidth peak out of PETSc log ? Is there a specific option for that or is > this not possible ? > > I have attached the PETSc log (MPI run over 2 processes). > > Thanks, > > FH > <PETSc.log>
