It seems performance are always the same whatever the number of threads.
I ran all these tests on my laptop (which is not a "great supercomputer" !... 
As Matt noticed, and as I knew it !)

FH

~/Bandwidth>export OMP_NUM_THREADS=1
~/Bandwidth>./stream_c.exe > stream_1thread.log
~/Bandwidth>export OMP_NUM_THREADS=2
~/Bandwidth>./stream_c.exe > stream_2threads.log
~/Bandwidth>export OMP_NUM_THREADS=4
~/Bandwidth>./stream_c.exe > stream_4threads.log
~/Bandwidth>grep Triad *log
stream_1thread.log:Triad:          13057.6     0.018427     0.018380     
0.018539
stream_2threads.log:Triad:          13044.9     0.018496     0.018398     
0.018936
stream_4threads.log:Triad:          13049.1     0.018449     0.018392     
0.018561

________________________________________
De : Jed Brown [[email protected]] de la part de Jed Brown 
[[email protected]]
Date d'envoi : vendredi 21 juin 2013 12:41
À : HOUSSEN Franck; Barry Smith
Cc: [email protected]
Objet : Re: [petsc-users] RE :  How to get bandwidth peak out of PETSc log ?

HOUSSEN Franck <[email protected]> writes:

> MatMult            13216 1.0 5.4614e+01 1.0 5.52e+10 1.0 0.0e+00 0.0e+00 
> 0.0e+00 21 56  0  0  0  21 56  0  0  0  1010

Much of your time is spent here, which is bandwidth limited.  It needs 6
bytes per flop (plus a little, if the vector is perfectly reused) , so
this number is about 6 GB/s.

> MatMult            13550 1.0 3.8204e+01 1.0 2.85e+10 1.0 2.7e+04 3.5e+04 
> 0.0e+00 23 55 79 93  0  23 55 79 93  0  1494

Here with two processes, you have about 9 GB/s.

Was your STREAM test (getting 11 GB/s) using multiple threads/processes?
Can you send STREAM results for one and for two threads?  50% of STREAM
is not very good (though it's actually the best you can do on some funny
architectures), 70-85% is what we expect.

If you're getting a low fraction of peak in MatMult, try reordering your
matrix to have lower bandwidth.  You can use MatGetOrdering with RCM for
this.
-------------------------------------------------------------
STREAM version $Revision: 5.10 $
-------------------------------------------------------------
This system uses 8 bytes per array element.
-------------------------------------------------------------
Array size = 10000000 (elements), Offset = 0 (elements)
Memory per array = 76.3 MiB (= 0.1 GiB).
Total memory required = 228.9 MiB (= 0.2 GiB).
Each kernel will be executed 10 times.
 The *best* time for each kernel (excluding the first iteration)
 will be used to compute the reported bandwidth.
-------------------------------------------------------------
Your clock granularity/precision appears to be 1 microseconds.
Each test below will take on the order of 10722 microseconds.
   (= 10722 clock ticks)
Increase the size of the arrays if this shows that
you are not getting at least 20 clock ticks per test.
-------------------------------------------------------------
WARNING -- The above is only a rough guideline.
For best results, please be sure you know the
precision of your system timer.
-------------------------------------------------------------
Function    Best Rate MB/s  Avg time     Min time     Max time
Copy:           11508.2     0.013937     0.013903     0.013992
Scale:          11433.5     0.014024     0.013994     0.014049
Add:            13026.5     0.018449     0.018424     0.018479
Triad:          13057.6     0.018427     0.018380     0.018539
-------------------------------------------------------------
Solution Validates: avg error less than 1.000000e-13 on all three arrays
-------------------------------------------------------------
-------------------------------------------------------------
STREAM version $Revision: 5.10 $
-------------------------------------------------------------
This system uses 8 bytes per array element.
-------------------------------------------------------------
Array size = 10000000 (elements), Offset = 0 (elements)
Memory per array = 76.3 MiB (= 0.1 GiB).
Total memory required = 228.9 MiB (= 0.2 GiB).
Each kernel will be executed 10 times.
 The *best* time for each kernel (excluding the first iteration)
 will be used to compute the reported bandwidth.
-------------------------------------------------------------
Your clock granularity/precision appears to be 1 microseconds.
Each test below will take on the order of 10965 microseconds.
   (= 10965 clock ticks)
Increase the size of the arrays if this shows that
you are not getting at least 20 clock ticks per test.
-------------------------------------------------------------
WARNING -- The above is only a rough guideline.
For best results, please be sure you know the
precision of your system timer.
-------------------------------------------------------------
Function    Best Rate MB/s  Avg time     Min time     Max time
Copy:           11490.1     0.013941     0.013925     0.013959
Scale:          11420.3     0.014031     0.014010     0.014063
Add:            13021.4     0.018453     0.018431     0.018503
Triad:          13044.9     0.018496     0.018398     0.018936
-------------------------------------------------------------
Solution Validates: avg error less than 1.000000e-13 on all three arrays
-------------------------------------------------------------
-------------------------------------------------------------
STREAM version $Revision: 5.10 $
-------------------------------------------------------------
This system uses 8 bytes per array element.
-------------------------------------------------------------
Array size = 10000000 (elements), Offset = 0 (elements)
Memory per array = 76.3 MiB (= 0.1 GiB).
Total memory required = 228.9 MiB (= 0.2 GiB).
Each kernel will be executed 10 times.
 The *best* time for each kernel (excluding the first iteration)
 will be used to compute the reported bandwidth.
-------------------------------------------------------------
Your clock granularity/precision appears to be 1 microseconds.
Each test below will take on the order of 10723 microseconds.
   (= 10723 clock ticks)
Increase the size of the arrays if this shows that
you are not getting at least 20 clock ticks per test.
-------------------------------------------------------------
WARNING -- The above is only a rough guideline.
For best results, please be sure you know the
precision of your system timer.
-------------------------------------------------------------
Function    Best Rate MB/s  Avg time     Min time     Max time
Copy:           11487.5     0.013973     0.013928     0.014067
Scale:          11414.0     0.014044     0.014018     0.014082
Add:            13029.3     0.018469     0.018420     0.018577
Triad:          13049.1     0.018449     0.018392     0.018561
-------------------------------------------------------------
Solution Validates: avg error less than 1.000000e-13 on all three arrays
-------------------------------------------------------------

Reply via email to