Re: [petsc-users] Make stream

2020-06-16 Thread Fande Kong
Thanks, Jed, It is fascinating. I will try to check if I can do anything to have this kind of improvement as well. Thanks, Fande, On Fri, Jun 12, 2020 at 7:43 PM Jed Brown wrote: > Jed Brown writes: > > > Fande Kong writes: > > > >>> There's a lot more to AMG setup than memory bandwidth

Re: [petsc-users] Make stream

2020-06-12 Thread Jed Brown
Jed Brown writes: > Fande Kong writes: > >>> There's a lot more to AMG setup than memory bandwidth (architecture >>> matters a lot, even between different generation CPUs). >> >> >> Could you elaborate a bit more on this? From my understanding, one big part >> of AMG SetUp is RAP that should be

Re: [petsc-users] Make stream

2020-06-09 Thread Jed Brown
Fande Kong writes: > Thanks so much, Barry, > > On Tue, Jun 9, 2020 at 3:08 PM Barry Smith wrote: > >> >>You might look at the notes about MPI binding. It might give you a bit >> better performance. >> https://www.mcs.anl.gov/petsc/documentation/faq.html#computers >> > > I am using

Re: [petsc-users] Make stream

2020-06-09 Thread Jed Brown
Fande Kong writes: >> There's a lot more to AMG setup than memory bandwidth (architecture >> matters a lot, even between different generation CPUs). > > > Could you elaborate a bit more on this? From my understanding, one big part > of AMG SetUp is RAP that should be pretty much bandwidth. The

Re: [petsc-users] Make stream

2020-06-09 Thread Fande Kong
Thanks, Jed, On Tue, Jun 9, 2020 at 3:19 PM Jed Brown wrote: > Fande Kong writes: > > > Hi All, > > > > I am trying to interpret the results from "make stream" on two compute > > nodes, where each node has 48 cores. > > > > If my calculations are memory bandwidth limited, such as AMG, MatVec,

Re: [petsc-users] Make stream

2020-06-09 Thread Fande Kong
Thanks so much, Barry, On Tue, Jun 9, 2020 at 3:08 PM Barry Smith wrote: > >You might look at the notes about MPI binding. It might give you a bit > better performance. > https://www.mcs.anl.gov/petsc/documentation/faq.html#computers > I am using mvapich2, and still trying to look for

Re: [petsc-users] Make stream

2020-06-09 Thread Junchao Zhang
Looks something was wrong that you need all cores to reach its bandwidth limit. Run on one node to see what happens. --Junchao Zhang On Tue, Jun 9, 2020 at 3:45 PM Fande Kong wrote: > Hi All, > > I am trying to interpret the results from "make stream" on two compute > nodes, where each node

Re: [petsc-users] Make stream

2020-06-09 Thread Jed Brown
Fande Kong writes: > Hi All, > > I am trying to interpret the results from "make stream" on two compute > nodes, where each node has 48 cores. > > If my calculations are memory bandwidth limited, such as AMG, MatVec, > GMRES, etc.. There's a lot more to AMG setup than memory bandwidth

Re: [petsc-users] Make stream

2020-06-09 Thread Barry Smith
You might look at the notes about MPI binding. It might give you a bit better performance. https://www.mcs.anl.gov/petsc/documentation/faq.html#computers The streams is exactly the DAXPY operation so this is the speed up

[petsc-users] Make stream

2020-06-09 Thread Fande Kong
Hi All, I am trying to interpret the results from "make stream" on two compute nodes, where each node has 48 cores. If my calculations are memory bandwidth limited, such as AMG, MatVec, GMRES, etc.. The best speedup I could get is 16.6938 if I start from one core?? The speedup for function