Re: [petsc-users] Obtaining compiling and building information from a.out

2018-04-04 Thread Matthew Knepley 
On Thu, Mar 29, 2018 at 1:48 AM, TAY wee-beng  wrote:

>
> On 28/3/2018 5:45 PM, Matthew Knepley wrote:
>
> On Tue, Mar 27, 2018 at 10:17 PM, TAY wee-beng  wrote:
>
>> Hi Dave,
>>
>> I looked at the output using log_view and re-compile. However, although I
>> use the same options "-xHost -g -O3 -openmp" (some filenames and dir names
>> are now different but they are actually the same), I still get different
>> results timing. I have attach both, fast and slow. So what else can I do to
>> pinpoint the differences?
>>
> The solver options must be different. In Fast, there is almost no time in
> LUFactor, but in Slow its half the time.
>
>Matt
>
> Hi Matt,
>
> Finally found that it was because I was using KSPRICHARDSON in the new
> code. KSPGMRES in the old code is much faster.
>
> I also tried cg, lsqr, fgmres. CG and GMRES seem similar in term of best
> performance. Are there any other recommendations to try? The Poisson eqn is
> symmetric.
>

Krylov solvers fundamentally do not matter for solving elliptic problems.
It comes down to your preconditioner.


> I also tried -pc_type mg -pc_mg_nlevels 2 instead of -poisson_pc_type gamg
> -poisson_pc_gamg_agg_nsmooths 1.
>
> It reduces the runtime from 2.25 to 1.46min.
>

Yes, GMG has faster setup time than AMG. The solve time should be similar.


> However, I found that my subroutine is very similar to my momentum
> subroutine, which is based on KSPSolve:
>
> 0. DMDACreate, DMDACreate3d etc
> 1. Assemble matrix
> 2. KSPSetOperators, KSPSetType etc
> 3. KSPSolve
>
> However, the 2D Poisson example in ex5f.F90 uses SNESSetDM, SNESSolve etc.
> So does it matter if I use SNESSolve or KSPSolve if I'm using -pc_type mg?
>

Not really.

  Thanks,

 Matt


> Thank you very much.
>>
>> Yours sincerely,
>>
>> 
>> TAY Wee-Beng (Zheng Weiming) 郑伟明
>> Personal research webpage: http://tayweebeng.wixsite.com/website
>> Youtube research showcase: 
>> https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA
>> linkedin: www.linkedin.com/in/tay-weebeng
>> 
>>
>> On 27/3/2018 5:22 PM, Dave May wrote:
>>
>>
>>
>> On 27 March 2018 at 10:16, TAY wee-beng  wrote:
>>
>>> Hi,
>>>
>>> I have been compiling and building different version of my CFD with the
>>> intel 2016, 2018 compilers, and also different compiling options.
>>>
>>> I tested a version of my a.out and it is much faster than the other
>>> a.out, using only 3 min instead of more than 10min to solve a certain case
>>> using GAMG.
>>>
>>> However, I can't recall how it was compiled. I only know that I used the
>>> intel 2016 compiler.
>>>
>>> So is there any way I can find out how the a.out was compiled? Like what
>>> options were used?
>>
>>
>> Since you posted to the list I presume "a.out" links against petsc...
>> If so, run your code with
>>   -log_view
>>
>> Upon calling PetscFinalize(), you will get all the options given to PETSc
>> configure, plus the CFLAGS, link lines, etc.
>>
>>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/ 
>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/

Re: [petsc-users] Obtaining compiling and building information from a.out

2018-04-04 Thread TAY wee-beng 

Hi,

I emailed as shown below a while ago but I have yet to get a response. 
Can someone help?


Thank you very much.

Yours sincerely,


TAY Wee-Beng (Zheng Weiming) 郑伟明
Personal research webpage: http://tayweebeng.wixsite.com/website
Youtube research showcase: 
https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA
linkedin: www.linkedin.com/in/tay-weebeng


On 29/3/2018 1:48 PM, TAY wee-beng wrote:


On 28/3/2018 5:45 PM, Matthew Knepley wrote:
On Tue, Mar 27, 2018 at 10:17 PM, TAY wee-beng > wrote:


Hi Dave,

I looked at the output using log_view and re-compile. However,
although I use the same options "-xHost -g -O3 -openmp" (some
filenames and dir names are now different but they are actually
the same), I still get different results timing. I have attach
both, fast and slow. So what else can I do to pinpoint the
differences?

The solver options must be different. In Fast, there is almost no 
time in LUFactor, but in Slow its half the time.


   Matt

Hi Matt,

Finally found that it was because I was using KSPRICHARDSON in the new 
code. KSPGMRES in the old code is much faster.


I also tried cg, lsqr, fgmres. CG and GMRES seem similar in term of 
best performance. Are there any other recommendations to try? The 
Poisson eqn is symmetric.


I also tried -pc_type mg -pc_mg_nlevels 2 instead of -poisson_pc_type 
gamg -poisson_pc_gamg_agg_nsmooths 1.


It reduces the runtime from 2.25 to 1.46min.

However, I found that my subroutine is very similar to my momentum 
subroutine, which is based on KSPSolve:


0. DMDACreate, DMDACreate3d etc
1. Assemble matrix
2. KSPSetOperators, KSPSetType etc
3. KSPSolve

However, the 2D Poisson example in ex5f.F90 uses SNESSetDM, SNESSolve 
etc. So does it matter if I use SNESSolve or KSPSolve if I'm using 
-pc_type mg?



Thank you very much.

Yours sincerely,


TAY Wee-Beng (Zheng Weiming) 郑伟明
Personal research webpage:http://tayweebeng.wixsite.com/website

Youtube research 
showcase:https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA

linkedin:www.linkedin.com/in/tay-weebeng



On 27/3/2018 5:22 PM, Dave May wrote:



On 27 March 2018 at 10:16, TAY wee-beng mailto:zon...@gmail.com>> wrote:

Hi,

I have been compiling and building different version of my
CFD with the intel 2016, 2018 compilers, and also different
compiling options.

I tested a version of my a.out and it is much faster than
the other a.out, using only 3 min instead of more than 10min
to solve a certain case using GAMG.

However, I can't recall how it was compiled. I only know
that I used the intel 2016 compiler.

So is there any way I can find out how the a.out was
compiled? Like what options were used?


Since you posted to the list I presume "a.out" links against
petsc...
If so, run your code with
  -log_view

Upon calling PetscFinalize(), you will get all the options given
to PETSc configure, plus the CFLAGS, link lines, etc.






--
What most experimenters take for granted before they begin their 
experiments is infinitely more interesting than any results to which 
their experiments lead.

-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/

Re: [petsc-users] Obtaining compiling and building information from a.out

2018-03-28 Thread TAY wee-beng 


On 28/3/2018 5:45 PM, Matthew Knepley wrote:
On Tue, Mar 27, 2018 at 10:17 PM, TAY wee-beng > wrote:


Hi Dave,

I looked at the output using log_view and re-compile. However,
although I use the same options "-xHost -g -O3 -openmp" (some
filenames and dir names are now different but they are actually
the same), I still get different results timing. I have attach
both, fast and slow. So what else can I do to pinpoint the
differences?

The solver options must be different. In Fast, there is almost no time 
in LUFactor, but in Slow its half the time.


   Matt

Hi Matt,

Finally found that it was because I was using KSPRICHARDSON in the new 
code. KSPGMRES in the old code is much faster.


I also tried cg, lsqr, fgmres. CG and GMRES seem similar in term of best 
performance. Are there any other recommendations to try? The Poisson eqn 
is symmetric.


I also tried -pc_type mg -pc_mg_nlevels 2 instead of -poisson_pc_type 
gamg -poisson_pc_gamg_agg_nsmooths 1.


It reduces the runtime from 2.25 to 1.46min.

However, I found that my subroutine is very similar to my momentum 
subroutine, which is based on KSPSolve:


0. DMDACreate, DMDACreate3d etc
1. Assemble matrix
2. KSPSetOperators, KSPSetType etc
3. KSPSolve

However, the 2D Poisson example in ex5f.F90 uses SNESSetDM, SNESSolve 
etc. So does it matter if I use SNESSolve or KSPSolve if I'm using 
-pc_type mg?



Thank you very much.

Yours sincerely,


TAY Wee-Beng (Zheng Weiming) 郑伟明
Personal research webpage:http://tayweebeng.wixsite.com/website

Youtube research 
showcase:https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA

linkedin:www.linkedin.com/in/tay-weebeng



On 27/3/2018 5:22 PM, Dave May wrote:



On 27 March 2018 at 10:16, TAY wee-beng mailto:zon...@gmail.com>> wrote:

Hi,

I have been compiling and building different version of my
CFD with the intel 2016, 2018 compilers, and also different
compiling options.

I tested a version of my a.out and it is much faster than the
other a.out, using only 3 min instead of more than 10min to
solve a certain case using GAMG.

However, I can't recall how it was compiled. I only know that
I used the intel 2016 compiler.

So is there any way I can find out how the a.out was
compiled? Like what options were used?


Since you posted to the list I presume "a.out" links against petsc...
If so, run your code with
  -log_view

Upon calling PetscFinalize(), you will get all the options given
to PETSc configure, plus the CFLAGS, link lines, etc.






--
What most experimenters take for granted before they begin their 
experiments is infinitely more interesting than any results to which 
their experiments lead.

-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/

Re: [petsc-users] Obtaining compiling and building information from a.out

2018-03-28 Thread Matthew Knepley 
On Tue, Mar 27, 2018 at 10:17 PM, TAY wee-beng  wrote:

> Hi Dave,
>
> I looked at the output using log_view and re-compile. However, although I
> use the same options "-xHost -g -O3 -openmp" (some filenames and dir names
> are now different but they are actually the same), I still get different
> results timing. I have attach both, fast and slow. So what else can I do to
> pinpoint the differences?
>
> The solver options must be different. In Fast, there is almost no time in
LUFactor, but in Slow its half the time.

   Matt

> Thank you very much.
>
> Yours sincerely,
>
> 
> TAY Wee-Beng (Zheng Weiming) 郑伟明
> Personal research webpage: http://tayweebeng.wixsite.com/website
> Youtube research showcase: 
> https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA
> linkedin: www.linkedin.com/in/tay-weebeng
> 
>
> On 27/3/2018 5:22 PM, Dave May wrote:
>
>
>
> On 27 March 2018 at 10:16, TAY wee-beng  wrote:
>
>> Hi,
>>
>> I have been compiling and building different version of my CFD with the
>> intel 2016, 2018 compilers, and also different compiling options.
>>
>> I tested a version of my a.out and it is much faster than the other
>> a.out, using only 3 min instead of more than 10min to solve a certain case
>> using GAMG.
>>
>> However, I can't recall how it was compiled. I only know that I used the
>> intel 2016 compiler.
>>
>> So is there any way I can find out how the a.out was compiled? Like what
>> options were used?
>
>
> Since you posted to the list I presume "a.out" links against petsc...
> If so, run your code with
>   -log_view
>
> Upon calling PetscFinalize(), you will get all the options given to PETSc
> configure, plus the CFLAGS, link lines, etc.
>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/

Re: [petsc-users] Obtaining compiling and building information from a.out

2018-03-27 Thread TAY wee-beng 

Hi Dave,

I looked at the output using log_view and re-compile. However, although 
I use the same options "-xHost -g -O3 -openmp" (some filenames and dir 
names are now different but they are actually the same), I still get 
different results timing. I have attach both, fast and slow. So what 
else can I do to pinpoint the differences?


Thank you very much.

Yours sincerely,


TAY Wee-Beng (Zheng Weiming) 郑伟明
Personal research webpage: http://tayweebeng.wixsite.com/website
Youtube research showcase: 
https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA
linkedin: www.linkedin.com/in/tay-weebeng


On 27/3/2018 5:22 PM, Dave May wrote:



On 27 March 2018 at 10:16, TAY wee-beng > wrote:


Hi,

I have been compiling and building different version of my CFD
with the intel 2016, 2018 compilers, and also different compiling
options.

I tested a version of my a.out and it is much faster than the
other a.out, using only 3 min instead of more than 10min to solve
a certain case using GAMG.

However, I can't recall how it was compiled. I only know that I
used the intel 2016 compiler.

So is there any way I can find out how the a.out was compiled?
Like what options were used?


Since you posted to the list I presume "a.out" links against petsc...
If so, run your code with
  -log_view

Upon calling PetscFinalize(), you will get all the options given to 
PETSc configure, plus the CFLAGS, link lines, etc.




  0.400   0.400   0.000E+000
  0.000E+000  0.000E+000  0.000E+000
   1.00   0.4000 -40
 AB,AA,BB,CC   -3.119500053173402.72450003441190 
   2.95445312572.9764635751 
 size_x,size_y,size_z 62x140x388
 total grid size =  3367840
 recommended cores (50k / core) =67.35680 
 myid,jsta,jend,ksta,kend   0   1  70   1
  97
 myid,jsta,jend,ksta,kend   7  71 140 292
 388
 myid,jsta,jend,ksta,kend   4   1  70 195
 291
 myid,jsta,jend,ksta,kend   5  71 140 195
 291
 myid,jsta,jend,ksta,kend   6   1  70 292
 388
 myid,jsta,jend,ksta,kend   1  71 140   1
  97
 myid,jsta,jend,ksta,kend   2   1  70  98
 194
 myid,jsta,jend,ksta,kend   3  71 140  98
 194
 min_area,max_area,min_grid_area,ratio  2.491427578150726E-003
  8.513928284063485E-003  6.251E-004   13.6222852545016 
 ratio bet max_area,min_grid_area not ideal
 max element length should be  3.535533905932738E-002
 body_cg_ini  0.117923968064102  -1.965876114406727E-005
   6.83221612299529 
 Warning - length difference between element and cell
 max_element_length,min_element_length,min_delta
  0.157615376756988   5.913693179946990E-002  2.500E-002
 maximum ngh_surfaces and ngh_vertics are2   1
 minimum ngh_surfaces and ngh_vertics are1   1
 min IIB_cell_no  66
 max IIB_cell_no 316
 IIB_cell_no_sum1647
 min equal_size1800
 max equal_size2000
 min I_cell_no 149
 max I_cell_no1955
 I_cell_no_sum7996
 size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
 e(IIB_global_cell_u),size(I_global_cell_u)
 3161955 316195516477996
 IIB_equal_cell_no_u1_max 316
 I_equal_cell_no_u1_max1955
 IIB_I_cell_no_uvw_total11647   0   07996
   0   0
 size(IIB_cell_u),IIB_cell_no_max_cur 316 824
 local IIB_cells size exceed, to increase size
 size(I_cell_u),I_cell_no_max_cur19554004
 local I_cells size exceed, to increase size
 IIB_cell_no_u1_max,I_cell_no_u1_max12366006
 size(IIB_cell_u),IIB_cell_no_max_cur 316 317
 IIB_cells size exceed, to increase size
 IIB_equal_cell_no_u1_max 475
 size(I_cell_u),I_cell_no_max_cur19551959
 I_cells size exceed, to increase size
 I_equal_cell_no_u1_max2938
 size(IIB_global_cell_u1),IIB_global_cell_no_u1_max_cur16471649
 IIB global cells size exceed, to increase size
 size(I_global_cell_u1),I_global_cell_no_u1_max_cur79968001
 I global cells size exceed, to increase size
 IIB_global_cell_no_u1_max,I_global_cell_no_u1_max2473   12001
 IIB_cell_no_u1_max,I_cell_no_u1_max12366006
 time,IIB_I_cell_no_uvw_total1  211649   0   0
8001   0

Re: [petsc-users] Obtaining compiling and building information from a.out

2018-03-27 Thread Dave May 
On 27 March 2018 at 10:16, TAY wee-beng  wrote:

> Hi,
>
> I have been compiling and building different version of my CFD with the
> intel 2016, 2018 compilers, and also different compiling options.
>
> I tested a version of my a.out and it is much faster than the other a.out,
> using only 3 min instead of more than 10min to solve a certain case using
> GAMG.
>
> However, I can't recall how it was compiled. I only know that I used the
> intel 2016 compiler.
>
> So is there any way I can find out how the a.out was compiled? Like what
> options were used?


Since you posted to the list I presume "a.out" links against petsc...
If so, run your code with
  -log_view

Upon calling PetscFinalize(), you will get all the options given to PETSc
configure, plus the CFLAGS, link lines, etc.

[petsc-users] Obtaining compiling and building information from a.out

2018-03-27 Thread TAY wee-beng 

Hi,

I have been compiling and building different version of my CFD with the 
intel 2016, 2018 compilers, and also different compiling options.


I tested a version of my a.out and it is much faster than the other 
a.out, using only 3 min instead of more than 10min to solve a certain 
case using GAMG.


However, I can't recall how it was compiled. I only know that I used the 
intel 2016 compiler.


So is there any way I can find out how the a.out was compiled? Like what 
options were used?


--
Thank you very much.

Yours sincerely,


TAY Wee-Beng (Zheng Weiming) 郑伟明
Personal research webpage: http://tayweebeng.wixsite.com/website
Youtube research showcase: 
https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA
linkedin: www.linkedin.com/in/tay-weebeng