On Thu, Feb 19, 2015 at 12:45 PM, Hong <[email protected]> wrote: > Fabian, > Too much time was spent on the matrix operations during setup phase, which > has plenty room for optimization. >
Yea, a little slow but not crazy. With 2M equation on one proc you may be having cache problems. > Can you provide us a stand-alone code used in your experiment so we can > investigate how to make our gamg more efficient? > > Hong > > On Wed, Feb 18, 2015 at 12:20 PM, Barry Smith <[email protected]> wrote: > >> >> Fabian, >> >> CG requires that the preconditioner be symmetric positive definite. >> ICC even if given a symmetric positive definite matrix can generate an >> indefinite preconditioner. >> >> Similarly if an algebraic multigrid application is not "strong enough" >> it can also result in a preconditioner that is indefinite. >> >> You never want to use ICC for pressure type problems it cannot compete >> with multigrid for large problems so let's forget about ICC and focus on >> the GAMG. >> >> > -pressure_mg_coarse_sub_pc_type svd >> > -pressure_mg_levels_ksp_rtol 1e-4 >> > -pressure_mg_levels_ksp_type richardson >> > -pressure_mg_levels_pc_type sor >> > -pressure_pc_gamg_agg_nsmooths 1 >> > -pressure_pc_type gamg >> >> There are many many tuning parameters for MG. >> >> First, is your pressure problem changing dramatically at each new >> solver? That is, for example, is the mesh moving or are there very >> different numerical values in the matrix? Is the nonzero structure of the >> pressure matrix changing? Currently the entire GAMG process is done for >> each new solve, if you use the flag >> >> -pressure_pc_gamg_reuse_interpolation true >> >> it will create the interpolation needed for GAMG once and reuse it for >> all the solves. Please try that and see what happens. >> >> Then I will have many more suggestions. >> >> >> Barry >> >> >> >> > On Feb 17, 2015, at 9:14 AM, Fabian Gabel <[email protected]> >> wrote: >> > >> > Dear PETSc team, >> > >> > I am trying to optimize the solver parameters for the linear system I >> > get, when I discretize the pressure correction equation Poisson equation >> > with Neumann boundary conditions) in a SIMPLE-type algorithm using a >> > finite volume method. >> > >> > The resulting system is symmetric and positive semi-definite. A basis to >> > the associated nullspace has been provided to the KSP object. >> > >> > Using a CG solver with ICC preconditioning the solver needs a lot of >> > inner iterations to converge (-ksp_monitor -ksp_view output attached for >> > a case with approx. 2e6 unknowns; the lines beginning with 000XXXX show >> > the relative residual regarding the initial residual in the outer >> > iteration no. 1 for the variables u,v,w,p). Furthermore I don't quite >> > understand, why the solver reports >> > >> > Linear solve did not converge due to DIVERGED_INDEFINITE_PC >> > >> > at the later stages of my Picard iteration process (iteration 0001519). >> > >> > I then tried out CG+GAMG preconditioning with success regarding the >> > number of inner iterations, but without advantages regarding wall time >> > (output attached). Also the DIVERGED_INDEFINITE_PC reason shows up >> > repeatedly after iteration 0001487. I used the following options >> > >> > -pressure_mg_coarse_sub_pc_type svd >> > -pressure_mg_levels_ksp_rtol 1e-4 >> > -pressure_mg_levels_ksp_type richardson >> > -pressure_mg_levels_pc_type sor >> > -pressure_pc_gamg_agg_nsmooths 1 >> > -pressure_pc_type gamg >> > >> > I would like to get an opinion on how the solver performance could be >> > increased further. -log_summary shows that my code spends 80% of the >> > time solving the linear systems for the pressure correction (STAGE 2: >> > PRESSCORR). Furthermore, do you know what could be causing the >> > DIVERGED_INDEFINITE_PC converged reason? >> > >> > Regards, >> > Fabian Gabel >> > <gamg.128.out.531314><icc.128.out.429762> >> >> >
