Ari, I would also check that your operator is not messed up in parallel. The solver is looking pretty solid.
Also, you can configure PETSc with hypre and use '-pc_type hypre'. If hypre is also good in serial but hosed on multi-proc then it is most probably your operator. On Tue, Jul 26, 2016 at 6:58 PM, Mark Adams <[email protected]> wrote: > So remove one of the -mg_levels_pc_type jacobi and add -mg_coarse_ksp_type > preonly, then verify that this works on one proc and then try two procs. > > On Tue, Jul 26, 2016 at 6:56 PM, Mark Adams <[email protected]> wrote: > >> Oh, actually this worked. You have this ...pc_type jacobi in there >> twice, so one of them was "unused". >> >> Try this with 2 processors now. >> >> On Tue, Jul 26, 2016 at 6:42 PM, Mark Adams <[email protected]> wrote: >> >>> >>> >>> On Tue, Jul 26, 2016 at 6:24 PM, Ari Rappaport <[email protected]> wrote: >>> >>>> So I commented out the line PCSetType(pc, PCGAMG). The line >>>> KSPSetFromOptions(ksp) was already in the code at the end of our >>>> initialization routine. I also added .petscrc to the working dir. Here is >>>> the current output. It seems as if Option left: name:-mg_levels_pc_type >>>> jacobi (no value) is still present in the output..I dunno. >>>> >>> >>> Yea, I dunno either. If you use -help you will get printout of the >>> available options. If you do this you will see stuff like -mg_levels_1_ >>> ... you can also see this in the ksp_view. There is a shortcut that lets >>> you _not_ put "_1" in. Try putting this in for each level like so: >>> >>> -mg_levels_1_pc_type jacobi >>> -mg_levels_2_pc_type jacobi >>> -mg_levels_3_pc_type jacobi >>> >>> I also notice that the coarse grid ksp is GMRES. This is our fault. It >>> should be preonly. Add: >>> >>> -mg_coarse_ksp_type preonly >>> >>> >>> >>> >>> >>> >>> >>> >>>> >>>> -Ari >>>> >>>> ----- Original Message ----- >>>> From: "Mark Adams" <[email protected]> >>>> To: "Ari Rappaport" <[email protected]>, "For users of the development >>>> version of PETSc" <[email protected]> >>>> Sent: Tuesday, July 26, 2016 4:03:03 PM >>>> Subject: Re: [petsc-dev] Algebraic Multigrid >>>> >>>> >>>> At the end of this you have: >>>> >>>> >>>> >>>> #PETSc Option Table entries: >>>> -ksp_view >>>> -mg_levels_pc_type jacobi >>>> -options_left >>>> #End of PETSc Option Table entries >>>> There is one unused database option. It is: >>>> Option left: name:-mg_levels_pc_type jacobi (no value) >>>> >>>> >>>> So this jacobi parameter is not being used. >>>> >>>> >>>> Do you call KPSSetFromOptions? Do you set solver parameters in the >>>> code? Like PCGAMG? >>>> >>>> >>>> You should not set anything in the code, it just confuses things at >>>> this point. Use KSPSetFromOptions(). You can hardwire stuff before this >>>> call, this just lets you set the defaults, but you should always call this >>>> last to let command line parameters override the defaults. >>>> >>>> >>>> You can put this in a .petscrc file in the working directory and try >>>> again. >>>> >>>> >>>> >>>> -ksp_type cg >>>> -ksp_max_it 50 >>>> >>>> -ksp_rtol 1.e-6 >>>> -ksp_converged_reason >>>> -pc_type gamg >>>> -pc_gamg_type agg >>>> -pc_gamg_agg_nsmooths 1 >>>> -pc_gamg_coarse_eq_limit 10 >>>> -pc_gamg_reuse_interpolation true >>>> -pc_gamg_square_graph 1 >>>> -pc_gamg_threshold -0.05 >>>> -mg_levels_ksp_max_it 2 >>>> -mg_levels_ksp_type chebyshev >>>> -mg_levels_esteig_ksp_type cg >>>> -mg_levels_esteig_ksp_max_it 10 >>>> -mg_levels_ksp_chebyshev_esteig 0,.05,0,1.05 >>>> -mg_levels_pc_type jacobi >>>> -pc_hypre_type boomeramg >>>> -pc_hypre_boomeramg_no_CF >>>> -pc_hypre_boomeramg_agg_nl 1 >>>> -pc_hypre_boomeramg_coarsen_type HMIS >>>> -pc_hypre_boomeramg_interp_type ext+i >>>> >>>> >>> >> >
