On Wed, Aug 17, 2016 at 4:33 PM, Ari Rappaport <[email protected]> wrote:
> Ok, thank you. So for AMS there is the option to use > PCHYPRESetEdgeConstantVectors. Is there any example somewhere of how this > function is supposed to be used? This is probably a wrapper for a hypre method so I'd check the hypre docs. > Also, it is possible to set the pc type to hypre inside the code? e.g. > PCSetType(pc, <HYPRE>) > > yes > -Ari > > ----- Original Message ----- > From: "Satish Balay" <[email protected]> > To: "Ari Rappaport" <[email protected]> > Cc: "petsc-dev" <[email protected]> > Sent: Wednesday, August 17, 2016 1:11:43 PM > Subject: Re: [petsc-dev] Algebraic Multigrid > > balay@asterix /home/balay/petsc/src/ksp/ksp/examples/tutorials (master=) > $ ./ex2 -pc_type hypre -h |grep pc_hypre_type > -pc_hypre_type <boomeramg> (choose one of) pilut parasails boomeramg ams > (PCHYPRESetType) > > > Satish > > On Wed, 17 Aug 2016, Ari Rappaport wrote: > > > Hi again, > > I am having a problem with the database option -pc_hypre_set_type ams > along with -pc_type hypre. When I run my simulation I get the message > "Option left: name:-pc_hypre_set_type value: ams". > > > > -Ari > > > > ----- Original Message ----- > > From: "Satish Balay" <[email protected]> > > To: "petsc-dev" <[email protected]> > > Cc: "Ari Rappaport" <[email protected]> > > Sent: Tuesday, August 16, 2016 3:31:16 PM > > Subject: Re: [petsc-dev] Algebraic Multigrid > > > > Or - not use '--with-fc=0'. For some reason '-lm' is getting pulled in > via gfortran - not gcc [on this machine] > > > > I've pushed the following fix - to maint/3.7 > > > > https://bitbucket.org/petsc/petsc/commits/2d90af56370c3153360ed340bfa06d > 4e662fceff > > > > Satish > > > > On Tue, 16 Aug 2016, Satish Balay wrote: > > > > > >> > > > /home/mantis/petsc/petsc-3.6.4/linux-gnu/externalpackages/ > hypre-2.10.0b-p4/src/utilities/exchange_data.c:205: undefined reference > to `ceil' > > > << > > > > > > Try adding configure flag: LIBS=-lm > > > > > > BTW: > > > > > > >>>>>>> > > > Working directory: /home/mantis/petsc/petsc-3.6.4 > > > > > > Executing: mpicc -show > > > stdout: gcc -fPIC -g3 -I/home/mantis/petsc/petsc-3.7.2/linux-gnu/include > -L/home/mantis/petsc/petsc-3.7.2/linux-gnu/lib -Wl,-rpath > -Wl,/home/mantis/petsc/petsc-3.7.2/linux-gnu/lib -Wl,--enable-new-dtags > -lmpi > > > <<<<<<< > > > > > > Mixing petsc versions might cause issues.. > > > > > > Satish > > > > > > On Tue, 16 Aug 2016, Ari Rappaport wrote: > > > > > > > Hi Barry, > > > > Sorry for the late reply. Here is my configure.log file. I am > assuming you meant the configure.log in the petsc-3.6.4 home directory. > > > > > > > > -Ari > > > > > > > > ----- Original Message ----- > > > > From: "Barry Smith" <[email protected]> > > > > To: "Ari Rappaport" <[email protected]> > > > > Cc: "Mark Adams" <[email protected]>, "For users of the development > version of PETSc" <[email protected]> > > > > Sent: Thursday, August 11, 2016 3:59:47 PM > > > > Subject: Re: [petsc-dev] Algebraic Multigrid > > > > > > > > > > > > > On Aug 11, 2016, at 4:42 PM, Ari Rappaport <[email protected]> > wrote: > > > > > > > > > > Hi Guys, > > > > > So the algebraic multigrid works great for a Poisson equation. > However, it's choking on our Maxwell's Equations solver which is a form of > curl/curl if I'm not mistaken. > > > > > > > > Maxwell's equations are tricky for multigrid. hypre has a solver > specific for Maxwell's equation, you need to check it out; note this is not > just running with hypre boomerAMG. If you run boomerAMG directly with > Maxwell's you will not get satisfactory results. > > > > > > > > > I was going to give hypre AMS a try, but I'm having trouble > installing it. I'm getting the error "Downloaded hypre could not be used". > > > > > > > > Send configure.log > > > > > > > > > > > > > I'm using Ubuntu 14.04 if that makes a difference. Also, is > non-algebraic multigrid a reasonable choice for Maxwell's Equations? > > > > > > > > Even with geometric multigrid Maxwell's equations are nontrivial; > you will need to do some literature search to determine how to handle the > restriction/interpolation and smoothing to work well for Maxwell. > > > > > > > > Barry > > > > > > > > > > > > > > Thanks, > > > > > Ari > > > > > > > > > > ----- Original Message ----- > > > > > From: "Barry Smith" <[email protected]> > > > > > To: "Ari Rappaport" <[email protected]> > > > > > Cc: "Mark Adams" <[email protected]>, "For users of the development > version of PETSc" <[email protected]> > > > > > Sent: Wednesday, August 10, 2016 2:02:10 PM > > > > > Subject: Re: [petsc-dev] Algebraic Multigrid > > > > > > > > > > > > > > >> On Aug 10, 2016, at 2:52 PM, Ari Rappaport <[email protected]> > wrote: > > > > >> > > > > >> Hi Mark, > > > > >> There was indeed a bug on our end. Now that we fixed it > everything is working correctly even in parallel. I have one question still > though. Is there a way to get the mg_levels_pc_type jacobi option > > > > >> set in the code without passing it as an argument to > PetscInitialize? I'm using PCGAMGSetType(pc, PCJACOBI) but it doesn't seem > to be working. > > > > > > > > > > You can call PetscOptionsSetValue("-<appropriate > prefix>_mg_levels_pc_type","jacobi") in the code anywhere before your > code creates the SNES or KSP object you are using. This is equivalent to > putting it on the command line. > > > > > > > > > > Barry > > > > > > > > > >> > > > > >> > > > > >> -Ari > > > > >> > > > > >> ----- Original Message ----- > > > > >> From: "Mark Adams" <[email protected]> > > > > >> To: "Ari Rappaport" <[email protected]> > > > > >> Cc: "For users of the development version of PETSc" < > [email protected]> > > > > >> Sent: Wednesday, July 27, 2016 7:24:24 PM > > > > >> Subject: Re: [petsc-dev] Algebraic Multigrid > > > > >> > > > > >> There is clearly something very wrong with your code. I would > suggest starting with an example code for instance > KSP/examples/tutorial/ex56.c and incrementally add your operator to get > something that is working. > > > > >> > > > > >> > > > > >> Also if you have multiple degrees of freedom per vertex, for > instance elasticity, then you want to set the block size of the matrix > accordingly. > > > > >> > > > > >> On Wednesday, July 27, 2016, Ari Rappaport < [email protected] > > wrote: > > > > >> > > > > >> > > > > >> Hi Mark, > > > > >> We added the Jacobi line and it appears to accept that flag now. > However, we are getting all zeros for the solution vector. And PETSc is > claiming to have converged in 7 iterations to the relative tolerance. > > > > >> > > > > >> -Ari > > > > >> > > > > >> ----- Original Message ----- > > > > >> From: "Mark Adams" < [email protected] > > > > > >> To: "Ari Rappaport" < [email protected] >, "For users of the > development version of PETSc" < [email protected] > > > > > >> Sent: Wednesday, July 27, 2016 3:26:03 PM > > > > >> Subject: Re: [petsc-dev] Algebraic Multigrid > > > > >> > > > > >> > > > > >> Please keep this on the Petsc list. > > > > >> > > > > >> We seem to have lost Jacobi smoother again. I'm suspecting that > you're some funny character in your line with Jacobi that is Confusing the > parser. Get back to the old file with the two two Jacobi entries and delete > the other line and get Jacobi in the KSP view output. There should be no > SOR in the output. > > > > >> > > > > >> > > > > >> On Wednesday, July 27, 2016, Ari Rappaport < [email protected] > > wrote: > > > > >> > > > > >> > > > > >> Hi Mark, > > > > >> I added all these new things. The PCAMG is now ending very > quickly but the residual is unreasonably large by about 10 orders of > magnitude. I noticed the line "Linear solve did not converge due to > DIVERGED_INDEFINITE_PC iterations 2" in the output, could this be causing > the problem? It only appears to be going for 2 iterations now. > > > > >> > > > > >> -Ari > > > > >> > > > > >> ----- Original Message ----- > > > > >> From: "Mark Adams" < [email protected] > > > > > >> To: "Ari Rappaport" < [email protected] > > > > > >> Cc: "For users of the development version of PETSc" < > [email protected] > > > > > >> Sent: Tuesday, July 26, 2016 5:07:34 PM > > > > >> Subject: Re: [petsc-dev] Algebraic Multigrid > > > > >> > > > > >> > > > > >> Ari, I would also check that your operator is not messed up in > parallel. The solver is looking pretty solid. > > > > >> > > > > >> > > > > >> Also, you can configure PETSc with hypre and use '-pc_type > hypre'. If hypre is also good in serial but hosed on multi-proc then it is > most probably your operator. > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> On Tue, Jul 26, 2016 at 6:58 PM, Mark Adams < [email protected] > > wrote: > > > > >> > > > > >> > > > > >> > > > > >> So remove one of the -mg_levels_pc_type jacobi and add > -mg_coarse_ksp_type preonly, then verify that this works on one proc and > then try two procs. > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> On Tue, Jul 26, 2016 at 6:56 PM, Mark Adams < [email protected] > > wrote: > > > > >> > > > > >> > > > > >> > > > > >> Oh, actually this worked. You have this ...pc_type jacobi in > there twice, so one of them was "unused". > > > > >> > > > > >> > > > > >> Try this with 2 processors now. > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> On Tue, Jul 26, 2016 at 6:42 PM, Mark Adams < [email protected] > > wrote: > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> On Tue, Jul 26, 2016 at 6:24 PM, Ari Rappaport < [email protected] > > wrote: > > > > >> > > > > >> > > > > >> So I commented out the line PCSetType(pc, PCGAMG). The line > KSPSetFromOptions(ksp) was already in the code at the end of our > initialization routine. I also added .petscrc to the working dir. Here is > the current output. It seems as if Option left: name:-mg_levels_pc_type > jacobi (no value) is still present in the output..I dunno. > > > > >> > > > > >> > > > > >> > > > > >> Yea, I dunno either. If you use -help you will get printout of > the available options. If you do this you will see stuff like -mg_levels_1_ > ... you can also see this in the ksp_view. There is a shortcut that lets > you _not_ put "_1" in. Try putting this in for each level like so: > > > > >> > > > > >> > > > > >> > > > > >> -mg_levels_1_pc_type jacobi > > > > >> > > > > >> -mg_levels_2_pc_type jacobi > > > > >> -mg_levels_3_pc_type jacobi > > > > >> > > > > >> > > > > >> > > > > >> I also notice that the coarse grid ksp is GMRES. This is our > fault. It should be preonly. Add: > > > > >> > > > > >> > > > > >> -mg_coarse_ksp_type preonly > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> -Ari > > > > >> > > > > >> ----- Original Message ----- > > > > >> From: "Mark Adams" < [email protected] > > > > > >> To: "Ari Rappaport" < [email protected] >, "For users of the > development version of PETSc" < [email protected] > > > > > >> Sent: Tuesday, July 26, 2016 4:03:03 PM > > > > >> Subject: Re: [petsc-dev] Algebraic Multigrid > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> At the end of this you have: > > > > >> > > > > >> > > > > >> > > > > >> #PETSc Option Table entries: > > > > >> -ksp_view > > > > >> -mg_levels_pc_type jacobi > > > > >> -options_left > > > > >> #End of PETSc Option Table entries > > > > >> There is one unused database option. It is: > > > > >> Option left: name:-mg_levels_pc_type jacobi (no value) > > > > >> > > > > >> > > > > >> So this jacobi parameter is not being used. > > > > >> > > > > >> > > > > >> Do you call KPSSetFromOptions? Do you set solver parameters in > the code? Like PCGAMG? > > > > >> > > > > >> > > > > >> You should not set anything in the code, it just confuses things > at this point. Use KSPSetFromOptions(). You can hardwire stuff before this > call, this just lets you set the defaults, but you should always call this > last to let command line parameters override the defaults. > > > > >> > > > > >> > > > > >> You can put this in a .petscrc file in the working directory and > try again. > > > > >> > > > > >> > > > > >> > > > > >> -ksp_type cg > > > > >> -ksp_max_it 50 > > > > >> > > > > >> -ksp_rtol 1.e-6 > > > > >> -ksp_converged_reason > > > > >> -pc_type gamg > > > > >> -pc_gamg_type agg > > > > >> -pc_gamg_agg_nsmooths 1 > > > > >> -pc_gamg_coarse_eq_limit 10 > > > > >> -pc_gamg_reuse_interpolation true > > > > >> -pc_gamg_square_graph 1 > > > > >> -pc_gamg_threshold -0.05 > > > > >> -mg_levels_ksp_max_it 2 > > > > >> -mg_levels_ksp_type chebyshev > > > > >> -mg_levels_esteig_ksp_type cg > > > > >> -mg_levels_esteig_ksp_max_it 10 > > > > >> -mg_levels_ksp_chebyshev_esteig 0,.05,0,1.05 > > > > >> -mg_levels_pc_type jacobi > > > > >> -pc_hypre_type boomeramg > > > > >> -pc_hypre_boomeramg_no_CF > > > > >> -pc_hypre_boomeramg_agg_nl 1 > > > > >> -pc_hypre_boomeramg_coarsen_type HMIS > > > > >> -pc_hypre_boomeramg_interp_type ext+i > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > >
