On Wed 2009-04-22 21:44, Xuefeng Li wrote: > It feels like KSP for the coarsest level is either not > functional or a direct solver, whereas KSP for finer > levels are iterative solvers. What is the KSP type > associated with SNES on the coarsest level? Is the above > observation by design in Petsc?
Yes, this is by design. The whole point of multigrid is to be able to propagate information globally in each iteration, while spending the minimum effort to do it. Usually this means that the global matrix is small enough to be easily solved with a direct solver, often redundantly. Run with -dmmg_view and look under 'mg_coarse_' to see what is running, and with '-help |grep mg_coarse' to see what you can tune on the command line. If you do iterations on the coarse level, you waste all your time on the network because each processor has almost nothing to do. If you use a domain decomposition preconditioner that deteriorates with the number of subdomains (anything without its *own* coarse-level solve) then you can't observe optimal scaling. The PETSc default is to use the 'preonly' KSP and a redundant direct solve (every process gets the whole matrix coarse-level matrix and solves it sequentially). Sometimes it's not feasible to make the coarse level small enough for such a direct solve to be effective (common with complex geometry). In this case, you can use -mg_coarse_pc_redundant_number and a parallel direct solver (mumps, superlu_dist, etc) to solve semi-redundantly (e.g. groups of 8 processors work together to factor the coarse-level matrix). You are welcome to try an iterative solver on the coarse level, redundantly or otherwise. Jed -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20090423/e21a709d/attachment-0001.pgp>
