Thanks Lorenzo If I well understood what you say is
Set up PCASM for the smoother_down[i] and then feed it up to smoother_up[i] Does this assure that only one memory allocation is done for PCASM? Eugenio ________________________________ From: Lorenzo Alessio Botti [[email protected]] Sent: Thursday, August 20, 2015 5:29 AM To: [email protected] Cc: Aulisa, Eugenio Subject: GMRES -> PCMG -> PCASM pre- post- smoother I tried to achieve this behaviour getting all the smothers and setting the same preconditioner to the down and up smoother on the same level. smoothers.resize(nLevels+1); smoothers_up.resize(nLevels); for (PetscInt i = 0; i < nLevels; i++) { PCMGGetSmootherDown(M_pc,nLevels-i,&(smoothers[i])); KSPSetInitialGuessNonzero(smoothers[i],PETSC_TRUE); // for full and wCicle PCMGGetSmootherUp(M_pc,nLevels-i,&(smoothers_up[i])); } PCMGSetNumberSmoothDown(M_pc,1); PCMGSetNumberSmoothUp(M_pc,1); … set coarse solver options here for (PetscInt i = 0; i < nLevels; i++) { PC pc; KSPSetType(smoothers[i], KSPGMRES); KSPGetPC(smoothers[i], &pc); KSPSetPCSide(smoothers[i], PC_RIGHT); PCSetType(pc, PCASM); PCFactorSetPivotInBlocks(pc, PETSC_TRUE); PCFactorSetAllowDiagonalFill(pc); PCFactorSetReuseFill(pc, PETSC_TRUE); PCFactorSetReuseOrdering(pc, PETSC_TRUE); KSPSetType(smoothers_up[i], KSPGMRES); KSPSetPC(smoothers_up[i], pc); KSPSetPCSide(smoothers_up[i], PC_RIGHT); KSPSetConvergenceTest(smoothers[i],KSPConvergedSkip,NULL,NULL); KSPSetConvergenceTest(smoothers_up[i],KSPConvergedSkip,NULL,NULL); KSPSetNormType(smoothers[i],KSP_NORM_NONE); KSPSetNormType(smoothers_up[i],KSP_NORM_NONE); } Is this correct? Note moreover that for Full Multigrid and W cicles to work as expected I need to add the KSPSetInitialGuessNonZero option. Bests Lorenzo Message: 4 Date: Thu, 20 Aug 2015 02:37:13 -0500 From: Barry Smith <[email protected]<mailto:[email protected]>> To: "Aulisa, Eugenio" <[email protected]<mailto:[email protected]>> Cc: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: [petsc-users] GMRES -> PCMG -> PCASM pre- post- smoother Message-ID: <[email protected]<mailto:[email protected]>> Content-Type: text/plain; charset="us-ascii" What you describe is not the expected behavior. I expected exactly the result that you expected. Do you perhaps have some PETSc options around that may be changing the post-smoother? On the command line or in the file petscrc or in the environmental variable PETSC_OPTIONS? Can you send us some code that we could run that reproduces the problem? Barry On Aug 19, 2015, at 9:26 PM, Aulisa, Eugenio <[email protected]<mailto:[email protected]>> wrote: Hi, I am solving an iteration of GMRES -> PCMG -> PCASM where I build my particular ASM domain decomposition. In setting the PCMG I would like at each level to use the same pre- and post-smoother and for this reason I am using ... PCMGGetSmoother ( pcMG, level , &subksp ); to extract and set at each level the ksp object. In setting PCASM then I use ... KSPGetPC ( subksp, &subpc ); PCSetType ( subpc, PCASM ); ... and then set my own decomposition ... PCASMSetLocalSubdomains(subpc,_is_loc_idx.size(),&_is_ovl[0],&_is_loc[0]); ... Now everything compiles, and runs with no memory leakage, but I do not get the expected convergence. When I checked the output of -ksp_view, I saw something that puzzled me: at each level >0, while in the MG pre-smoother the ASM domain decomposition is the one that I set, for example with 4 processes I get ... Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (level-2) 4 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=1 using preconditioner applied to right hand side for initial guess tolerances: relative=1e-12, absolute=1e-20, divergence=1e+50 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (level-2) 4 MPI processes type: asm Additive Schwarz: total subdomain blocks = 198, amount of overlap = 0 Additive Schwarz: restriction/interpolation type - RESTRICT [0] number of local blocks = 52 [1] number of local blocks = 48 [2] number of local blocks = 48 [3] number of local blocks = 50 Local solve info for each block is in the following KSP and PC objects: - - - - - - - - - - - - - - - - - - ... in the post-smoother I have the default ASM decomposition with overlapping 1: ... Up solver (post-smoother) on level 2 ------------------------------- KSP Object: (level-2) 4 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=2 tolerances: relative=1e-12, absolute=1e-20, divergence=1e+50 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (level-2) 4 MPI processes type: asm Additive Schwarz: total subdomain blocks = 4, amount of overlap = 1 Additive Schwarz: restriction/interpolation type - RESTRICT Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (level-2sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning ... %%%%%%%%%%%%%%%%%%%%%%%% So it seams that by using PCMGGetSmoother ( pcMG, level , &subksp ); I was capable to set both the pre- and post- smoothers to be PCASM but everything I did after that applied only to the pre-smoother, while the post-smoother got the default PCASM options. I know that I can use PCMGGetSmootherDown and PCMGGetSmootherUp, but that would probably double the memory allocation and the computational time in the ASM. Is there any way I can just use PCMGGetSmoother and use the same PCASM in the pre- and post- smoother? I hope I was clear enough. Thanks a lot for your help, Eugenio
