> On Jan 25, 2016, at 11:13 AM, Hoang Giang Bui <[email protected]> wrote: > > OK, let's come back to my problem. I got your point about the interaction > between components in one block. In my case, the interaction is strong. > > As you said, I try this: > > ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > ierr = PCFieldSplitGetSubKSP(pc, &nsplits, &sub_ksp); CHKERRQ(ierr); > ksp_U = sub_ksp[0]; > ierr = KSPGetOperators(ksp_U, &A_U, &P_U); CHKERRQ(ierr); > ierr = MatSetBlockSize(A_U, 3); CHKERRQ(ierr); > ierr = MatSetBlockSize(P_U, 3); CHKERRQ(ierr); > ierr = PetscFree(sub_ksp); CHKERRQ(ierr); > > But it seems doesn't work. The output from -ksp_view shows that matrix passed > to Hypre still has bs=1
Hmm, this is strange. MatSetBlockSize() should have either set the block size to 3 or generated an error. Can you run in the debugger on one process and put a break point in MatSetBlockSize() and see what it is setting the block size to. Then in PCSetUp_hypre() you can see what it is passing to hypre as the block size and maybe figure out how it becomes 1. Barry > > KSP Object: (fieldsplit_u_) 8 MPI processes > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using NONE norm type for convergence test > PC Object: (fieldsplit_u_) 8 MPI processes > type: hypre > HYPRE BoomerAMG preconditioning > HYPRE BoomerAMG: Cycle type V > HYPRE BoomerAMG: Maximum number of levels 25 > HYPRE BoomerAMG: Maximum number of iterations PER hypre call 1 > HYPRE BoomerAMG: Convergence tolerance PER hypre call 0 > HYPRE BoomerAMG: Threshold for strong coupling 0.25 > HYPRE BoomerAMG: Interpolation truncation factor 0 > HYPRE BoomerAMG: Interpolation: max elements per row 0 > HYPRE BoomerAMG: Number of levels of aggressive coarsening 0 > HYPRE BoomerAMG: Number of paths for aggressive coarsening 1 > HYPRE BoomerAMG: Maximum row sums 0.9 > HYPRE BoomerAMG: Sweeps down 1 > HYPRE BoomerAMG: Sweeps up 1 > HYPRE BoomerAMG: Sweeps on coarse 1 > HYPRE BoomerAMG: Relax down symmetric-SOR/Jacobi > HYPRE BoomerAMG: Relax up symmetric-SOR/Jacobi > HYPRE BoomerAMG: Relax on coarse Gaussian-elimination > HYPRE BoomerAMG: Relax weight (all) 1 > HYPRE BoomerAMG: Outer relax weight (all) 1 > HYPRE BoomerAMG: Using CF-relaxation > HYPRE BoomerAMG: Measure type local > HYPRE BoomerAMG: Coarsen type PMIS > HYPRE BoomerAMG: Interpolation type classical > linear system matrix = precond matrix: > Mat Object: (fieldsplit_u_) 8 MPI processes > type: mpiaij > rows=792333, cols=792333 > total: nonzeros=1.39004e+08, allocated nonzeros=1.39004e+08 > total number of mallocs used during MatSetValues calls =0 > using I-node (on process 0) routines: found 30057 nodes, limit used > is 5 > > In other test, I can see the block size bs=3 in the section of Mat Object > > Regardless the setup cost of Hypre AMG, I saw it gives quite a radical > performance, providing that the material parameters does not vary strongly, > and the geometry is regular enough. > > > Giang > > On Fri, Jan 22, 2016 at 2:57 PM, Matthew Knepley <[email protected]> wrote: > On Fri, Jan 22, 2016 at 7:27 AM, Hoang Giang Bui <[email protected]> wrote: > DO you mean the option pc_fieldsplit_block_size? In this thread: > > http://petsc-users.mcs.anl.narkive.com/qSHIOFhh/fieldsplit-error > > No. "Block Size" is confusing on PETSc since it is used to do several things. > Here block size > is being used to split the matrix. You do not need this since you are > prescribing your splits. The > matrix block size is used two ways: > > 1) To indicate that matrix values come in logically dense blocks > > 2) To change the storage to match this logical arrangement > > After everything works, we can just indicate to the submatrix which is > extracted that it has a > certain block size. However, for the Laplacian I expect it not to matter. > > It assumes you have a constant number of fields at each grid point, am I > right? However, my field split is not constant, like > [u1_x u1_y u1_z p_1 u2_x u2_y u2_z u3_x u3_y u3_z > p_3 u4_x u4_y u4_z] > > Subsequently the fieldsplit is > [u1_x u1_y u1_z u2_x u2_y u2_z u3_x u3_y u3_z u4_x > u4_y u4_z] > [p_1 p_3] > > Then what is the option to set block size 3 for split 0? > > Sorry, I search several forum threads but cannot figure out the options as > you said. > > > > You can still do that. It can be done with options once the decomposition is > working. Its true that these solvers > work better with the block size set. However, if its the P2 Laplacian it does > not really matter since its uncoupled. > > Yes, I agree it's uncoupled with the other field, but the crucial factor > defining the quality of the block preconditioner is the approximate inversion > of individual block. I would merely try block Jacobi first, because it's > quite simple. Nevertheless, fieldsplit implements other nice things, like > Schur complement, etc. > > I think concepts are getting confused here. I was talking about the > interaction of components in one block (the P2 block). You > are talking about interaction between blocks. > > Thanks, > > Matt > > Giang > > > > On Fri, Jan 22, 2016 at 11:15 AM, Matthew Knepley <[email protected]> wrote: > On Fri, Jan 22, 2016 at 3:40 AM, Hoang Giang Bui <[email protected]> wrote: > Hi Matt > I would rather like to set the block size for block P2 too. Why? > > Because in one of my test (for problem involves only [u_x u_y u_z]), the > gmres + Hypre AMG converges in 50 steps with block size 3, whereby it > increases to 140 if block size is 1 (see attached files). > > You can still do that. It can be done with options once the decomposition is > working. Its true that these solvers > work better with the block size set. However, if its the P2 Laplacian it does > not really matter since its uncoupled. > > This gives me the impression that AMG will give better inversion for "P2" > block if I can set its block size to 3. Of course it's still an hypothesis > but worth to try. > > Another question: In one of the Petsc presentation, you said the Hypre AMG > does not scale well, because set up cost amortize the iterations. How is it > quantified? and what is the memory overhead? > > I said the Hypre setup cost is not scalable, but it can be amortized over the > iterations. You can quantify this > just by looking at the PCSetUp time as your increase the number of processes. > I don't think they have a good > model for the memory usage, and if they do, I do not know what it is. > However, generally Hypre takes more > memory than the agglomeration MG like ML or GAMG. > > Thanks, > > Matt > > > Giang > > On Mon, Jan 18, 2016 at 5:25 PM, Jed Brown <[email protected]> wrote: > Hoang Giang Bui <[email protected]> writes: > > > Why P2/P2 is not for co-located discretization? > > Matt typed "P2/P2" when me meant "P2/P1". > > > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener >
