Thank you for your answer, I will then investigate more the topic of scalable preconditioners, as well as try different preconditioners.
> On Tue, May 1, 2012 at 6:20 PM, <tibo at berkeley.edu> wrote: > >> Dear petsc users, >> >> I am solving large systems of non linear PDEs. To do so, the most >> expensive operation is to solve linear systems Ax=b, where A is block >> tridiagonal. To do so, I am using petsc. >> >> A is created using MatCreateMPIAIJ and x and b using VecCreateMPI, and I >> do not modify default parameters of the KSP context for now (ie the >> Krylov >> method is GMRES and the preconditioner method is ILU(0) if I use 1 >> processor - sequential matrix - and Block Jacobi with ILU(0) for each >> sub-block if I use more than 1 processor). >> >> For any number n of processors used, I do get the correct result. >> However, >> it seems that the more processors I have, the more iterations are done >> on >> each linear solve (n = 1 gives about 1-2 iterations per solve, n = 2 >> gives >> 8-12 iterations per solve, n = 4 gives 15-20 iterations per solve). >> >> While I can understand the difference between n = 1 and n = 2, since the >> preconditioning method changes from ILU(0) to Block Jacobi, I don't >> understand why this is the case from n = 2 to 4 for example, since it >> seems to me that the method used to solve Ax=b will be the same >> (although >> the partitioning is different) and so the operations will be the same, >> even though there will be more communication. >> >> My first question is then: Is this normal behavior or am I probably >> wrong >> somewhere ? >> > > Its not the same from 2-4. You have 4 smaller blocks in BJACOBI and less > coupling. > > Also, since the increase in the number of iterations more than offsets the >> decrease in time spent solving the system when n increase, my program >> runs >> slower with an increasing number of processors, which is the opposite of >> what is desired...Would you have suggestions as what I could change to >> correct this ? >> > > Scalable preconditioning is a research topic. However, the next thing to > try is AMG. > > >> I would be happy to provide more details about the >> problem/datastructures >> used if needed, >> > > The best thing to try usually is a literature search for people using > scalable preconditioners > for your kind of physics. > > Matt > > >> thank you for your help, >> >> Tibo >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener >
