On Thu, Apr 23, 2009 at 2:13 PM, Nguyen, Hung V ERDC-ITL-MS < Hung.V.Nguyen at usace.army.mil> wrote:
> Hello Matt, > > >ILU is incredibly unpredictable. > > I got the same result when running without setting ILU in sub_pc_type. It > seems to me that direct solver is set up as default when solving for > sub_ksp_type(?). Please let me know if it is not correct. > You can see what the default is using -ksp_view > >You have not provided the Jacobi numbers, but the particular nonzero > pattern > of the non-overlapping matrix must be > >much more amenable. Also, this is a really really bad preconditioner for > your system. > > Indeed, the asm preconditioner is not really bad preconditioner for some of > my ill-conditioned systems. In some other SPD linear systems, I have found > that cg with asm preconditioner converges better than others. And, it does > scale well within the size of matrix, see attached file. However, it > doesn't > scale in this case. Here is the solver time for cg/jacobi. The performance > of > cg/asm is better than cg/jacobi in the range from 1 to 4 processors. > > Number Pes Solver Time (secs) #it Solver Time(secs) > cg/asm cg/jacobi > 1 31.317345 544 1999.276566 > 2 263.172225 6959 1188.067975 > 4 734.828840 23233 984.062940 > 8 805.217591 41250 538.102407 > 16 611.813716 49262 308.547316 > 32 345.331928 49792 170.074248 > 64 212.084555 53771 92.398144 > > >I would put my time into figuring out why my system is so ill-conditioned > and try to formulate a good preconditioner, like an approximate system, > etc. > > The linear system is from groundwater flow in a water repellent soil that > can > cause a very ill-conditioned linear system. This is why people develop special purpose discretizations for these problems. Matt > > -Hung > > -----Original Message----- > From: petsc-users-bounces at mcs.anl.gov > [mailto:petsc-users-bounces at mcs.anl.gov] On Behalf Of Matthew Knepley > Sent: Thursday, April 23, 2009 11:12 AM > To: PETSc users list > Subject: Re: Cg/asm doesn't scale > > On Thu, Apr 23, 2009 at 11:07 AM, Nguyen, Hung V ERDC-ITL-MS > <Hung.V.Nguyen at usace.army.mil> wrote: > > > Hello, > > I tried to solver the SPD linear system with using cg/asm > preconditioner and > found that it doesn't scale well, see table below. Note: it does > scale well > with cg/jacobi preconditioner. > > Do you know why it doesn't scale? > > > ILU is incredibly unpredictable. You have not provided the Jacobi numbers, > but the particular nonzero pattern of the non-overlapping matrix must be > much > more amenable. Also, this is a really really bad preconditioner for your > system. I would put my time into figuring out why my system is so > ill-conditioned and try to formulate a good preconditioner, like an > approximate system, etc. > > Matt > > > > Thanks, > > -hung > > Number Pes Solver Time (secs) #it > 1 31.317345 544 > 2 263.172225 6959 > 4 734.828840 23233 > 8 805.217591 41250 > 16 611.813716 49262 > 32 345.331928 49792 > 64 212.084555 53771 > > > --- > 1 : aprun -n 1 ./test_matrix_read -ksp_type cg -pc_type asm > -pc_asm_type > basic -sub_pc_type ilu -sub_ksp_type preonly -ksp_rtol 1.0e-12 > -ksp_max_it > 100000 > Time in PETSc solver: 31.317345 seconds > The number of iteration = 544 > The solution residual error = 1.658653e-08 > 2 norm 7.885361e-07 > infinity norm 6.738382e-09 > 1 norm 2.124207e-04 > > Application 679466 resources: utime 0, stime 0 > ************************ Beginning new run ************************ > > 2 : aprun -n 2 ./test_matrix_read -ksp_type cg -pc_type asm > -pc_asm_type > basic -sub_pc_type ilu -sub_ksp_type preonly -ksp_rtol 1.0e-12 > -ksp_max_it > 100000 > Time in PETSc solver: 263.172225 seconds > The number of iteration = 6959 > The solution residual error = 1.794494e-08 > 2 norm 6.579571e-07 > infinity norm 8.745052e-09 > 1 norm 1.907733e-04 > > -- Here is info about matrix A: > > Computed <structure:nrows> as <178353> > Computed <structure:symmetry> as <0> > Computed <structure:nnzeros> as <3578321> > Computed <structure:max-nnzeros-per-row> as <27> > Computed <structure:min-nnzeros-per-row> as <6> > Computed <structure:left-bandwidth> as <76553> > Computed <structure:right-bandwidth> as <76553> > > > > > > > > > -- > What most experimenters take for granted before they begin their > experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20090423/a5582b4c/attachment.htm>
