Re: [Libmesh-users] dof numbering

2015-02-25 Thread Manav Bhatia
I finally got a chance to get back to this problem. I played around with the different orderings available in petsc, -pc_factor_mat_ordering_type, and Quotient Minimum Degree seemed to do the trick. I am able to solve the system with about 7.5 GB of memory footprint in about 4.5 minutes, where

Re: [Libmesh-users] dof numbering

2015-02-06 Thread Manav Bhatia
Ok. Just so that I understand your point: you are saying that because petsc (and other advanced direct solvers) will calculate a new ordering for the matrix factorization, it does not help to calculate it in advance. However, for a more basic direct solver, which does not calculate the reorderi

Re: [Libmesh-users] dof numbering

2015-02-06 Thread Manav Bhatia
> On Feb 6, 2015, at 9:40 PM, Jed Brown wrote: > > Manav Bhatia writes: >> I am not sure if I understand your point. Are you saying that using a >> fill-reducing ordering for the dofs would not change the performance >> of a direct solver? > > Correct. The solver is already computing an order

Re: [Libmesh-users] dof numbering

2015-02-06 Thread Manav Bhatia
> On Feb 6, 2015, at 9:29 PM, Jed Brown wrote: > > Manav Bhatia writes: > >> I am in agreement with that. However my situation is more rudimentary than >> that. >> >> If I run the same problem in ABAQUS, it uses a multifraontal direct solver >> and gets done in two minutes. >> >> If I u

Re: [Libmesh-users] dof numbering

2015-02-06 Thread Manav Bhatia
I am in agreement with that. However my situation is more rudimentary than that. If I run the same problem in ABAQUS, it uses a multifraontal direct solver and gets done in two minutes. If I use the -ksp_type preonly -pc_type lu on my code built on libmesh and petsc, it takes ~16GB in storag

Re: [Libmesh-users] dof numbering

2015-02-06 Thread David Knezevic
I'm not sure why MUMPS wouldn't be available on your machine, it works fine for me (I just use the --download-mumps option when configuring PETSc). But you seem to be suggesting that MUMPS has a builtin fill-reduction > algorithm. Correct? > Yes, have a look at ICNTL(7) in the MUMPS manual. It p

Re: [Libmesh-users] dof numbering

2015-02-06 Thread David Knezevic
It sounds like you're right at the limit of what your computer can handle with a direct solver. Which direct solver are you using? I generally use MUMPS or SuperLU instead of PETSc's built in direct solver (e.g. -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package superlu_dist). I think Supe

Re: [Libmesh-users] dof numbering

2015-02-06 Thread Manav Bhatia
I tried the MUMPS option through petsc using this set of options: -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -mat_mumps_icntl_ 22 1 -mat_mumps_icntl_ 23 1 However, MUMPS does not seems to be running on my machine with the PETSC build. I build 3.5.3 today with the do