Hi Hong, Looks like this is my fault since I'm using -ksp_type preonly -pc_type gamg. If I use the default ksp (GMRES) then everything works fine on a smaller problem.
Just to confirm, -ksp_type preonly is to be used only with direct-solve preconditioners like LU,Cholesky, right ? Thank You, Sajid Ali Applied Physics Northwestern University