Jan: I can reproduce reported behavior using petsc/src/ksp/ksp/examples/tutorials/ex10.c on your mat.dat and rhs.dat.
Using petsc sequential lu with default ordering 'nd', I get ./ex10 -f0 mat.dat -rhs rhs.dat -pc_type lu Number of iterations = 0 Residual norm 0.0220971 Changing to ./ex10 -f0 mat.dat -rhs rhs.dat -pc_type lu -pc_factor_mat_ordering_type natural Number of iterations = 1 Residual norm < 1.e-12 Back to superlu_dist, I get mpiexec -n 3 ./ex10 -f0 /homes/hzhang/tmp/mat.dat -rhs /homes/hzhang/tmp/rhs.dat -pc_type lu -pc_factor_mat_solver_package superlu_dist Number of iterations = 4 Residual norm 25650.8 which uses default ordering (-ksp_view) Row permutation LargeDiag Column permutation METIS_AT_PLUS_A Run it with mpiexec -n 3 ./ex10 -f0 mat.dat -rhs rhs.dat -pc_type lu -pc_factor_mat_solver_package superlu_dist -mat_superlu_dist_rowperm NATURAL -mat_superlu_dist_colperm NATURAL Number of iterations = 1 Residual norm < 1.e-12 i.e., your problem is sensitive to matrix ordering, which I do not know why. I checked condition number of your mat.dat using superlu: ./ex10 -f0 mat.dat -rhs rhs.dat -pc_type lu -pc_factor_mat_solver_package superlu -mat_superlu_conditionnumber Recip. condition number = 1.137938e-03 Number of iterations = 1 Residual norm < 1.e-12 As you see, matrix is well-conditioned. Why is it so sensitive to matrix ordering? Hong Using attached petsc4py code, matrix and right-hand side, SuperLU_dist > returns totally wrong solution for mixed Laplacian: > > $ tar -xzf report.tar.gz > $ python test-solve.py -pc_factor_mat_solver_package mumps > -ksp_final_residual > KSP final norm of residual 3.81865e-15 > $ python test-solve.py -pc_factor_mat_solver_package umfpack > -ksp_final_residual > KSP final norm of residual 3.68546e-14 > $ python test-solve.py -pc_factor_mat_solver_package superlu_dist > -ksp_final_residual > KSP final norm of residual 1827.72 > > Moreover final residual is random when run using mpirun -np 3. Maybe > a memory corruption issue? This is reproducible using PETSc 3.6.2 (and > SuperLU_dist configured by PETSc) and much older, see > http://fenicsproject.org/pipermail/fenics-support/2014-March/000439.html > but has never been reported upstream. > > The code for assembling the matrix and rhs using FEniCS is also > included for the sake of completeness. > > Jan >
