Hi,
Thank you all for your answers.
I have tried your suggestions and here is what I found.
Barry you were right about the first case. But in the second case, I am not
using a Schur fieldsplit but a multiplicative fieldsplit :
-fieldsplit_TEBV_fieldsplit_EBV_pc_type fieldsplit
-fieldsplit_TEBV_fieldsplit_EBV_pc_fieldsplit_type multiplicative
So, there should not be any Schur complement approximation Sp.
When I ran a test with
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_view_pre
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_error_if_not_converged, I got
this error:
0 SNES Function norm 6.368031218939e-02
0 KSP Residual norm 6.368031218939e-02
Linear fieldsplit_ni_ solve converged due to CONVERGED_ATOL iterations 0
Linear fieldsplit_TEBV_fieldsplit_tau_ solve converged due to
CONVERGED_RTOL iterations 1
Linear fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_EP_ solve converged
due to CONVERGED_RTOL iterations 3
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Zero pivot in LU factorization:
https://petsc.org/release/faq/#zeropivot
[0]PETSC ERROR: Zero pivot in row 1658
[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.16.3-751-g2f43bd9bc3 GIT
Date: 2022-01-26 22:34:02 -0600
[0]PETSC ERROR: ./main on a macx named pn2032683.lanl.gov by zjorti Fri Jun 10
16:17:35 2022
[0]PETSC ERROR: Configure options PETSC_ARCH=macx --with-fc=0
--with-mpi-dir=/Users/zjorti/.brew --download-hypre --with-debugging=0
--with-cxx-dialect=C++11 --download-superlu_dist --download-parmetis
--download-metis --download-ptscotch --download-cmake
Then I tried this flag
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_view_pmat
binary:BVmat:binary_matlab and checked the resulting matrix in Matlab.
For Matlab, this is a full rank matrix, and the LU factorization there was
carried out without any issues.
I also outputted the BV block directly from the Jacobian matrix.
Once again, according to Matlab, it is a full rank matrix and it computes the
LU factorization without any problem.
________________________________
From: Barry Smith <[email protected]>
Sent: Friday, June 10, 2022 7:32 AM
To: Jorti, Zakariae
Cc: [email protected]; Tang, Xianzhu
Subject: [EXTERNAL] Re: [petsc-users] Question about SuperLU
It is difficult to tell exactly how the preconditioner is being formed with
the information below it looks like in the
first case: the original B diagonal block and V diagonal block of the matrix
are being factored separately with SuperLU_DIST
second case: the B block is factored with SuperLU_DIST and an explicit
approximation to a Schur complement of the V block (Schur complement on
eliminating the B block) is formed using "Preconditioner for the Schur
complement formed from Sp, an assembled approximation to S, which uses A00's
%sdiagonal's inverse" (this is the printout from a KSPView() for this part of
the preconditioner).
My guess is you have a "Stokes"-like problem where the V block is identically
0 so, of course, the SuperLU_DIST will fail on it. But the approximation of the
Schur complement onto that block is not singular so SuperLU_DIST has no
trouble. If I am wrong and the V block is not identically 0 then it may be
singular (or possibly, but less likely just badly order) so that SuperLU_DIST
encounters a zero pivot.
You can run with -ksp_view_pre to have the KSP print the KSP solver algorithm
details BEFORE the linear solve (hence they would get printed despite your
failed solve). That would be useful to see exactly what your preconditioner is.
You can use -ksp_view_pmat (with appropriate prefix) to display the matrix
that is going to be factored. Thus you can quickly verify what V is.
If you run with -ksp_error_if_not_converged then the solver will stop exactly
when the zero pivot is encountered; this would include some information from
SuperLU_DIST which might include the row number etc.
Notes on PETSc improvements needed.
1) The man page for KSPCheckSolve() is terribly misleading
2) It would be nice to have a view that displayed the nested fieldsplit
preconditioners more clearly
On Jun 9, 2022, at 5:19 PM, Jorti, Zakariae via petsc-users
<[email protected]<mailto:[email protected]>> wrote:
Hi,
I am solving non-linear problem that has 5 unknowns {ni, T, E, B, V}, and for
the preconditioning part, I am using a FieldSplit preconditioner. At the last
fieldsplit/level, we are left with a {B,V} block that tried to precondition in
2 different ways:
a) SuperLU:
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_type preonly
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_type lu
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_factor_mat_solver_type
superlu_dist
b) a Schur-based fieldsplit preconditioner that uses SuperLU for both V and B
blocks:
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_type gmres
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_type fieldsplit
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_fieldsplit_type schur
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_fieldsplit_schur_precondition
selfp -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_B_ksp_type
preonly -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_B_pc_type lu
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_B_pc_factor_mat_solver_type
superlu_dist
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_V_ksp_type preonly
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_V_pc_type lu
-fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_V_pc_factor_mat_solver_type
superlu_dist
Option a) yields the following error:
" Linear fieldsplit_ni_ solve converged due to CONVERGED_ATOL iterations 0
Linear fieldsplit_TEBV_fieldsplit_tau_ solve converged due to
CONVERGED_RTOL iterations 1
Linear fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_EP_ solve converged
due to CONVERGED_RTOL iterations 5
Linear fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ solve did not
converge due to DIVERGED_PC_FAILED iterations 0
PC failed due to FACTOR_NUMERIC_ZEROPIVOT "
whereas options b) seems to be working well.
Is it possible that the SuperLU on the {V,B} block uses a reordering that
introduces a zero pivot or could there be another explanation for this error?
Many thanks.
Best,
Zakariae
