Hi, I ran my 3d structured CFD code for a cavity flow with different grid size from 20x20x20 to 80x80x80. Whether fast or not, all they converged and gave me the expected result.
but when I tried with 120x120x120 grid, errors appeared as follows. Since I believe my physical memory is large enough for these test. So I just wonder what really happened with the GMRES solver + hypre PC. I hope you may leave me a hint. Thanks. Zhenyu mpirun -np 8 ./nsproj ../input/sample_120.zzy.dat -px 2 -py 2 -pz 2 -ksp_type gmres -pc_type hypre -ksp_rtol 1.e-2 ....GRID POINT DIMENSIONS W/ GHOST CELLS : 122 X 122 X 122 px= 2, py= 2, pz= 2 ....BEGINNING SIMULATION time: 0, SolvePressure iter= 0, residnorm= 0.000000e+00 time: 20, SolvePressure iter= 2, residnorm= 1.329163e+21 [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Error in external library! [0]PETSC ERROR: Error in HYPRE solver, error code 1! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./nsproj on a arch-linux2-c-opt named amax by zhenyu Sat Aug 24 16:05:05 2013 [0]PETSC ERROR: Libraries linked from /home/zhenyu/petsc-3.4.2/arch-linux2-c-opt/lib [0]PETSC ERROR: Configure run at Sat Aug 24 12:43:12 2013 [1]PETSC ERROR: --------------------- Error Message ------------------------------------ [1]PETSC ERROR: Error in external library! [1]PETSC ERROR: Error in HYPRE solver, error code 1! [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 [1]PETSC ERROR: See docs/changes/index.html for recent updates. [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [1]PETSC ERROR: See docs/index.html for manual pages. [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: ./nsproj on a arch-linux2-c-opt named amax by zhenyu Sat Aug 24 16:05:05 2013 [1]PETSC ERROR: Libraries linked from /home/zhenyu/petsc-3.4.2/arch-linux2-c-opt/lib [1]PETSC ERROR: Configure run at Sat Aug 24 12:43:12 2013 [1]PETSC ERROR: Configure options --with-debugging=0 --with-shared-libraries=1 --with-dynamic-loading=1 --with-x=1 --with-blas-lapack-dir=/usr/lib/lapack --with-valgrind=1 --with-cc=gcc --with-fc=gfortran --with-clanguage=C++ --with-c++-support=1 --with-sieve=1 --with-opt-sieve=1 --download-generator --download-triangle --download-ctetgen --with-ctetgen --download-chaco --download-boost=1 --download-hypre=1 --download-metis --download-parmetis --download-hdf5 --download-openmpi --download-f-blas-lapack --download-superlu_dist [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: PCApply_HYPRE() line 170 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/impls/hypre/hypre.c [1]PETSC ERROR: [2]PETSC ERROR: --------------------- Error Message ------------------------------------ [2]PETSC ERROR: Error in external library! [2]PETSC ERROR: Error in HYPRE solver, error code 1! [2]PETSC ERROR: ------------------------------------------------------------------------ [2]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 [2]PETSC ERROR: See docs/changes/index.html for recent updates. [2]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [2]PETSC ERROR: See docs/index.html for manual pages. [2]PETSC ERROR: ------------------------------------------------------------------------ [2]PETSC ERROR: ./nsproj on a arch-linux2-c-opt named amax by zhenyu Sat Aug 24 16:05:05 2013 [2]PETSC ERROR: Libraries linked from /home/zhenyu/petsc-3.4.2/arch-linux2-c-opt/lib [2]PETSC ERROR: Configure run at Sat Aug 24 12:43:12 2013 [2]PETSC ERROR: Configure options --with-debugging=0 --with-shared-libraries=1 --with-dynamic-loading=1 --with-x=1 --with-blas-lapack-dir=/usr/lib/lapack --with-valgrind=1 --with-cc=gcc --with-fc=gfortran --with-clanguage=C++ --with-c++-support=1 --with-sieve=1 --with-opt-sieve=1 --download-generator --download-triangle --download-ctetgen --with-ctetgen --download-chaco --download-boost=1 --download-hypre=1 --download-metis --download-parmetis --download-hdf5 --download-openmpi --download-f-blas-lapack --download-superlu_dist [2]PETSC ERROR: ------------------------------------------------------------------------ [2]PETSC ERROR: PCApply_HYPRE() line 170 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/impls/hypre/hypre.c [2]PETSC ERROR: PCApply() line 442 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/interface/precon.c [2]PETSC ERROR: KSP_PCApply() line 227 in /home/zhenyu/petsc-3.4.2/include/petsc-private/kspimpl.h [2]PETSC ERROR: KSPInitialResidual() line 64 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itres.c [3]PETSC ERROR: --------------------- Error Message ------------------------------------ [3]PETSC ERROR: Error in external library! [3]PETSC ERROR: Error in HYPRE solver, error code 1! [3]PETSC ERROR: ------------------------------------------------------------------------ [3]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 [3]PETSC ERROR: See docs/changes/index.html for recent updates. [3]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [3]PETSC ERROR: See docs/index.html for manual pages. [3]PETSC ERROR: ------------------------------------------------------------------------ [3]PETSC ERROR: ./nsproj on a arch-linux2-c-opt named amax by zhenyu Sat Aug 24 16:05:05 2013 [3]PETSC ERROR: Libraries linked from /home/zhenyu/petsc-3.4.2/arch-linux2-c-opt/lib [3]PETSC ERROR: Configure run at Sat Aug 24 12:43:12 2013 [3]PETSC ERROR: Configure options --with-debugging=0 --with-shared-libraries=1 --with-dynamic-loading=1 --with-x=1 --with-blas-lapack-dir=/usr/lib/lapack --with-valgrind=1 --with-cc=gcc --with-fc=gfortran --with-clanguage=C++ --with-c++-support=1 --with-sieve=1 --with-opt-sieve=1 --download-generator --download-triangle --download-ctetgen --with-ctetgen --download-chaco --download-boost=1 --download-hypre=1 --download-metis --download-parmetis --download-hdf5 --download-openmpi --download-f-blas-lapack --download-superlu_dist [3]PETSC ERROR: ------------------------------------------------------------------------ [3]PETSC ERROR: PCApply_HYPRE() line 170 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/impls/hypre/hypre.c [3]PETSC ERROR: PCApply() line 442 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/interface/precon.c [3]PETSC ERROR: [4]PETSC ERROR: --------------------- Error Message ------------------------------------ [4]PETSC ERROR: Error in external library! [4]PETSC ERROR: Error in HYPRE solver, error code 1! [4]PETSC ERROR: ------------------------------------------------------------------------ [4]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 [4]PETSC ERROR: See docs/changes/index.html for recent updates. [4]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [4]PETSC ERROR: See docs/index.html for manual pages. [4]PETSC ERROR: ------------------------------------------------------------------------ [4]PETSC ERROR: ./nsproj on a arch-linux2-c-opt named amax by zhenyu Sat Aug 24 16:05:05 2013 [4]PETSC ERROR: Libraries linked from /home/zhenyu/petsc-3.4.2/arch-linux2-c-opt/lib [4]PETSC ERROR: Configure run at Sat Aug 24 12:43:12 2013 [4]PETSC ERROR: Configure options --with-debugging=0 --with-shared-libraries=1 --with-dynamic-loading=1 --with-x=1 --with-blas-lapack-dir=/usr/lib/lapack --with-valgrind=1 --with-cc=gcc --with-fc=gfortran --with-clanguage=C++ --with-c++-support=1 --with-sieve=1 --with-opt-sieve=1 --download-generator --download-triangle --download-ctetgen --with-ctetgen --download-chaco --download-boost=1 --download-hypre=1 --download-metis --download-parmetis --download-hdf5 --download-openmpi --download-f-blas-lapack --download-superlu_dist [4]PETSC ERROR: ------------------------------------------------------------------------ [4]PETSC ERROR: PCApply_HYPRE() line 170 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/impls/hypre/hypre.c [4]PETSC ERROR: PCApply() line 442 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/interface/precon.c [4]PETSC ERROR: KSP_PCApply() line 227 in /home/zhenyu/petsc-3.4.2/include/petsc-private/kspimpl.h [4]PETSC ERROR: KSPInitialResidual() line 64 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itres.c [4]PETSC ERROR: KSPSolve_GMRES() line 239 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/impls/gmres/gmres.c [4]PETSC ERROR: KSPSolve() line 441 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c [4]PETSC ERROR: SolvePressure() line 898 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nssolver_petsc.cc [4]PETSC ERROR: start_sim() line 190 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nsproj_petsc.cc [5]PETSC ERROR: --------------------- Error Message ------------------------------------ [5]PETSC ERROR: Error in external library! [5]PETSC ERROR: Error in HYPRE solver, error code 1! [5]PETSC ERROR: ------------------------------------------------------------------------ [5]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 [5]PETSC ERROR: See docs/changes/index.html for recent updates. [5]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [5]PETSC ERROR: See docs/index.html for manual pages. [5]PETSC ERROR: ------------------------------------------------------------------------ [5]PETSC ERROR: ./nsproj on a arch-linux2-c-opt named amax by zhenyu Sat Aug 24 16:05:05 2013 [5]PETSC ERROR: Libraries linked from /home/zhenyu/petsc-3.4.2/arch-linux2-c-opt/lib [5]PETSC ERROR: Configure run at Sat Aug 24 12:43:12 2013 [5]PETSC ERROR: Configure options --with-debugging=0 --with-shared-libraries=1 --with-dynamic-loading=1 --with-x=1 --with-blas-lapack-dir=/usr/lib/lapack --with-valgrind=1 --with-cc=gcc --with-fc=gfortran --with-clanguage=C++ --with-c++-support=1 --with-sieve=1 --with-opt-sieve=1 --download-generator --download-triangle --download-ctetgen --with-ctetgen --download-chaco --download-boost=1 --download-hypre=1 --download-metis --download-parmetis --download-hdf5 --download-openmpi --download-f-blas-lapack --download-superlu_dist [5]PETSC ERROR: ------------------------------------------------------------------------ [5]PETSC ERROR: PCApply_HYPRE() line 170 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/impls/hypre/hypre.c [5]PETSC ERROR: PCApply() line 442 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/interface/precon.c [5]PETSC ERROR: KSP_PCApply() line 227 in /home/zhenyu/petsc-3.4.2/include/petsc-private/kspimpl.h [5]PETSC ERROR: KSPInitialResidual() line 64 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itres.c [5]PETSC ERROR: KSPSolve_GMRES() line 239 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/impls/gmres/gmres.c [5]PETSC ERROR: KSPSolve() line 441 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c [5]PETSC ERROR: SolvePressure() line 898 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nssolver_petsc.cc [5]PETSC ERROR: start_sim() line 190 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nsproj_petsc.cc [6]PETSC ERROR: --------------------- Error Message ------------------------------------ [6]PETSC ERROR: Error in external library! [6]PETSC ERROR: Error in HYPRE solver, error code 1! [6]PETSC ERROR: ------------------------------------------------------------------------ [6]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 [6]PETSC ERROR: See docs/changes/index.html for recent updates. [6]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [6]PETSC ERROR: See docs/index.html for manual pages. [6]PETSC ERROR: ------------------------------------------------------------------------ [6]PETSC ERROR: ./nsproj on a arch-linux2-c-opt named amax by zhenyu Sat Aug 24 16:05:05 2013 [6]PETSC ERROR: Libraries linked from /home/zhenyu/petsc-3.4.2/arch-linux2-c-opt/lib [6]PETSC ERROR: Configure run at Sat Aug 24 12:43:12 2013 [6]PETSC ERROR: Configure options --with-debugging=0 --with-shared-libraries=1 --with-dynamic-loading=1 --with-x=1 --with-blas-lapack-dir=/usr/lib/lapack --with-valgrind=1 --with-cc=gcc --with-fc=gfortran --with-clanguage=C++ --with-c++-support=1 --with-sieve=1 --with-opt-sieve=1 --download-generator --download-triangle --download-ctetgen --with-ctetgen --download-chaco --download-boost=1 --download-hypre=1 --download-metis --download-parmetis --download-hdf5 --download-openmpi --download-f-blas-lapack --download-superlu_dist [6]PETSC ERROR: ------------------------------------------------------------------------ [6]PETSC ERROR: PCApply_HYPRE() line 170 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/impls/hypre/hypre.c [6]PETSC ERROR: PCApply() line 442 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/interface/precon.c [6]PETSC ERROR: KSP_PCApply() line 227 in /home/zhenyu/petsc-3.4.2/include/petsc-private/kspimpl.h [6]PETSC ERROR: KSPInitialResidual() line 64 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itres.c [6]PETSC ERROR: KSPSolve_GMRES() line 239 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/impls/gmres/gmres.c [6]PETSC ERROR: KSPSolve() line 441 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c [7]PETSC ERROR: --------------------- Error Message ------------------------------------ [7]PETSC ERROR: Error in external library! [7]PETSC ERROR: Error in HYPRE solver, error code 1! [7]PETSC ERROR: ------------------------------------------------------------------------ [7]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 [7]PETSC ERROR: See docs/changes/index.html for recent updates. [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [7]PETSC ERROR: See docs/index.html for manual pages. [7]PETSC ERROR: ------------------------------------------------------------------------ [7]PETSC ERROR: ./nsproj on a arch-linux2-c-opt named amax by zhenyu Sat Aug 24 16:05:05 2013 [7]PETSC ERROR: Libraries linked from /home/zhenyu/petsc-3.4.2/arch-linux2-c-opt/lib [7]PETSC ERROR: Configure run at Sat Aug 24 12:43:12 2013 [7]PETSC ERROR: Configure options --with-debugging=0 --with-shared-libraries=1 --with-dynamic-loading=1 --with-x=1 --with-blas-lapack-dir=/usr/lib/lapack --with-valgrind=1 --with-cc=gcc --with-fc=gfortran --with-clanguage=C++ --with-c++-support=1 --with-sieve=1 --with-opt-sieve=1 --download-generator --download-triangle --download-ctetgen --with-ctetgen --download-chaco --download-boost=1 --download-hypre=1 --download-metis --download-parmetis --download-hdf5 --download-openmpi --download-f-blas-lapack --download-superlu_dist [7]PETSC ERROR: ------------------------------------------------------------------------ [7]PETSC ERROR: PCApply_HYPRE() line 170 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/impls/hypre/hypre.c [7]PETSC ERROR: PCApply() line 442 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/interface/precon.c [7]PETSC ERROR: KSP_PCApply() line 227 in /home/zhenyu/petsc-3.4.2/include/petsc-private/kspimpl.h [7]PETSC ERROR: KSPInitialResidual() line 64 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itres.c [7]PETSC ERROR: KSPSolve_GMRES() line 239 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/impls/gmres/gmres.c [7]PETSC ERROR: KSPSolve() line 441 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c [7]PETSC ERROR: SolvePressure() line 898 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nssolver_petsc.cc [7]PETSC ERROR: start_sim() line 190 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nsproj_petsc.cc [0]PETSC ERROR: Configure options --with-debugging=0 --with-shared-libraries=1 --with-dynamic-loading=1 --with-x=1 --with-blas-lapack-dir=/usr/lib/lapack --with-valgrind=1 --with-cc=gcc --with-fc=gfortran --with-clanguage=C++ --with-c++-support=1 --with-sieve=1 --with-opt-sieve=1 --download-generator --download-triangle --download-ctetgen --with-ctetgen --download-chaco --download-boost=1 --download-hypre=1 --download-metis --download-parmetis --download-hdf5 --download-openmpi --download-f-blas-lapack --download-superlu_dist [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PCApply_HYPRE() line 170 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/impls/hypre/hypre.c [0]PETSC ERROR: PCApply() line 442 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSP_PCApply() line 227 in /home/zhenyu/petsc-3.4.2/include/petsc-private/kspimpl.h [0]PETSC ERROR: KSPInitialResidual() line 64 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itres.c [0]PETSC ERROR: KSPSolve_GMRES() line 239 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPSolve() line 441 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: SolvePressure() line 898 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nssolver_petsc.cc [0]PETSC ERROR: start_sim() line 190 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nsproj_petsc.cc PCApply() line 442 in /home/zhenyu/petsc-3.4.2/src/ksp/pc/interface/precon.c [1]PETSC ERROR: KSP_PCApply() line 227 in /home/zhenyu/petsc-3.4.2/include/petsc-private/kspimpl.h [1]PETSC ERROR: KSPInitialResidual() line 64 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itres.c [1]PETSC ERROR: KSPSolve_GMRES() line 239 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/impls/gmres/gmres.c [1]PETSC ERROR: KSPSolve() line 441 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c [1]PETSC ERROR: SolvePressure() line 898 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nssolver_petsc.cc [1]PETSC ERROR: start_sim() line 190 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nsproj_petsc.cc [2]PETSC ERROR: KSPSolve_GMRES() line 239 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/impls/gmres/gmres.c [2]PETSC ERROR: KSPSolve() line 441 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c [2]PETSC ERROR: SolvePressure() line 898 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nssolver_petsc.cc [2]PETSC ERROR: start_sim() line 190 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nsproj_petsc.cc KSP_PCApply() line 227 in /home/zhenyu/petsc-3.4.2/include/petsc-private/kspimpl.h [3]PETSC ERROR: KSPInitialResidual() line 64 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itres.c [3]PETSC ERROR: KSPSolve_GMRES() line 239 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/impls/gmres/gmres.c [3]PETSC ERROR: KSPSolve() line 441 in /home/zhenyu/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c [3]PETSC ERROR: SolvePressure() line 898 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nssolver_petsc.cc [3]PETSC ERROR: start_sim() line 190 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nsproj_petsc.cc [6]PETSC ERROR: SolvePressure() line 898 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nssolver_petsc.cc [6]PETSC ERROR: start_sim() line 190 in "unknowndirectory/"/home/zhenyu/work/nsproj_0.3_zzy/src/nsproj_petsc.cc -------------------------------------------------------------------------- mpirun noticed that the job aborted, but has no info as to the process that caused that situation. --------------------------------------------------------------------------
