Perhaps you have done enough memory preallocation for the multiprocessors and it is going very--very slowly with memory allocations?
Barry On Oct 12, 2009, at 8:03 AM, Andreas Grassl wrote: > Hello, > > I'm trying to work with MUMPS-Cholesky as direct solver and CG as > iterative > solver and have problems with SBAIJ-Matrices. > > I have a MatSetValues-line not regarding the symmetry but force it > by the > command line option -mat_ignore_lower_triangular. > > The problems show up at "bigger" problems (~450000 DOF) with "more" > processes (1 > and 2 work fine, 4 gives a hangup) and/or trying to save the matrix. > The machine > has 4 cores and 12 GB memory. NNZ per row is ~80. > > Attached you find the output of the program run with -info. > > Any hints where to search? > > Cheers, > > ando > > -- > /"\ Grassl Andreas > \ / ASCII Ribbon Campaign Uni Innsbruck Institut f. Mathematik > X against HTML email Technikerstr. 13 Zi 709 > / \ +43 (0)512 507 6091 > same call as 2procinfo without -ksp_info_binary > > mpiexec -np 2 externalsolver > [0] PetscInitialize(): PETSc successfully started: number of > processors = 2 > [1] PetscInitialize(): PETSc successfully started: number of > processors = 2 > [1] PetscInitialize(): Running on machine: mat1.uibk.ac.at > [0] PetscInitialize(): Running on machine: mat1.uibk.ac.at > [0] PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [0] PetscCommDuplicate(): returning tag 2147483647 > [1] PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [1] PetscCommDuplicate(): returning tag 2147483647 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483646 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483646 > [0] MatSetUpPreallocation(): Warning not preallocating matrix storage > [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [0] PetscCommDuplicate(): returning tag 2147483647 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [1] PetscCommDuplicate(): returning tag 2147483647 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483646 > [0] PetscCommDuplicate(): returning tag 2147483646 > [0] MatSetOption_MPISBAIJ(): Option SYMMETRIC ignored > [0] MatStashScatterBegin_Private(): No of messages: 1 > [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 30544064 > [0] MatStashScatterBegin_Private(): No of messages: 0 > [0] MatAssemblyBegin_MPISBAIJ(): Stash has 3818007 entries,uses 8 > mallocs. > [0] MatAssemblyBegin_MPISBAIJ(): Block-Stash has 3818007 entries, > uses 8 mallocs. > [0] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 221545 X 221545, block > size 1; storage space: 16676676 unneeded, 5477824 used > [0] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues > is 0 > [0] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 62 > [1] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 225367 X 225367, block > size 1; storage space: 16793278 unneeded, 5743422 used > [1] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues > is 0 > [1] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 65 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483645 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483644 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483643 > [1] PetscCommDuplicate(): returning tag 2147483639 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483645 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483644 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483643 > [0] PetscCommDuplicate(): returning tag 2147483639 > [0] PetscCommDuplicate(): returning tag 2147483634 > [1] PetscCommDuplicate(): returning tag 2147483634 > [1] PetscCommDuplicate(): returning tag 2147483630 > [1] PetscCommDuplicate(): returning tag 2147483625 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483642 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483641 > [1] PetscCommDuplicate(): returning tag 2147483620 > [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter > [0] VecScatterCreate(): General case: MPI to Seq > [0] PetscCommDuplicate(): returning tag 2147483630 > [0] PetscCommDuplicate(): returning tag 2147483625 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483642 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483641 > [0] PetscCommDuplicate(): returning tag 2147483620 > [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter > [0] VecScatterCreate(): General case: MPI to MPI > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483640 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483639 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDupli[1] PetscCommDuplicate(): Using internal PETSc > communicator 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483640 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483639 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483638 > [1] MatAssemblyEnd_SeqBAIJ(): Matrix size: 225367 X 0, block size 1; > storage space: 22536700 unneeded, 0 used > [1] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues > is 0 > [1] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 0 > [1] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 225367)/ > (num_localrows 225367) > 0.6. Use CompressedRow routines. > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483614 > [1] PetscCommDuplicate(): returning tag 2147483613 > [1]before KSPsetup > [1] PetscCommDuplicate(): Using cate(): returning tag 2147483638 > [0] MatAssemblyEnd_SeqBAIJ(): Matrix size: 221545 X 6275, block size > 1; storage space: 22060506 unneeded, 93994 used > [0] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues > is 0 > [0] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 32 > [0] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 216534)/ > (num_localrows 221545) > 0.6. Use CompressedRow routines. > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483614 > [0] PetscCommDuplicate(): returning tag 2147483613 > [0]before KSPsetup > [0] PCSetUp(): Setting up new PC > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483637 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483636 > [0] PetscCommDuplicate(): returning tag 2147483612 > [0] PetscCommDuplicate(): Using internainternal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483637 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483636 > [1] PetscCommDuplicate(): returning tag 2147483612 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483635 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483634 > l PETSc communicator 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483635 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483634 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483633 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483632 > [1] PetscCommDuplicate(): returning tag 2147483607 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483633 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483632 > [0] PetscCommDuplicate(): returning tag 2147483607 > [0] PetscCommDuplicate(): returning tag 2147483602 > [0] VecScatterCreate(): Special case: processor zero gets entire > parallel vector, rest get none > [1] PetscCommDuplicate(): returning tag 2147483602 > > DMUMPS 4.8.4 > L D L^T Solver for symmetric positive definite matrices > Type of parallelism: Working host > > ****** ANALYSIS STEP ******** > > Density: NBdense, Average, Median = 0 49 50 > Ordering based on METIS > ** Peak of sequential stack size (number of real entries) : > 16763905. > A root of estimated size 2965 has been selected for > Scalapack. > > Leaving analysis phase with ... > INFOG(1) = 0 > INFOG(2) = 0 > -- (20) Number of entries in factors (estim.) = 120657071 > -- (3) Storage of factors (REAL, estimated) = 137364268 > -- (4) Storage of factors (INT , estimated) = 6138804 > -- (5) Maximum frontal size (estimated) = 3705 > -- (6) Number of nodes in the tree = 21863 > -- (7) Ordering option effectively used = 5 > ICNTL(6) Maximum transversal option = 0 > ICNTL(7) Pivot order option = 7 > Percentage of memory relaxation (effective) = 200 > Number of level 2 nodes = 1 > Number of split nodes = 0 > RINFO(1) Operations during elimination (estim) = 1.114D+11 > Distributed matrix entry format (ICNTL(18)) = 3 > ** Rank of proc needing largest memory in IC facto : 0 > ** Estimated corresponding MBYTES for IC facto : 2085 > ** Estimated avg. MBYTES per work. proc at facto (IC) : 2044 > ** TOTAL space in MBYTES for IC factorization : 4088 > ** Rank of proc needing largest memory for OOC facto : 1 > ** Estimated corresponding MBYTES for OOC facto : 644 > ** Estimated avg. MBYTES per work. proc at facto (OOC) : 630 > ** TOTAL space in MBYTES for OOC factorization : 1261 > > ****** FACTORIZATION STEP ******** > > > GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ... > NUMBER OF WORKING PROCESSES = 2 > REAL SPACE FOR FACTORS = 137364268 > INTEGER SPACE FOR FACTORS = 6138804 > MAXIMUM FRONTAL SIZE (ESTIMATED) = 3705 > NUMBER OF NODES IN THE TREE = 21863 > Maximum effective relaxed size of S = 239060000 > Average effective relaxed size of S = 234725641 > > REDISTRIB: TOTAL DATA LOCAL/SENT = 9086109 2238491 > GLOBAL TIME FOR MATRIX DISTRIBUTION = 0.7322 > ** Memory relaxation parameter ( ICNTL(14) ) : 200 > ** Rank of processor needing largest memory in facto : 0 > ** Space in MBYTES used by this processor for facto : 2085 > ** Avg. Space in MBYTES per working proc during facto : 2044 > > ELAPSED TIME FOR FACTORIZATION = 28.1724 > Maximum effective space used in S (KEEP(67) = 77516871 > Average effective space used in S (KEEP(67) = 75395654 > ** EFF Min: Rank of processor needing largest memory : 0 > ** EFF Min: Space in MBYTES used by this processor : 741 > ** EFF Min: Avg. Space in MBYTES per working proc : 720 > > GLOBAL STATISTICS > RINFOG(2) OPERATIONS DURING NODE ASSEMBLY = 2.565D+08 > ------(3) OPERATIONS DURING NODE ELIMINATION = 1.114D+11 > INFOG (9) REAL SPACE FOR FACTORS = 137364268 > INFOG(10) INTEGER SPACE FOR FACTORS = 6138809 > INFOG(11) MAXIMUM FRONT SIZE = 3705 > INFOG(29) NUMBER OF ENTRIES IN FACTORS = 116259976 > INFOG(12) NB OF NEGATIVE PIVOTS = 0 > INFOG(14) NUMBER OF MEMORY COMPRESS = 0 > [0]after KSPsetup > [0]RHS setup > [1]after KSPsetup > [1]RHS setup > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483601 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483601 > [0] PetscCommDuplicate(): returning tag 2147483596 > [1] PetscCommDuplicate(): returning tag 2147483596 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483591 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483591 > [0] VecAssemblyBegin_MPI(): Stash has 225367 entries, uses 12 mallocs. > [0] VecAssemblyBegin_MPI(): Block-Stash has 0 entries, uses 0 mallocs. > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483631 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483631 > > > ****** SOLVE & CHECK STEP ******** > > > STATISTICS PRIOR SOLVE PHASE ........... > NUMBER OF RIGHT-HAND-SIDES = 1 > BLOCKING FACTOR FOR MULTIPLE RHS = 1 > ICNTL (9) = 1 > --- (10) = 0 > --- (11) = 0 > --- (20) = 0 > --- (21) = 1 > ** Rank of processor needing largest memory in solve : 0 > ** Space in MBYTES used by this processor for solve : 2013 > ** Avg. Space in MBYTES per working proc during solve : 1975 > > > LEAVING SOLVER WITH: INFOG(1) ............ = 0 > INFOG(2) ............ = 0 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483630 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483629 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483630 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483629 > [0] PetscCommDuplicate(): returning tag 2147483590 > [1] PetscCommDuplicate(): returning tag 2147483590 > [1] VecScatterCreate(): General case: Seq to MPI > [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter > [0] VecScatterCreate(): General case: Seq to MPI > [0] PetscCommDuplicate(): returning tag 2147483586 > KSP Object: > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: cholesky > Cholesky: out-of-place factorization > matrix ordering: natural > Cholesky: factor fill ratio needed 0 > Factored matrix follows > Matrix Object: > type=mpisbaij, rows=446912, cols=446912 > package used to perform factorization: mumps > [1] PetscCommDuplicate(): returning tag 2147483586 > [1] PetscCommDuplicate(): returning tag 2147483585 > [1] PetscCommDuplicate(): returning tag 2147483584 > [1] PetscCommDuplicate(): returning tag 2147483583 > [1] PetscCommDuplicate(): returning tag 2147483582 > [1] PetscCommDuplicate(): returning tag 2147483581 > [1] PetscCommDuplicate(): returning tag 2147483580 > total: nonzeros=0, allocated nonzeros=893824 > MUMPS run parameters: > SYM (matrix type): 1 > PAR (host participation): 1 > ICNTL(1) (output for error): 6 > ICNTL(2) (output of diagnostic msg):0 > ICNTL(3) (output for global info): 6 > ICNTL(4) (level of printing): -1 > ICNTL(5) (input mat struct): 0 > ICNTL(6) (matrix prescaling): 0 > ICNTL(7) (matrix ordering): 7 > ICNTL(8) (scalling strategy): 77 > ICNTL(9) (A/A^T x=b is solved): 1 > ICNTL(10) (max num of refinements): 0 > ICNTL(11) (error analysis): 0 > ICNTL(12) (efficiency control): 1 > ICNTL(13) (efficiency control): 0 > ICNTL(14) (percentage of estimated workspace increase): > 200 > ICNTL(18)[1] PetscCommDuplicate(): returning tag > 2147483579 > (input mat struct): 3 > ICNTL(19) (Shur complement info): 0 > ICNTL(20) (rhs sparse pattern): 0 > ICNTL(21) (solution struct): 1 > ICNTL(22) (in-core/out-of-core facility): 0 > ICNTL(23) (max size of memory can be allocated locally):0 > ICNTL(24) (detection of null pivot rows): 0 > ICNTL(25) (computation of a null space basis): 0 > ICNTL(26) (Schur options for rhs or solution): 0 > ICNTL(27) (experimental parameter): > -8 > CNTL(1) (relative pivoting threshold): 0 > CNTL(2) (stopping criterion of refinement): 1.49012e-08 > CNTL(3) (absolute pivoting threshold): 0 > CNTL(4) (value of static pivoting): -1 > CNTL(5) (fixation for null pivots): 0 > RINFO(1) (local estimated flops for the elimination after > analysis): > [0] 5.16132e+10 > [0] PetscCommDuplicate(): returning tag 2147483585 > [1] 5.97957e+10 > RINFO(2) (local estimated flops for the assembly after > factorization): > [0] 1.33366e+08 > [0] PetscCommDuplicate(): returning tag 2147483584 > [1] 1.23105e+08 > RINFO(3) (local estimated flops for the elimination after > factorization): > [0] 5.16124e+10 > [0] PetscCommDuplicate(): returning tag 2147483583 > [1] 5.9795e+10 > INFO(15) (estimated size of (in MB) MUMPS internal data for > running numerical factorization): > [0] 2085 > [0] PetscCommDuplicate(): returning tag 2147483582 > [1] 2003 > INFO(16) (size of (in MB) MUMPS internal data used during > numerical factorization): > [0] 2085 > [0] PetscCommDuplicate(): returning tag 2147483581 > [1] 2003 > INFO(23) (num of pivots eliminated on this processor after > factorization): > [0] 236610 > [0] PetscCommDuplicate(): returning tag 2147483580 > [1] 210302 > RINFOG(1) (global estimated flops for the elimination > after analysis): 1.11409e+11 > RINFOG(2) (global estimated flops for the assembly > after factorization): 2.56471e+08 > RINFOG(3) (global estimated flops for the elimination > after factorization): 1.11407e+11 > INFOG(3) (estimated real workspace for factors on all > processors after analysis): 137364268 > INFOG(4) (estimated integer workspace for factors on > all processors after analysis): 6138804 > INFOG(5) (estimated maximum front size in the complete > tree): 3705 > INFOG(6) (number of nodes in the complete tree): 21863 > INFOG(7) (ordering option effectively uese after > analysis): 5 > INFOG(8) (structural symmetry in percent of the > permuted matrix after analysis): 100 > INFOG(9) (total real/complex workspace to store the > matrix factors after factorization): 137364268 > INFOG(10) (total integer space store the matrix factors > after factorization): 6138809 > INFOG(11) (order of largest frontal matrix after > factorization): 3705 > INFOG(12) (number of off-diagonal pivots): 0 > INFOG(13) (number of delayed pivots after > factorization): 0 > INFOG(14) (number of memory compress after > factorization): 0 > INFOG(15) (number of steps of iterative refinement > after solution): 0 > INFOG(16) (estimated size (in MB) of all MUMPS internal > data for factorization after analysis: value on the most memory > consuming processor): 2085 > INFOG(17) (estimated size of all MUMPS internal data > for factorization after analysis: sum over all processors): 4088 > INFOG(18) (size of all MUMPS internal data allocated > during factorization: value on the most memory consuming processor): > 2085 > INFOG(19) (size of all MUMPS internal data allocated > during factorization: sum over all processors): 4088 > INFOG(20) (estimated number of entries in the factors): > 120657071 > INFOG(21) (size in MB of memory effectively used during > factorization - value on the most memory consuming processor): 741 > INFOG(22) (size in MB of memory effectively used during > factorization - sum over all processors): 1441 > INFOG(23) (after analysis: value of ICNTL(6) > effectively used): 0 > INFOG(24) (after analysis: value of ICNTL(12) > effectively used): 1 > INFOG(25) (after factorization: number of pivots > modified by static pivoting): 0 > linear system matrix = precond matrix: > Matrix Object: > type=mpisbaij, rows=446912, cols=446912 > total: nonzeros=11315240, allocated nonzeros=89382400 > block size is 1 > [0] PetscCommDuplicate(): returning tag 2147483579 > [1] PetscCommDuplicate(): returning tag 2147483578 > [1] PetscCommDuplicate(): returning tag 2147483577 > [1] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483578 > [0] PetscCommDuplicate(): returning tag 2147483577 > [0] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784 > [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm 1140850689 > [0] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374783 > [0] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374783 > [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483576 > [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm 1140850689 > [1] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374783 > [1] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374783 > [0] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784 > [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm 1140850688 > [0] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374784 > [0] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374784 > [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm -2080374784 > [0] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784 > ************************************************************************************************************************ > *** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript - > r -fCourier9' to print this document *** > ************************************************************************************************************************ > > ---------------------------------------------- PETSc Performance > Summary: ---------------------------------------------- > > [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483576 > [1] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784 > [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm 1140850688 > [1] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374784 > [1] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374784 > [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm -2080374784 > [1] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784 > externalsolver on a linux32-i named mat1.uibk.ac.at with 2 > processors, by csae1801 Mon Oct 12 12:08:38 2009 > Using Petsc Release Version 3.0.0, Patch 8, Fri Aug 21 14:02:12 CDT > 2009 > > Max Max/Min Avg Total > Time (sec): 3.989e+01 1.00000 3.989e+01 > Objects: 3.600e+01 1.00000 3.600e+01 > Flops: 0.000e+00 0.00000 0.000e+00 0.000e+00 > Flops/sec: 0.000e+00 0.00000 0.000e+00 0.000e+00 > MPI Messages: 2.550e+01 1.08511 2.450e+01 4.900e+01 > MPI Message Lengths: 6.596e+07 1.00053 2.692e+06 1.319e+08 > MPI Reductions: 5.800e+01 1.00000 > > Flop counting convention: 1 flop = 1 real number operation of type > (multiply/divide/add/subtract) > e.g., VecAXPY() for real vectors of > length N --> 2N flops > and VecAXPY() for complex vectors of > length N --> 8N flops > > Summary of Stages: ----- Time ------ ----- Flops ----- --- > Messages --- -- Message Lengths -- -- Reductions -- > Avg %Total Avg %Total counts > %Total Avg %Total counts %Total > 0: Main Stage: 2.8402e-01 0.7% 0.0000e+00 0.0% 1.000e > +00 2.0% 8.163e-02 0.0% 0.000e+00 0.0% > 1: decomposition: 4.7451e-01 1.2% 0.0000e+00 0.0% 0.000e > +00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% > 2: Matrix Create: 7.2766e-01 1.8% 0.0000e+00 0.0% 0.000e > +00 0.0% 0.000e+00 0.0% 2.000e+00 3.4% > 3: Matrix fill: 2.5720e+00 6.4% 0.0000e+00 0.0% 1.600e+01 > 32.7% 2.486e+06 92.4% 1.500e+01 25.9% > 4: Solver setup: 3.2841e+01 82.3% 0.0000e+00 0.0% 0.000e > +00 0.0% 0.000e+00 0.0% 8.000e+00 13.8% > 5: RHS setup: 3.2125e-02 0.1% 0.0000e+00 0.0% 4.000e > +00 8.2% 1.094e+05 4.1% 4.000e+00 6.9% > 6: Solve: 4.9751e-01 1.2% 0.0000e+00 0.0% 2.500e+01 > 51.0% 5.944e+04 2.2% 6.000e+00 10.3% > 7: Postproc: 2.4653e+00 6.2% 0.0000e+00 0.0% 3.000e > +00 6.1% 3.679e+04 1.4% 0.000e+00 0.0% > > ------------------------------------------------------------------------------------------------------------------------ > See the 'Profiling' chapter of the users' manual for details on > interpreting output. > Phase summary info: > Count: number of times phase was executed > Time and Flops: Max - maximum over all processors > Ratio - ratio of maximum to minimum over all > processors > Mess: number of messages sent > Avg. len: average message length > Reduct: number of global reductions > Global: entire computation > Stage: stages of a computation. Set stages with > PetscLogStagePush() and PetscLogStagePop(). > %T - percent time in this phase %F - percent flops in > this phase > %M - percent messages in this phase %L - percent message > lengths in this phase > %R - percent reductions in this phase > Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max > time over all processors) > ------------------------------------------------------------------------------------------------------------------------ > Event Count Time (sec) > Flops --- Global --- --- Stage --- > Total > Max Ratio Max Ratio Max Ratio Mess Avg > len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > ------------------------------------------------------------------------------------------------------------------------ > > --- Event Stage 0: Main Stage > > PetscBarrier 1 1.0 3.1269e-0328.4 0.00e+00 0.0 0.0e+00 0.0e > +00 0.0e+00 0 0 0 0 0 1 0 0 0 0 0 > > --- Event Stage 1: decomposition > > > --- Event Stage 2: Matrix Create > > > --- Event Stage 3: Matrix fill > > MatAssemblyBegin 1 1.0 3.3235e-01 1.9 0.00e+00 0.0 6.0e+00 2.0e > +07 3.0e+00 1 0 12 92 5 10 0 38100 20 0 > MatAssemblyEnd 1 1.0 1.0536e+00 1.0 0.00e+00 0.0 1.0e+01 1.8e > +04 1.2e+01 3 0 20 0 21 41 0 62 0 80 0 > > --- Event Stage 4: Solver setup > > MatCholFctrSym 1 1.0 4.0531e-06 2.1 0.00e+00 0.0 0.0e+00 0.0e > +00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatCholFctrNum 1 1.0 3.2830e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e > +00 5.0e+00 82 0 0 0 9 100 0 0 0 62 0 > KSPSetup 1 1.0 6.9141e-06 1.4 0.00e+00 0.0 0.0e+00 0.0e > +00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > PCSetUp 1 1.0 3.2841e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e > +00 8.0e+00 82 0 0 0 14 100 0 0 0100 0 > > --- Event Stage 5: RHS setup > > VecAssemblyBegin 1 1.0 6.2160e-03 1.6 0.00e+00 0.0 4.0e+00 1.3e > +06 3.0e+00 0 0 8 4 5 16 0100100 75 0 > VecAssemblyEnd 1 1.0 8.4589e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e > +00 0.0e+00 0 0 0 0 0 26 0 0 0 0 0 > > --- Event Stage 6: Solve > > MatSolve 1 1.0 4.9525e-01 1.0 0.00e+00 0.0 6.0e+00 4.8e > +05 4.0e+00 1 0 12 2 7 100 0 24 98 67 0 > MatView 2 1.0 1.4269e-03 1.0 0.00e+00 0.0 1.8e+01 2.7e > +03 2.0e+00 0 0 37 0 3 0 0 72 2 33 0 > VecSet 1 1.0 8.1515e-04 1.1 0.00e+00 0.0 0.0e+00 0.0e > +00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecScatterBegin 2 1.0 6.7840e-03 1.3 0.00e+00 0.0 2.0e+00 1.3e > +06 0.0e+00 0 0 4 2 0 1 0 8 86 0 0 > VecScatterEnd 1 1.0 2.1610e-03 1.7 0.00e+00 0.0 0.0e+00 0.0e > +00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > KSPSolve 1 1.0 4.9603e-01 1.0 0.00e+00 0.0 6.0e+00 4.8e > +05 4.0e+00 1 0 12 2 7 100 0 24 98 67 0 > PCApply 1 1.0 4.9526e-01 1.0 0.00e+00 0.0 6.0e+00 4.8e > +05 4.0e+00 1 0 12 2 7 100 0 24 98 67 0 > > --- Event Stage 7: Postproc > > VecView 1 1.0 2.1809e+00 2.0 0.00e+00 0.0 2.0e+00 9.0e > +05 0.0e+00 4 0 4 1 0 66 0 67100 0 0 > ------------------------------------------------------------------------------------------------------------------------ > > Memory usage is given in bytes: > > Object Type Creations Destructions Memory Descendants' > Mem. > > --- Event Stage 0: Main Stage > > IS L to G Mapping 0 1 889756 0 > Matrix 0 3 547690752 0 > Vec 0 6 3702888 0 > Vec Scatter 0 2 1736 0 > Viewer 0 1 544 0 > > --- Event Stage 1: decomposition > > > --- Event Stage 2: Matrix Create > > IS L to G Mapping 1 0 0 0 > Matrix 3 0 0 0 > > --- Event Stage 3: Matrix fill > > Index Set 4 4 152680 0 > Vec 7 1 1304 0 > Vec Scatter 2 0 0 0 > > --- Event Stage 4: Solver setup > > Index Set 3 2 1008 0 > Matrix 3 0 0 0 > Vec 2 1 1773664 0 > Vec Scatter 1 0 0 0 > Krylov Solver 1 0 0 0 > Preconditioner 1 0 0 0 > > --- Event Stage 5: RHS setup > > Vec 2 0 0 0 > Viewer 1 0 0 0 > > --- Event Stage 6: Solve > > Index Set 2 2 947456 0 > Vec 1 0 0 0 > Vec Scatter 1 0 0 0 > Viewer 1 0 0 0 > > --- Event Stage 7: Postproc > > Index Set 0 1 504 0 > Matrix 0 3 8870136 0 > Vec 0 4 7125104 0 > Vec Scatter 0 2 1320 0 > Krylov Solver 0 1 832 0 > Preconditioner 0 1 728 0 > Viewer 0 1 544 0 > = > = > = > = > = > = > = > = > = > = > = > = > = > = > = > =================================================================[1] > PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [1] PetscCommDuplicate(): returning tag 2147483647 > [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm 1140850688 > [1] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374784 > [1] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374784 > [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm -2080374784 > ======================================== > Average time to get PetscTime(): 1.3113e-06 > Average time for MPI_Barrier(): 0.000144005 > [0] PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [0] PetscCommDuplicate(): returning tag 2147483647 > Average time for zero size MPI_Send(): 6.54459e-05 > [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm 1140850688 > [0] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374784 > [0] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374784 > [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user > MPI_Comm -2080374784 > #PETSc Option Table entries: > -dump_data false > -info > -ksp_compute_singularvalues > -ksp_rtol 1e-6 > -ksp_type preonly > -ksp_view > -log_summary > -mat_ignore_lower_triangular > -mat_mumps_icntl_14 200 > -mat_mumps_sym 1 > -mat_type sbaij > -monitorname errormonitor.dat > -nz 100 > -options_left > -pc_factor_mat_solver_package mumps > -pc_type cholesky > -preload false > -verb false > #End o PETSc Option Table entries > Compiled without FORTRAN kernels > Compiled with full precision matrices (default) > sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 > sizeof(PetscScalar) 8 > Configure run at: Wed Oct 7 18:16:40 2009 > Configure options: --with-scalar-type=real --with-debugging=0 --with- > precision=double --with-cc=icc --with-fc=ifort --with-cxx=icpc -- > with-shared=0 --with-mpi=1 --with-external-packages-dir=/home/lux/ > csae1801/petsc/externalpackages --download-mpich=ifneeded --with- > scalapack=1 --download-scalapack=ifneeded --download-f-blas- > lapack=ifneeded --with-blacs=1 --download-blacs=ifneeded --with- > parmetis=1 --download-parmetis=ifneeded --with-mumps=1 --download- > mumps=ifneeded --with-hypre=1 --with-hypre-dir=/home/lux/csae1801/ > petsc/externalpackages/hypre-2.4.0b/src/hypre --with-spooles=1 -- > download-spooles=ifneeded --with-superlu_dist=1 --download- > superlu_dist=ifneeded PETSC_ARCH=linux32-intel-c-prod > ----------------------------------------- > Libraries compiled on Mi 7. Okt 18:17:41 CEST 2009 on mat1.uibk.ac.at > Machine characteristics: Linux mat1.uibk.ac.at 2.6.18-53.el5xen #1 > SMP Wed Oct 10 16:48:44 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux > Using PETSc directory: /home/lux/csae1801/petsc/petsc-3.0.0-p8 > Using PETSc arch: linux32-intel-c-prod > ----------------------------------------- > Using C compiler: /home/lux/csae1801/petsc/petsc-3.0.0-p8/linux32- > intel-c-prod/bin/mpicc -O > Using Fortran compiler: /home/lux/csae1801/petsc/petsc-3.0.0-p8/ > linux32-intel-c-prod/bin/mpif90 -O > ----------------------------------------- > Using include paths: -I/home/lux/csae1801/petsc/petsc-3.0.0-p8/ > linux32-intel-c-prod/include -I/home/lux/csae1801/petsc/petsc-3.0.0- > p8/include -I/home/lux/csae1801/petsc/petsc-3.0.0-p8/linux32-intel-c- > prod/include -I/home/lux/csae1801/petsc/externalpackages/ > hypre-2.4.0b/src/hypre/include > ------------------------------------------ > Using C linker: /home/lux/csae1801/petsc/petsc-3.0.0-p8/linux32- > intel-c-prod/bin/mpicc -O > Using Fortran linker: /home/lux/csae1801/petsc/petsc-3.0.0-p8/ > linux32-intel-c-prod/bin/mpif90 -O > Using libraries: -Wl,-rpath,/home/lux/csae1801/petsc/petsc-3.0.0-p8/ > linux32-intel-c-prod/lib -L/home/lux/csae1801/petsc/petsc-3.0.0-p8/ > linux32-intel-c-prod/lib -lpetscts -lpetscsnes -lpetscksp -lpetscdm - > lpetscmat -lpetscvec -lpetsc -lX11 -Wl,-rpath,/home/lux/ > csae1801/petsc/petsc-3.0.0-p8/linux32-intel-c-prod/lib -L/home/lux/ > csae1801/petsc/petsc-3.0.0-p8/linux32-intel-c-prod/lib - > lsuperlu_dist_2.3 -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common > -lpord -lparmetis -lmetis -lscalapack -lblacs -Wl,-rpath,/home/lux/ > csae1801/petsc/externalpackages/hypre-2.4.0b/src/hypre/lib -L/home/ > lux/csae1801/petsc/externalpackages/hypre-2.4.0b/src/hypre/lib - > lHYPRE -lmpichcxx -lstdc++ -lspooles -lflapack -lfblas -lnsl -laio - > lrt -lPEPCF90 -ldl -L/home/lux/csae1801/petsc/petsc-3.0.0-p8/linux32- > intel-c-prod/lib -lmpich -lpthread -lrt -L/opt/intel/cmkl/10.1.1.019/ > lib/em64t -L/opt/intel/Compiler/11.0/074/ipp/em64t/lib -L/opt/intel/ > Compiler/11.0/074/mkl/lib/em64t -L/opt/intel/Compiler/11.0/074/tbb/ > em64t/cc4.1.0_libc2.4_kernel2.6.16.21/lib -L/opt/intel/Compiler/ > 11.0/074/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -limf - > lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -lmpichf90 -lifport - > lifcore -lm -lm -lmpichcxx -lstdc++ -lmpichcxx -lstdc++ -ldl -lmpich > -lpthread -lrt -limf -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s - > ldl > ------------------------------------------ > #PETSc Option Table entries: > -dump_data false > -info > -ksp_compute_singularvalues > -ksp_rtol 1e-6 > -ksp_type preonly > -ksp_view > -log_summary > -mat_ignore_lower_triangular > -mat_mumps_icntl_14 200 > -mat_mumps_sym 1 > -mat_type sbaij > -monitorname errormonitor.dat > -nz 100 > -options_left > -pc_factor_mat_solver_package mumps > -pc_type cholesky > -preload false > -verb false > #End o PETSc Option Table entries > There are no unused options. > > 1olarr > > before sol put > sol put succes > before fullsol > /DIANA/AP/LS41 12:08:39 35.66-CPU 11.16-IO SOLVE > /DIANA/AP/LS41 12:08:39 35.66-CPU 11.18-IO FILLUP > /DIANA/AP/LS41 12:08:39 35.66-CPU 11.18-IO FILLUP > /DIANA/AP/LS41 12:08:39 35.66-CPU 11.18-IO POST > /DIANA/AP/LS41 12:08:41 36.79-CPU 12.27-IO POST > /DIANA/AP/LS41 12:08:41 36.79-CPU 12.27-IO LINSTA > /DIANA/DC/END 12:08:41 36.79-CPU 12.27-IO STOP > DIANA JOB 3446 finished > > 1 proc externalsolver stuck at 100% with 12%memory out of 12GB > 1 proc externalsolver at 0% > > mpiexec -np 2 externalsolver > [0] PetscInitialize(): PETSc successfully started: number of > processors = 2 > [0] PetscInitialize(): Running on machine: mat1.uibk.ac.at > [1] PetscInitialize(): PETSc successfully started: number of > processors = 2 > [1] PetscInitialize(): Running on machine: mat1.uibk.ac.at > [1] PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [1] PetscCommDuplicate(): returning tag 2147483647 > [0] PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [0] PetscCommDuplicate(): returning tag 2147483647 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483646 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483646 > [0] MatSetUpPreallocation(): Warning not preallocating matrix storage > [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [1] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [1] PetscCommDuplicate(): returning tag 2147483647 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483646 > [0] PetscCommDuplicate(): returning tag 2147483647 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483646 > [0] MatSetOption_MPISBAIJ(): Option SYMMETRIC ignored > [0] MatStashScatterBegin_Private(): No of messages: 1 > [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 30544064 > [0] MatStashScatterBegin_Private(): No of messages: 0 > [0] MatAssemblyBegin_MPISBAIJ(): Stash has 3818007 entries,uses 8 > mallocs. > [0] MatAssemblyBegin_MPISBAIJ(): Block-Stash has 3818007 entries, > uses 8 mallocs. > [0] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 221545 X 221545, block > size 1; storage space: 16676676 unneeded, 5477824 used > [0] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues > is 0 > [0] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 62 > [1] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 225367 X 225367, block > size 1; storage space: 16793278 unneeded, 5743422 used > [1] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues > is 0 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483645 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483644 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483643 > [0] PetscCommDuplicate(): returning tag 2147483639 > [0] PetscCommDuplicate(): returning tag 2147483634 > [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter > [0] VecScatterCreate(): General case: MPI to Seq > [0] PetscCommDuplicate(): returning tag 2147483630 > [0] PetscCommDuplicate(): returning tag 2147483625 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483642 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483641 > [0] PetscCommDuplicate[1] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros > blocks in any row is 65 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483645 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483644 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483643 > [1] PetscCommDuplicate(): returning tag 2147483639 > [1] PetscCommDuplicate(): returning tag 2147483634 > [1] PetscCommDuplicate(): returning tag 2147483630 > [1] PetscCommDuplicate(): returning tag 2147483625 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483642 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483641 > [1] PetscCommDuplicate(): returning tag 2147483620 > [1] PetscC(): returning tag 2147483620 > [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter > [0] VecScatterCreate(): General case: MPI to MPI > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483640 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483639 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483638 > [0] MatAssemblyEnd_SeqBAIJ(): Matrix size: 221545 X 6275, block size > 1; storage space: 22060506 unneeded, 93994 used > [0] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues > is 0 > [0] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 32 > [0] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 216534)/ > (num_localrows 221545) > 0.6. Use CompressedRow routines. > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [0] PetsommDuplicate(): Using internal PETSc communicator 1140850689 > -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483640 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483639 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483638 > [1] MatAssemblyEnd_SeqBAIJ(): Matrix size: 225367 X 0, block size 1; > storage space: 22536700 unneeded, 0 used > [1] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues > is 0 > [1] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 0 > [1] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 225367)/ > (num_localrows 225367) > 0.6. Use CompressedRow routines. > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483614 > [1] PetscCommDuplicate(): returning tag 2147483613 > [1]before KSPsetup > [1] PetscCommDuplicate(): Using internal PcCommDuplicate(): > returning tag 2147483614 > [0] PetscCommDuplicate(): returning tag 2147483613 > [0]before KSPsetup > [0] PCSetUp(): Setting up new PC > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483637 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483636 > [0] PetscCommDuplicate(): returning tag 2147483612 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483635 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483634 > ETSc communicator 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483637 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483636 > [1] PetscCommDuplicate(): returning tag 2147483612 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483635 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483634 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483633 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483632 > [0] PetscCommDuplicate(): returning tag 2147483607 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483633 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483632 > [1] PetscCommDuplicate(): returning tag 2147483607 > [0] PetscCommDuplicate(): returning tag 2147483602 > [0] VecScatterCreate(): Special case: processor zero gets entire > parallel vector, rest get none > [1] PetscCommDuplicate(): returning tag 2147483602 > > DMUMPS 4.8.4 > L D L^T Solver for symmetric positive definite matrices > Type of parallelism: Working host > > ****** ANALYSIS STEP ******** > > Density: NBdense, Average, Median = 0 49 50 > Ordering based on METIS > ** Peak of sequential stack size (number of real entries) : > 16763905. > A root of estimated size 2965 has been selected for > Scalapack. > > Leaving analysis phase with ... > INFOG(1) = 0 > INFOG(2) = 0 > -- (20) Number of entries in factors (estim.) = 120657071 > -- (3) Storage of factors (REAL, estimated) = 137364268 > -- (4) Storage of factors (INT , estimated) = 6138804 > -- (5) Maximum frontal size (estimated) = 3705 > -- (6) Number of nodes in the tree = 21863 > -- (7) Ordering option effectively used = 5 > ICNTL(6) Maximum transversal option = 0 > ICNTL(7) Pivot order option = 7 > Percentage of memory relaxation (effective) = 200 > Number of level 2 nodes = 1 > Number of split nodes = 0 > RINFO(1) Operations during elimination (estim) = 1.114D+11 > Distributed matrix entry format (ICNTL(18)) = 3 > ** Rank of proc needing largest memory in IC facto : 0 > ** Estimated corresponding MBYTES for IC facto : 2085 > ** Estimated avg. MBYTES per work. proc at facto (IC) : 2044 > ** TOTAL space in MBYTES for IC factorization : 4088 > ** Rank of proc needing largest memory for OOC facto : 1 > ** Estimated corresponding MBYTES for OOC facto : 644 > ** Estimated avg. MBYTES per work. proc at facto (OOC) : 630 > ** TOTAL space in MBYTES for OOC factorization : 1261 > > ****** FACTORIZATION STEP ******** > > > GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ... > NUMBER OF WORKING PROCESSES = 2 > REAL SPACE FOR FACTORS = 137364268 > INTEGER SPACE FOR FACTORS = 6138804 > MAXIMUM FRONTAL SIZE (ESTIMATED) = 3705 > NUMBER OF NODES IN THE TREE = 21863 > Maximum effective relaxed size of S = 239060000 > Average effective relaxed size of S = 234725641 > > REDISTRIB: TOTAL DATA LOCAL/SENT = 9086109 2238491 > GLOBAL TIME FOR MATRIX DISTRIBUTION = 0.8155 > ** Memory relaxation parameter ( ICNTL(14) ) : 200 > ** Rank of processor needing largest memory in facto : 0 > ** Space in MBYTES used by this processor for facto : 2085 > ** Avg. Space in MBYTES per working proc during facto : 2044 > > ELAPSED TIME FOR FACTORIZATION = 28.2595 > Maximum effective space used in S (KEEP(67) = 77516871 > Average effective space used in S (KEEP(67) = 75395654 > ** EFF Min: Rank of processor needing largest memory : 0 > ** EFF Min: Space in MBYTES used by this processor : 741 > ** EFF Min: Avg. Space in MBYTES per working proc : 720 > > GLOBAL STATISTICS > RINFOG(2) OPERATIONS DURING NODE ASSEMBLY = 2.565D+08 > ------(3) OPERATIONS DURING NODE ELIMINATION = 1.114D+11 > INFOG (9) REAL SPACE FOR FACTORS = 137364268 > INFOG(10) INTEGER SPACE FOR FACTORS = 6138809 > INFOG(11) MAXIMUM FRONT SIZE = 3705 > INFOG(29) NUMBER OF ENTRIES IN FACTORS = 116259976 > INFOG(12) NB OF NEGATIVE PIVOTS = 0 > INFOG(14) NUMBER OF MEMORY COMPRESS = 0 > [0]after KSPsetup > [0]RHS setup > [1]after KSPsetup > [1]RHS setup > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483601 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483601 > [1] PetscCommDuplicate(): returning tag 2147483596 > [0] PetscCommDuplicate(): returning tag 2147483596 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483591 > [0] PetscCommDuplicate(): returning tag 2147483591 > [0] VecAssemblyBegin_MPI(): Stash has 225367 entries, uses 12 mallocs. > [0] VecAssemblyBegin_MPI(): Block-Stash has 0 entries, uses 0 mallocs. > [0] PetscCommDuplicate(): returning tag 2147483590 > [1] PetscCommDuplicate(): returning tag 2147483590 > [1] PetscCommDuplicate(): returning tag 2147483589 > [0] PetscCommDuplicate(): returning tag 2147483589 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483631 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483630 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483631 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483630 > > 4 processes of externalsolver stuck with 0% cpu > > > mpiexec -np 4 externalsolver > [1] PetscInitialize(): PETSc successfully started: number of > processors = 4 > [1] PetscInitialize(): Running on machine: mat1.uibk.ac.at > [3] PetscInitialize(): PETSc successfully started: number of > processors = 4 > [3] PetscInitialize(): Running on machine: mat1.uibk.ac.at > [0] PetscInitialize(): PETSc successfully started: number of > processors = 4 > [0] PetscInitialize(): Running on machine: mat1.uibk.ac.at > [2] PetscInitialize(): PETSc successfully started: number of > processors = 4 > [2] PetscInitialize(): Running on machine: mat1.uibk.ac.at > [3] PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [3] PetscCommDuplicate(): returning tag 2147483647 > [1] PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [1] PetscCommDuplicate(): returning tag 2147483647 > [2] PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [2] PetscCommDuplicate(): returning tag 2147483647 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [2] PetscCommDuplicate(): returning tag 2147483646 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [3] PetscCommDuplicate(): returning tag 2147483646 > [0] PetscCommDuplicate(): Duplicating a communicator 1140850688 > -2080374784 max tags = 2147483647 > [0] PetscCommDuplicate(): returning tag 2147483647 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483646 > [0] MatSetUpPreallocation(): Warning not preallocating matrix storage > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483646 > [1] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [3] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [3] PetscCommDuplicate(): returning tag 2147483647 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483646 > [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [0] PetscCommDuplicate(): returning tag 2147483647 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483646 > [1] PetscCommDuplicate(): returning tag 2147483647 > [2] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [2] PetscCommDuplicate(): returning tag 2147483647 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483646 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483646 > [0] MatSetOption_MPISBAIJ(): Option SYMMETRIC ignored > [0] MatStashScatterBegin_Private(): No of messages: 3 > [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 11261544 > [0] MatStashScatterBegin_Private(): Mesg_to: 2: size: 10384840 > [0] MatStashScatterBegin_Private(): Mesg_to: 3: size: 393344 > [0] MatStashScatterBegin_Private(): No of messages: 0 > [0] MatAssemblyBegin_MPISBAIJ(): Stash has 2754963 entries,uses 8 > mallocs. > [0] MatAssemblyBegin_MPISBAIJ(): Block-Stash has 2754963 entries, > uses 8 mallocs. > [3] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 112413 X 112413, block > size 1; storage space: 8404901 unneeded, 2836399 used > [3] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues > is 0 > [3] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 65 > [1] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 113299 X 113299, block > size 1; storage space: 8647559 unneeded, 2682341 used > [1] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues > is 0 > [1] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 62 > [0] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 112871 X 112871, block > size 1; storage space: 8670798 unneeded, 2616302 used > [0] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues > is 0 > [0] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 52 > [2] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 108329 X 108329, block > size 1; storage space: 8247737 unneeded, 2585163 used > [2] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues > is 0 > [2] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 52 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483645 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483644 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483643 > [1] PetscCommDuplicate(): returning tag 2147483639 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483645 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483644 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483643 > [3] PetscCommDuplicate(): returning tag 2147483639 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483645 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483644 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483643 > [2] PetscCommDuplicate(): returning tag 2147483639 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483645 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483644 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483643 > [0] PetscCommDuplicate(): returning tag 2147483639 > [0] PetscCommDuplicate(): returning tag 2147483634 > [2] PetscCommDuplicate(): returning tag 2147483634 > [1] PetscCommDuplicate(): returning tag 2147483634 > [3] PetscCommDuplicate(): returning tag 2147483634 > [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter > [0] VecScatterCreate(): General case: MPI to Seq > [0] PetscCommDuplicate(): returning tag 2147483630 > [0] PetscCommDuplicate(): returning tag 2147483625 > [1] PetscCommDuplicate(): returning tag 2147483630 > [1] PetscCommDuplicate(): returning tag 2147483625 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483642 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483641 > [1] PetscCommDuplicate(): returning tag 2147483620 > [2] PetscCommDuplicate(): returning tag 2147483630 > [2] PetscCommDuplicate(): returning tag 2147483625 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483642 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483641 > [2] PetscCommDuplicate(): returning tag 2147483620 > [3] PetscCommDuplicate(): returning tag 2147483630 > [3] PetscCommDuplicate(): returning tag 2147483625 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483642 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483641 > [3] PetscCommDuplicate(): returning tag 2147483620 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483640 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483639 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483638 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483642 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483641 > [0] PetscCommDuplicate(): returning tag 2147483620 > [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter > [0] VecScatterCreate(): General case: MPI to MPI > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483640 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483639 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483638 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483640 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483639 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483638 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483640 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483639 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483638 > [3] MatAssemblyEnd_SeqBAIJ(): Matrix size: 112413 X 0, block size 1; > storage space: 11241300 unneeded, 0 used > [3] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues > is 0 > [3] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 0 > [3] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 112413)/ > (num_localrows 112413) > 0.6. Use CompressedRow routines. > [1] MatAssemblyEnd_SeqBAIJ(): Matrix size: 113299 X 2952, block size > 1; storage space: 11286236 unneeded, 43664 used > [1] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues > is 0 > [1] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 29 > [1] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 110902)/ > (num_localrows 113299) > 0.6. Use CompressedRow routines. > [2] MatAssemblyEnd_SeqBAIJ(): Matrix size: 108329 X 9258, block size > 1; storage space: 10624060 unneeded, 208840 used > [2] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues > is 0 > [2] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 43 > [2] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 92865)/ > (num_localrows 108329) > 0.6. Use CompressedRow routines. > [0] MatAssemblyEnd_SeqBAIJ(): Matrix size: 112871 X 17849, block > size 1; storage space: 10944569 unneeded, 342531 used > [0] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues > is 0 > [0] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 43 > [0] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 92260)/ > (num_localrows 112871) > 0.6. Use CompressedRow routines. > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483614 > [0] PetscCommDuplicate(): returning tag 2147483613 > [0]before KSPsetup > [0] PCSetUp(): Setting up new PC > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483637 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483636 > [0] PetscCommDuplicate(): returning tag 2147483612 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [1] PetscCommDuplicate(): returning tag 2147483614 > [1] PetscCommDuplicate(): returning tag 2147483613 > [1]before KSPsetup > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483637 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483636 > [1] PetscCommDuplicate(): returning tag 2147483612 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [2] PetscCommDuplicate(): returning tag 2147483614 > [2] PetscCommDuplicate(): returning tag 2147483613 > [2]before KSPsetup > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483637 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483636 > [2] PetscCommDuplicate(): returning tag 2147483612 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483635 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483634 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483635 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483634 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850688 -2080374784 > [3] PetscCommDuplicate(): returning tag 2147483614 > [3] PetscCommDuplicate(): returning tag 2147483613 > [3]before KSPsetup > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483637 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483636 > [3] PetscCommDuplicate(): returning tag 2147483612 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483635 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483634 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483635 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483634 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483633 > [1] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [1] PetscCommDuplicate(): returning tag 2147483632 > [1] PetscCommDuplicate(): returning tag 2147483607 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483633 > [2] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [2] PetscCommDuplicate(): returning tag 2147483632 > [2] PetscCommDuplicate(): returning tag 2147483607 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483633 > [0] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [0] PetscCommDuplicate(): returning tag 2147483632 > [0] PetscCommDuplicate(): returning tag 2147483607 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483633 > [3] PetscCommDuplicate(): Using internal PETSc communicator > 1140850689 -2080374783 > [3] PetscCommDuplicate(): returning tag 2147483632 > [3] PetscCommDuplicate(): returning tag 2147483607 > [0] PetscCommDuplicate(): returning tag 2147483602 > [1] PetscCommDuplicate(): returning tag 2147483602 > [2] PetscCommDuplicate(): returning tag 2147483602 > [3] PetscCommDuplicate(): returning tag 2147483602 > [0] VecScatterCreate(): Special case: processor zero gets entire > parallel vector, rest get none > > DMUMPS 4.8.4 > L D L^T Solver for symmetric positive definite matrices > Type of parallelism: Working host > > ****** ANALYSIS STEP ******** > > Density: NBdense, Average, Median = 0 49 50 > Ordering based on METIS > ** Peak of sequential stack size (number of real entries) : > 16763905. > A root of estimated size 2965 has been selected for > Scalapack. > > Leaving analysis phase with ... > INFOG(1) = 0 > INFOG(2) = 0 > -- (20) Number of entries in factors (estim.) = 120657071 > -- (3) Storage of factors (REAL, estimated) = 137362626 > -- (4) Storage of factors (INT , estimated) = 6167135 > -- (5) Maximum frontal size (estimated) = 3705 > -- (6) Number of nodes in the tree = 21863 > -- (7) Ordering option effectively used = 5 > ICNTL(6) Maximum transversal option = 0 > ICNTL(7) Pivot order option = 7 > Percentage of memory relaxation (effective) = 200 > Number of level 2 nodes = 5 > Number of split nodes = 0 > RINFO(1) Operations during elimination (estim) = 1.114D+11 > Distributed matrix entry format (ICNTL(18)) = 3 > ** Rank of proc needing largest memory in IC facto : 0 > ** Estimated corresponding MBYTES for IC facto : 1112 > ** Estimated avg. MBYTES per work. proc at facto (IC) : 1083 > ** TOTAL space in MBYTES for IC factorization : 4333 > ** Rank of proc needing largest memory for OOC facto : 1 > ** Estimated corresponding MBYTES for OOC facto : 465 > ** Estimated avg. MBYTES per work. proc at facto (OOC) : 421 > ** TOTAL space in MBYTES for OOC factorization : 1684 > > ****** FACTORIZATION STEP ******** > > > GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ... > NUMBER OF WORKING PROCESSES = 4 > REAL SPACE FOR FACTORS = 137362626 > INTEGER SPACE FOR FACTORS = 6167135 > MAXIMUM FRONTAL SIZE (ESTIMATED) = 3705 > NUMBER OF NODES IN THE TREE = 21863 > Maximum effective relaxed size of S = 125678965 > Average effective relaxed size of S = 122625031 > > REDISTRIB: TOTAL DATA LOCAL/SENT = 2497290 8939630 > GLOBAL TIME FOR MATRIX DISTRIBUTION = 0.6324 > ** Memory relaxation parameter ( ICNTL(14) ) : 200 > ** Rank of processor needing largest memory in facto : 0 > ** Space in MBYTES used by this processor for facto : 1112 > ** Avg. Space in MBYTES per working proc during facto : 1083 >
