On Thu, Sep 18, 2008 at 1:31 PM, Nguyen, Hung V ERDC-ITL-MS <Hung.V.Nguyen at usace.army.mil> wrote: > All, > > I have a test code that reads matrix in CSR format, rhs and provided > solution. I run this code on Cray XT3 system with 1, 2, 4, 8 and 16 cores. > The timing spent on KSP are 53.50, 213.69, 110.43, 66.90, and 35.60 secs for > 1, 2, 4, 8, and 16 cores; respectively. Why is a running time on 1 core > smaller than others (2, 4, 8 cores)? Does re-ordering of matrix help? Did I > do something wrong in the code?
It appears that you are using Block-Jacobi ILU(0). This is a completely different preconditioner for each different number of processes. Matt > Thanks for your help. > > Note: the matrix, rhs, provided and computed solution are written to matlab > files via petsc functions. It seems to me that we got the right solution > (using matlab to verify). Then, Here is some info about matrix A: > > hvnguyen:sapphire01% head matrix.m > % Size = 59409 59409 > % Nonzeros = 1113875 > zzz = zeros(1113875,3); > >>> cond (Mat_0) > Warning: Using CONDEST instead of COND for sparse matrix. >> In cond at 28 > > ans = > > 3.9310e+09 > > -- 1 pes: > hvnguyen:sapphire09% yod -np 1 ./test_matrix -ksp_type cg -pc_type bjacobi > -ksp_rtol 1.0e-15 -ksp_max_it 50000 > yod: -sz is 1, so -proc is reset from VN to 0 > reading nlrn ... > reading lrn ... > reading cmatrix and rld ... > Number of iterations and Time in PETSc solver = 2603 > 53.50315904617310 secs > 2 norm of the error from the provided & computed solution = > 3.2424850631419981E-008 > Maximum error from the provided & computed solution (infinity norm) = > 6.1342397827957029E-010 > 1 norm of the error from the provided & computed solution > 4.8108500117322262E-006 > > -- 2 pes: > hvnguyen:sapphire09% yod -np 2 ./test_matrix -ksp_type cg -pc_type bjacobi > -ksp_rtol 1.0e-15 -ksp_max_it 50000 reading nlrn ... > reading lrn ... > reading cmatrix and rld ... > Number of iterations and Time in PETSc solver = 17934 > 213.6974561214447 secs > 2 norm of the error from the provided & computed solution = > 2.4050101709163445E-008 > Maximum error from the provided & computed solution (infinity norm) = > 4.5970760531588439E-010 > 1 norm of the error from the provided & computed solution > 3.0008169642927509E-006 > FORTRAN STOP > > -- 4 peshvnguyen:sapphire09% yod -np 4 ./test_matrix -ksp_type cg -pc_type > bjacobi -ksp_rtol 1.0e-15 -ksp_max_it 50000 reading nlrn ... > reading lrn ... > reading cmatrix and rld ... > Number of iterations and Time in PETSc solver = 19363 > 110.4340059757233 secs > 2 norm of the error from the provided & computed solution = > 1.5643187342954139E-008 > Maximum error from the provided & computed solution (infinity norm) = > 3.4082958677572606E-010 > 1 norm of the error from the provided & computed solution > 2.0655232735563973E-006 > > --- 8 pes > hvnguyen:sapphire09% yod -np 8 ./test_matrix -ksp_type cg -pc_type bjacobi > -ksp_rtol 1.0e-15 -ksp_max_it 50000 reading nlrn ... > reading lrn ... > reading cmatrix and rld ... > Number of iterations and Time in PETSc solver = 25953 > 66.90357208251953 secs > 2 norm of the error from the provided & computed solution = > 2.0977543976205624E-008 > Maximum error from the provided & computed solution (infinity norm) = > 4.0076741925076931E-010 > 1 norm of the error from the provided & computed solution > 2.6098351291261861E-006 > > --- 16 pes > hvnguyen:sapphire09% yod -np 16 ./test_matrix -ksp_type cg -pc_type bjacobi > -ksp_rtol 1.0e-15 -ksp_max_it 50000 reading nlrn ... > reading lrn ... > reading cmatrix and rld ... > Number of iterations and Time in PETSc solver = 25842 > 35.60693192481995 secs > 2 norm of the error from the provided & computed solution = > 1.5121379940662423E-008 > Maximum error from the provided & computed solution (infinity norm) = > 3.1954527912603226E-010 > 1 norm of the error from the provided & computed solution > 2.0324863426877444E-006 > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
