Just have it do one or a few iterations.
On Mon, Nov 19, 2012 at 1:36 PM, Thomas Witkowski < thomas.witkowski at tu-dresden.de> wrote: > I can do this! Should I stop the run after KSPSetUp? Or do you want to > see the log_summary file from the whole run? > > Thomas > > Am 19.11.2012 13:33, schrieb Jed Brown: > > Always, always, always send -log_summary when asking about performance. > > > On Mon, Nov 19, 2012 at 11:26 AM, Thomas Witkowski < > thomas.witkowski at tu-dresden.de> wrote: > >> I have some scaling problem in KSPSetUp, maybe some of you can help me to >> fix it. It takes 4.5 seconds on 64 cores, and 4.0 cores on 128 cores. The >> matrix has around 11 million rows and is not perfectly balanced, but the >> number of maximum rows per core in the 128 cases is exactly halfe of the >> number in the case when using 64 cores. Besides the scaling, why does the >> setup takes so long? I though that just some objects are created but no >> calculation is going on! >> >> The KSPView on the corresponding solver objects is as follows: >> >> KSP Object:(ns_) 64 MPI processes >> type: fgmres >> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >> Orthogonalization with no iterative refinement >> GMRES: happy breakdown tolerance 1e-30 >> maximum iterations=100, initial guess is zero >> tolerances: relative=1e-06, absolute=1e-08, divergence=10000 >> right preconditioning >> has attached null space >> using UNPRECONDITIONED norm type for convergence test >> PC Object:(ns_) 64 MPI processes >> type: fieldsplit >> FieldSplit with Schur preconditioner, factorization FULL >> Preconditioner for the Schur complement formed from the block >> diagonal part of A11 >> Split info: >> Split number 0 Defined by IS >> Split number 1 Defined by IS >> KSP solver for A00 block >> KSP Object: (ns_fieldsplit_velocity_) 64 MPI processes >> type: preonly >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >> left preconditioning >> using DEFAULT norm type for convergence test >> PC Object: (ns_fieldsplit_velocity_) 64 MPI processes >> type: none >> linear system matrix = precond matrix: >> Matrix Object: 64 MPI processes >> type: mpiaij >> rows=11068107, cols=11068107 >> total: nonzeros=315206535, allocated nonzeros=315206535 >> total number of mallocs used during MatSetValues calls =0 >> not using I-node (on process 0) routines >> KSP solver for S = A11 - A10 inv(A00) A01 >> KSP Object: (ns_fieldsplit_pressure_) 64 MPI processes >> type: gmres >> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >> Orthogonalization with no iterative refinement >> GMRES: happy breakdown tolerance 1e-30 >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >> left preconditioning >> using DEFAULT norm type for convergence test >> PC Object: (ns_fieldsplit_pressure_) 64 MPI processes >> type: none >> linear system matrix followed by preconditioner matrix: >> Matrix Object: 64 MPI processes >> type: schurcomplement >> rows=469678, cols=469678 >> Schur complement A11 - A10 inv(A00) A01 >> A11 >> Matrix Object: 64 MPI processes >> type: mpiaij >> rows=469678, cols=469678 >> total: nonzeros=0, allocated nonzeros=0 >> total number of mallocs used during MatSetValues calls =0 >> using I-node (on process 0) routines: found 1304 nodes, >> limit used is 5 >> A10 >> Matrix Object: 64 MPI processes >> type: mpiaij >> rows=469678, cols=11068107 >> total: nonzeros=89122957, allocated nonzeros=89122957 >> total number of mallocs used during MatSetValues calls =0 >> not using I-node (on process 0) routines >> KSP of A00 >> KSP Object: (ns_fieldsplit_velocity_) 64 MPI >> processes >> type: preonly >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, >> divergence=10000 >> left preconditioning >> using DEFAULT norm type for convergence test >> PC Object: (ns_fieldsplit_velocity_) 64 MPI >> processes >> type: none >> linear system matrix = precond matrix: >> Matrix Object: 64 MPI processes >> type: mpiaij >> rows=11068107, cols=11068107 >> total: nonzeros=315206535, allocated nonzeros=315206535 >> total number of mallocs used during MatSetValues calls >> =0 >> not using I-node (on process 0) routines >> A01 >> Matrix Object: 64 MPI processes >> type: mpiaij >> rows=11068107, cols=469678 >> total: nonzeros=88821041, allocated nonzeros=88821041 >> total number of mallocs used during MatSetValues calls =0 >> not using I-node (on process 0) routines >> Matrix Object: 64 MPI processes >> type: mpiaij >> rows=469678, cols=469678 >> total: nonzeros=0, allocated nonzeros=0 >> total number of mallocs used during MatSetValues calls =0 >> using I-node (on process 0) routines: found 1304 nodes, limit >> used is 5 >> linear system matrix = precond matrix: >> Matrix Object: 64 MPI processes >> type: mpiaij >> rows=11537785, cols=11537785 >> total: nonzeros=493150533, allocated nonzeros=510309207 >> total number of mallocs used during MatSetValues calls =0 >> not using I-node (on process 0) routines >> >> >> >> >> Thomas >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20121119/f0095ca1/attachment.html>
