Ata, You might be scaling past the reasonable limit for libMesh. I don't know what solver you are using, but for a strong scaling study we generally don't go below 10,000 local DOFs. This is the recommended floor for PETSc too: http://www.mcs.anl.gov/petsc/documentation/faq.html#slowerparallel
Before you start drawing conclusions about scaling, you might start with a bigger problem and see if it scales well to the ~20,000 local DOF range. Cody On Mon, Jun 10, 2013 at 8:42 AM, Ataollah Mesgarnejad < [email protected]> wrote: > Dear all, > > I've been doing some scaling tests on my code. When I look at time (or % > of time) spent at each stage in libMesh log I see that the > enforce_constraints_exactly stage in DofMap is scaling very bad. I was > wondering if anyone can comment. > > Here is my EquationSystems.print_info(): > > EquationSystems > n_systems()=2 > System #0, "elasticity_system" > Type "TransientLinearImplicit" > Variables={ "u" "v" } > Finite Element Types="LAGRANGE", "JACOBI_20_00" > Infinite Element Mapping="CARTESIAN" > Approximation Orders="FIRST", "THIRD" > n_dofs()=48660 > n_local_dofs()=930 > n_constrained_dofs()=1048 > n_local_constrained_dofs()=56 > n_vectors()=3 > n_matrices()=1 > DofMap Sparsity > Average On-Processor Bandwidth <= 13.6478 > Average Off-Processor Bandwidth <= 0.904233 > Maximum On-Processor Bandwidth <= 20 > Maximum Off-Processor Bandwidth <= 16 > DofMap Constraints > Number of DoF Constraints = 1048 > Average DoF Constraint Length= 0 > Number of Node Constraints = 0 > System #1, "fracture_system" > Type "TransientNonlinearImplicit" > Variables="psi" > Finite Element Types="LAGRANGE", "JACOBI_20_00" > Infinite Element Mapping="CARTESIAN" > Approximation Orders="FIRST", "THIRD" > n_dofs()=24330 > n_local_dofs()=465 > n_constrained_dofs()=167 > n_local_constrained_dofs()=0 > n_vectors()=3 > n_matrices()=1 > DofMap Sparsity > Average On-Processor Bandwidth <= 6.82388 > Average Off-Processor Bandwidth <= 0.452117 > Maximum On-Processor Bandwidth <= 10 > Maximum Off-Processor Bandwidth <= 8 > DofMap Constraints > Number of DoF Constraints = 167 > Average DoF Constraint Length= 0 > Number of Node Constraints = 0 > > > and here is the how scaling looks for every stage that took > 1% of time: > > > > Best, > Ata > > ------------------------------------------------------------------------------ > How ServiceNow helps IT people transform IT departments: > 1. A cloud service to automate IT design, transition and operations > 2. Dashboards that offer high-level views of enterprise services > 3. A single system of record for all IT processes > http://p.sf.net/sfu/servicenow-d2d-j > _______________________________________________ > Libmesh-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/libmesh-users > > ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j _______________________________________________ Libmesh-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-users
