Re: [Libmesh-users] AMR speed
Hello Rossi, It seems it is the projection functions that are computationally expensive. Would it be possible for you to run with the PatchRecovery estimator, and see if that results in a similar performance ? Thanks. > On Apr 27, 2017, at 12:14, Vikram Garg wrote: > > Rossi, yes compiling with perflog should give you all the details as in > the example. > > > > > > On Thu, Apr 27, 2017 at 10:54 AM, Rossi, Simone > wrote: > >> Dear Vikram, >> as in the examples, I am using the libmesh::KellyErrorEstimator. >> >> I’m compiling libmesh with the --enable-perflog option. Does it >> automatically give all the details you have listed in the example? >> >> For the time being, I am attaching two perfLogs I had saved with only >> “coarse scale” data for 2 levels of refinements. >> It looks like that most of the time is spent in the AMR step, probably in >> the call to reinit(). >> >> Thanks, >> Simone >> >> NO AMR: >> >> --- >> - >> | perf_log Performance: Alive time=18.0494, Active time=18.0426 >>| >> --- >> - >> | Event nCalls Total Time Avg TimeTotal >> Time Avg Time% of Active Time | >> | w/o Sub w/o Sub With >> SubWith Subw/o SWith S | >> |--- >> -| >> | no amr matrix assembly 1 0.1545 0.154465 >> 0.1545 0.1544650.86 0.86 | >> | no amr linear solve 1014.8069 0.047593 >>4.8069 0.04759326.6426.64| >> | no amr rhs assembly 10112.0348 0.119156 >> 12.0348 0.11915666.7066.70| >> | time loop 1 1.0464 >> 1.04642217.8884 17.888405 5.80 99.15| >> --- >> - >> | Totals:20418.0426 >> 100.00| >> --- >> - >> >> >> AMR: >> >> >> >> | perf_log Performance: Alive time=209.305, Active time=209.298 >>| >> --- >> - >> | Event nCalls Total Time Avg TimeTotal >> Time Avg Time% of Active Time | >> | w/o Sub w/o Sub With >> SubWith Subw/o SWith S | >> |--- >> -| >> | >>| >> | amr 303195.11020.643928 >> 195.11020.64392893.2293.22| >> | amr solve 30313.9907 0.046174 >> 13.9907 0.0461746.68 6.68 | >> | time loop 1 0.1974 0.197370 >> 209.2990209.299042 0.09 100.00 | >> --- >> - >> | Totals: 607209.2983 >> 100.00| >> --- >> - >> >> >> On Apr 27, 2017, at 11:02, Vikram Garg wrote: >> >> Hello Rossi, >>Two questions: >> >> 1) Which error estimator/indicator are you using to mark elements for >> refinement ? >> >> 2) Can you send the perfLog output from libMesh ? You might need to >> recompile libMesh with the option --enable-perflog. >> >> Looks something like this: >> >> --- >> -- >> | libMesh Performance: Alive time=0.013423, Active time=0.007095 >> | >> --- >> -- >> | Event nCalls Total Time Avg Time >> Total Time Avg Time% of Active Time | >> | w/o Sub w/o Sub >> With SubWith Subw/o SWith S | >> |--- >> ---
Re: [Libmesh-users] AMR speed
Ok, I ran again the tests with different max_h_levels with the perflog enabled. Let me know if you see anything here. Thanks, Simone NO AMR - | libMesh Performance: Alive time=77.5482, Active time=40.2976 | - | Event nCalls Total Time Avg TimeTotal Time Avg Time% of Active Time | | w/o Sub w/o Sub With SubWith Subw/o SWith S | |-| | | | | | DefaultCoupling | | operator() 98306 0.1609 0.020.1609 0.020.40 0.40 | | | | DofMap | | add_neighbors_to_send_list() 1 0.0959 0.0959300.3744 0.3743690.24 0.93 | | build_sparsity() 1 0.4701 0.4700551.1433 1.1432971.17 2.84 | | create_dof_constraints() 1 0.0137 0.0136730.0137 0.0136730.03 0.03 | | distribute_dofs()1 0.0126 0.0125780.4376 0.4376470.03 1.09 | | dof_indices()11010048 9.9728 0.019.9728 0.0124.7524.75| | prepare_send_list() 2 0. 0.020. 0.020.00 0.00 | | reinit() 1 0.0507 0.0506920.0507 0.0506920.13 0.13 | | | | EquationSystems | | build_parallel_solution_vector() 5 1.4241 0.2848112.4934 0.4986733.53 6.19 | | build_solution_vector() 5 0.0002 0.502.4936 0.4987240.00 6.19 | | | | ExodusII_IO | | write_nodal_data() 3 0.0774 0.0258160.0774 0.0258160.19 0.19 | | | | FE | | compute_shape_functions()10027008 11.7027 0.0111.7027 0.0129.0429.04| | init_shape_functions() 1020.0007 0.070.0007 0.070.00 0.00 | | | | FEMap | | compute_affine_map() 10027008 9.9328 0.019.9328 0.0124.6524.65| | init_reference_to_physical_map() 1020.0008 0.080.0008 0.080.00 0.00 | | | | GMVIO | | write_nodal_data() 2 0.2260 0.1130200.2260 0.1130200.56 0.56 | | | | GenericProjector | | operator() 1 0.8425 0.8425292.0842 2.0842322.09 5.17 | | project_edges98304 0.0765 0.010.0765 0.010.19 0.19 | | project_interior 98304 0.0765 0.00
Re: [Libmesh-users] AMR speed
Rossi, yes compiling with perflog should give you all the details as in the example. On Thu, Apr 27, 2017 at 10:54 AM, Rossi, Simone wrote: > Dear Vikram, > as in the examples, I am using the libmesh::KellyErrorEstimator. > > I’m compiling libmesh with the --enable-perflog option. Does it > automatically give all the details you have listed in the example? > > For the time being, I am attaching two perfLogs I had saved with only > “coarse scale” data for 2 levels of refinements. > It looks like that most of the time is spent in the AMR step, probably in > the call to reinit(). > > Thanks, > Simone > > NO AMR: > > --- > - > | perf_log Performance: Alive time=18.0494, Active time=18.0426 >| > --- > - > | Event nCalls Total Time Avg TimeTotal > Time Avg Time% of Active Time | > | w/o Sub w/o Sub With > SubWith Subw/o SWith S | > |--- > -| > | no amr matrix assembly 1 0.1545 0.154465 > 0.1545 0.1544650.86 0.86 | > | no amr linear solve 1014.8069 0.047593 >4.8069 0.04759326.6426.64| > | no amr rhs assembly 10112.0348 0.119156 > 12.0348 0.11915666.7066.70| > | time loop 1 1.0464 > 1.04642217.8884 17.888405 5.80 99.15| > --- > - > | Totals:20418.0426 > 100.00| > --- > - > > > AMR: > > > > | perf_log Performance: Alive time=209.305, Active time=209.298 >| > --- > - > | Event nCalls Total Time Avg TimeTotal > Time Avg Time% of Active Time | > | w/o Sub w/o Sub With > SubWith Subw/o SWith S | > |--- > -| > | >| > | amr 303195.11020.643928 > 195.11020.64392893.2293.22| > | amr solve 30313.9907 0.046174 > 13.9907 0.0461746.68 6.68 | > | time loop 1 0.1974 0.197370 > 209.2990209.299042 0.09 100.00 | > --- > - > | Totals: 607209.2983 > 100.00| > --- > - > > > On Apr 27, 2017, at 11:02, Vikram Garg wrote: > > Hello Rossi, >Two questions: > > 1) Which error estimator/indicator are you using to mark elements for > refinement ? > > 2) Can you send the perfLog output from libMesh ? You might need to > recompile libMesh with the option --enable-perflog. > > Looks something like this: > > --- > -- > | libMesh Performance: Alive time=0.013423, Active time=0.007095 >| > --- > -- > | Event nCalls Total Time Avg Time > Total Time Avg Time% of Active Time | > | w/o Sub w/o Sub > With SubWith Subw/o SWith S | > |--- > --| > | > | > | > | > | DofMap >| > | add_neighbors_to_send_list() 6 0.0001 0.12 > 0.0001 0.121.01 1.01 | > | build_sparsity() 6 0.0002 0.33 > 0.0011 0.00018
Re: [Libmesh-users] AMR speed
Dear Vikram, as in the examples, I am using the libmesh::KellyErrorEstimator. I’m compiling libmesh with the --enable-perflog option. Does it automatically give all the details you have listed in the example? For the time being, I am attaching two perfLogs I had saved with only “coarse scale” data for 2 levels of refinements. It looks like that most of the time is spent in the AMR step, probably in the call to reinit(). Thanks, Simone NO AMR: | perf_log Performance: Alive time=18.0494, Active time=18.0426 | | Event nCalls Total Time Avg TimeTotal Time Avg Time% of Active Time | | w/o Sub w/o Sub With Sub With Subw/o SWith S | || | no amr matrix assembly 1 0.1545 0.154465 0.1545 0.1544650.86 0.86 | | no amr linear solve 1014.8069 0.047593 4.8069 0.04759326.6426.64| | no amr rhs assembly 10112.0348 0.119156 12.0348 0.11915666.7066.70| | time loop 1 1.0464 1.04642217.8884 17.888405 5.80 99.15| | Totals:20418.0426 100.00| AMR: | perf_log Performance: Alive time=209.305, Active time=209.298 | | Event nCalls Total Time Avg TimeTotal Time Avg Time% of Active Time | | w/o Sub w/o Sub With Sub With Subw/o SWith S | || | | | amr 303195.11020.643928195.1102 0.64392893.2293.22| | amr solve 30313.9907 0.04617413.9907 0.0461746.68 6.68 | | time loop 1 0.1974 0.197370 209.2990209.299042 0.09 100.00 | | Totals: 607209.2983 100.00| On Apr 27, 2017, at 11:02, Vikram Garg mailto:vikram.v.g...@gmail.com>> wrote: Hello Rossi, Two questions: 1) Which error estimator/indicator are you using to mark elements for refinement ? 2) Can you send the perfLog output from libMesh ? You might need to recompile libMesh with the option --enable-perflog. Looks something like this: - | libMesh Performance: Alive time=0.013423, Active time=0.007095 | - | Event nCalls Total Time Avg TimeTotal Time Avg Time% of Active Time | | w/o Sub w/o Sub With SubWith Subw/o SWith S | |-| | | | | | DofMap | | add_neighbors_to_send_list() 6 0.0001 0.120.0001 0.121.01 1.01 | | build_sparsity()
Re: [Libmesh-users] AMR speed
Hello Rossi, Two questions: 1) Which error estimator/indicator are you using to mark elements for refinement ? 2) Can you send the perfLog output from libMesh ? You might need to recompile libMesh with the option --enable-perflog. Looks something like this: - | libMesh Performance: Alive time=0.013423, Active time=0.007095 | - | Event nCalls Total Time Avg Time Total Time Avg Time% of Active Time | | w/o Sub w/o Sub With SubWith Subw/o SWith S | |-| | | | | | DofMap | | add_neighbors_to_send_list() 6 0.0001 0.12 0.0001 0.121.01 1.01 | | build_sparsity() 6 0.0002 0.33 0.0011 0.0001872.78 15.84| | create_dof_constraints() 6 0. 0.01 0. 0.010.07 0.07 | | distribute_dofs()6 0.0001 0.25 0.0004 0.662.09 5.57 | | dof_indices()6880.0010 0.01 0.0010 0.0114.3614.36| | old_dof_indices()3000.0001 0.00 0.0001 0.000.96 0.96 | | prepare_send_list() 7 0. 0.00 0. 0.000.01 0.01 | | reinit() 6 0.0002 0.41 0.0002 0.413.48 3.48 | | | | EquationSystems | | build_solution_vector() 1 0.0001 0.56 0.0001 0.640.79 0.90 | Thanks. On Wed, Apr 26, 2017 at 10:09 PM, Rossi, Simone wrote: > Dear Roy, dear Paul, dear all, > I am testing AMR in libmesh using simple linear elements. > My test case is a propagating front described by a reaction-diffusion > equation with a cubic bistable reaction term. > I followed the adaptivity examples to create this test case. > > The run times for 100 timesteps using AMR can be more than 10 times slower > than when using a fine uniform grid. > For example, with a 16 x 16 x 16 uniform grid, 100 iterations take about > 18 seconds with a single processor. > With AMR, using a 2 x 2 x 2 grid and 3 levels of refinement, 100 > iterations take about 800 seconds. > > I’m attaching the code I’m using. > Without AMR, I build the matrix ( mass + dt * stiffness ) once and I > update the rhs at every timestep. > Conversely, with AMR I am building the matrix and the rhs at every > timestep for all the refinement levels. > Do you have any suggestions? > > Thanks a lot for your help, > All the best, > Simone > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Libmesh-users mailing list > Libmesh-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/libmesh-users > > -- Vikram Garg Postdoctoral Associate The University of Texas at Austin http://vikramvgarg.wordpress.com/ http://www.runforindia.org/runners/vikramg -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Libmesh-users mailing list Libmesh-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libmesh-users
