Bruno, thanks for your quick response.
I varied the preconditioner 6 times with the resulting memory usage
essentially the same. See results in attached file. Does anyone have
something else that may be important to resulting in the large memory usage
difference. Another fact I noticed was there was much less difference
between the peak virtual memory and peak resident memory in step-17 vs.
step-8.
I would like to repeat, that simple beam tests result in correct maximum
deflection and angle, for a number of loads and situations.
template <int dim>
unsigned int ElasticProblem<dim>::solve ()
{
SolverControl solver_control (solution.size(),
1e-8*system_rhs.l2_norm());
PETScWrappers::SolverCG cg (solver_control,
mpi_communicator);
//#### Original PETScWrappers::PreconditionBlockJacobi
preconditioner(system_matrix);
//#### PETScWrappers::PreconditionJacobi preconditioner(system_matrix);
//#### PETScWrappers::PreconditionSSOR preconditioner(system_matrix);
//#### PETScWrappers::PreconditionSOR preconditioner(system_matrix);
//#### PETScWrappers::PreconditionNone preconditioner(system_matrix);
//#### PETScWrappers::PreconditionLU preconditioner(system_matrix);
//#### std::cerr << "PETScWrappers::PreconditionLU
preconditioner(system_matrix);" << std::endl;
PETScWrappers::PreconditionILU preconditioner(system_matrix);
std::cerr << "PETScWrappers::PreconditionILU
preconditioner(system_matrix);" << std::endl;
cg.solve (system_matrix, solution, system_rhs,
preconditioner);
Vector<double> localized_solution (solution);
hanging_node_constraints.distribute (localized_solution);
solution = localized_solution;
return solver_control.last_step();
}
Thanks again.
Pete Griffin
On Monday, July 25, 2016 at 1:59:53 PM UTC-4, Pete Griffin wrote:
>
> I found that running step-8 and step-17 on a single processor Intel® Core™
> i7-3630QM CPU @ 2.40GHz × 8 used substantially more Peak resident memory (>
> 5x) than I thought it would. This surprised me since I thought from reading
> step-17 that the memory increase was on the order of the solution vector
> which should have been << 2x greater. I verified some of the larger memory
> usage numbers using top.
>
> Is my assumption correct that 5x Peak resident memory is more than it
> should be?
>
> The results of other simulations with a beam with body-force load and with
> traction loads with and without HP and with and without MPI/PETSc all show
> the same results and they agree with beam theory.
>
> The output of the modified step-8.cc and step-17.cc are attached along
> with a plot of peak virtual memory and peak resident memory
> vs. DOF. The changes between the original distributed step-8 and step-17,
> with comments and extra newlines excluded (modified file >), are as below:
>
> Thanks beforehand
>
> Pete Griffin
>
>
> ======================================================================================
> diff ~/Documents/Zipstore2/dealii-8.4.1-PETSc/examples/step-8/step-8.cc
> step-8.cc
> 56c47,48
> < // This again is C++:
> ---
> > #include <deal.II/base/utilities.h>
>
> 767a394,402
> >
> > Utilities::System::MemoryStats stats;
> > Utilities::System::get_memory_stats(stats);
> > std::stringstream Str;
> >
> > Str.str("");
> > Str << " Peak virtual memory: " << stats.VmSize/1024 << " MB,
> Peak resident memory: "
> > << stats.VmRSS/1024 << " MB" << std::endl;
> > std::cout << Str.str();
>
> 781c411
> < Step8::ElasticProblem<2> elastic_problem_2d;
> ---
> > Step8::ElasticProblem<3> elastic_problem_2d;
>
>
> ======================================================================================
> diff ~/Documents/Zipstore2/dealii-8.4.1-PETSc/examples/step-17/step-17.cc
> ../step-17/step-17.cc
> 84a50
> > #include <deal.II/base/utilities.h>
> 1015c355
> < for (unsigned int cycle=0; cycle<10; ++cycle)
> ---
> > for (unsigned int cycle=0; cycle<8; ++cycle)
> 1018d357
> <
> 1022c361
> < triangulation.refine_global (3);
> ---
> > triangulation.refine_global (2);
> 1049a383,391
> >
> > Utilities::System::MemoryStats stats;
> > Utilities::System::get_memory_stats(stats);
> > std::stringstream Str;
> >
> > Str.str("");
> > Str << " Peak virtual memory: " << stats.VmSize/1024 << " MB,
> Peak resident memory: "
> > << stats.VmRSS/1024 << " MB" << std::endl;
> > std::cout << Str.str();
> 1073,1074c403
> <
> < ElasticProblem<2> elastic_problem;
> ---
> > ElasticProblem<3> elastic_problem;
>
>
> =============================================================================================
>
>
--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see
https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
~/awork/dealii-work/examples/step-8$ make release run
[100%] Run step-8 with Release configuration
Cycle 0:
Number of active cells: 64
Number of degrees of freedom: 375
Peak virtual memory: 600 MB, Peak resident memory: 47 MB
Cycle 1:
Number of active cells: 204
Number of degrees of freedom: 1059
Peak virtual memory: 1069 MB, Peak resident memory: 51 MB
Cycle 2:
Number of active cells: 708
Number of degrees of freedom: 3231
Peak virtual memory: 1072 MB, Peak resident memory: 56 MB
Cycle 3:
Number of active cells: 2388
Number of degrees of freedom: 9891
Peak virtual memory: 1083 MB, Peak resident memory: 71 MB
Cycle 4:
Number of active cells: 7484
Number of degrees of freedom: 29277
Peak virtual memory: 1114 MB, Peak resident memory: 104 MB
Cycle 5:
Number of active cells: 23220
Number of degrees of freedom: 82779
Peak virtual memory: 1203 MB, Peak resident memory: 193 MB
Cycle 6:
Number of active cells: 71996
Number of degrees of freedom: 247185
Peak virtual memory: 1470 MB, Peak resident memory: 468 MB
Cycle 7:
Number of active cells: 223140
Number of degrees of freedom: 743361
Peak virtual memory: 2276 MB, Peak resident memory: 1300 MB
Using PETScWrappers::PreconditionBlockJacobi preconditioner(system_matrix);
~/awork/dealii-work/examples/step-17$ make release run
[100%] Run step-17 with Release configuration
Cycle 0:
Number of active cells: 64
Number of degrees of freedom: 375 (by partition: 375)
Solver converged in 6 iterations.
Peak virtual memory: 786 MB, Peak resident memory: 56 MB
Cycle 1:
Number of active cells: 204
Number of degrees of freedom: 1059 (by partition: 1059)
Solver converged in 10 iterations.
Peak virtual memory: 798 MB, Peak resident memory: 72 MB
Cycle 2:
Number of active cells: 708
Number of degrees of freedom: 3231 (by partition: 3231)
Solver converged in 17 iterations.
Peak virtual memory: 825 MB, Peak resident memory: 98 MB
Cycle 3:
Number of active cells: 2388
Number of degrees of freedom: 9891 (by partition: 9891)
Solver converged in 29 iterations.
Peak virtual memory: 908 MB, Peak resident memory: 182 MB
Cycle 4:
Number of active cells: 7484
Number of degrees of freedom: 29277 (by partition: 29277)
Solver converged in 48 iterations.
Peak virtual memory: 1142 MB, Peak resident memory: 416 MB
Cycle 5:
Number of active cells: 23220
Number of degrees of freedom: 82779 (by partition: 82779)
Solver converged in 74 iterations.
Peak virtual memory: 1798 MB, Peak resident memory: 1071 MB
Cycle 6:
Number of active cells: 71996
Number of degrees of freedom: 247185 (by partition: 247185)
Solver converged in 113 iterations.
Peak virtual memory: 3772 MB, Peak resident memory: 3045 MB
Cycle 7:
Number of active cells: 223140
Number of degrees of freedom: 743361 (by partition: 743361)
Solver converged in 156 iterations.
Peak virtual memory: 9709 MB, Peak resident memory: 8981 MB
==================================================================
Using PETScWrappers::PreconditionJacobi preconditioner(system_matrix);
[100%] Switch CMAKE_BUILD_TYPE to Release
-- Autopilot invoked
-- Run $ make info to print a detailed help message
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/pgriffin/awork/dealii-work/examples/step-17
Scanning dependencies of target step-17
[ 50%] Building CXX object CMakeFiles/step-17.dir/step-17.cc.o
[100%] Linking CXX executable step-17
[100%] Built target step-17
Built target release
[ 66%] Built target step-17
[100%] Run step-17 with Release configuration
Cycle 0:
Number of active cells: 64
Number of degrees of freedom: 375 (by partition: 375)
Solver converged in 8 iterations.
Peak virtual memory: 786 MB, Peak resident memory: 56 MB
Cycle 1:
Number of active cells: 204
Number of degrees of freedom: 1059 (by partition: 1059)
Solver converged in 23 iterations.
Peak virtual memory: 798 MB, Peak resident memory: 72 MB
Cycle 2:
Number of active cells: 708
Number of degrees of freedom: 3231 (by partition: 3231)
Solver converged in 39 iterations.
Peak virtual memory: 824 MB, Peak resident memory: 97 MB
Cycle 3:
Number of active cells: 2388
Number of degrees of freedom: 9891 (by partition: 9891)
Solver converged in 65 iterations.
Peak virtual memory: 905 MB, Peak resident memory: 178 MB
Cycle 4:
Number of active cells: 7484
Number of degrees of freedom: 29277 (by partition: 29277)
Solver converged in 102 iterations.
Peak virtual memory: 1142 MB, Peak resident memory: 415 MB
Cycle 5:
Number of active cells: 23220
Number of degrees of freedom: 82779 (by partition: 82779)
Solver converged in 152 iterations.
Peak virtual memory: 1798 MB, Peak resident memory: 1071 MB
Cycle 6:
Number of active cells: 71996
Number of degrees of freedom: 247185 (by partition: 247185)
Solver converged in 239 iterations.
Peak virtual memory: 3771 MB, Peak resident memory: 3044 MB
Cycle 7:
Number of active cells: 223140
Number of degrees of freedom: 743361 (by partition: 743361)
Solver converged in 351 iterations.
Peak virtual memory: 9705 MB, Peak resident memory: 8978 MB
[100%] Built target run
==================================================================
Using PETScWrappers::PreconditionSSOR preconditioner(system_matrix);
[100%] Switch CMAKE_BUILD_TYPE to Release
-- Autopilot invoked
-- Run $ make info to print a detailed help message
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/pgriffin/awork/dealii-work/examples/step-17
Scanning dependencies of target step-17
[ 50%] Building CXX object CMakeFiles/step-17.dir/step-17.cc.o
[100%] Linking CXX executable step-17
[100%] Built target step-17
Built target release
[ 66%] Built target step-17
[100%] Run step-17 with Release configuration
Cycle 0:
Number of active cells: 64
Number of degrees of freedom: 375 (by partition: 375)
Solver converged in 8 iterations.
Peak virtual memory: 786 MB, Peak resident memory: 56 MB
Cycle 1:
Number of active cells: 204
Number of degrees of freedom: 1059 (by partition: 1059)
Solver converged in 13 iterations.
Peak virtual memory: 798 MB, Peak resident memory: 72 MB
Cycle 2:
Number of active cells: 708
Number of degrees of freedom: 3231 (by partition: 3231)
Solver converged in 22 iterations.
Peak virtual memory: 824 MB, Peak resident memory: 97 MB
Cycle 3:
Number of active cells: 2388
Number of degrees of freedom: 9891 (by partition: 9891)
Solver converged in 35 iterations.
Peak virtual memory: 905 MB, Peak resident memory: 179 MB
Cycle 4:
Number of active cells: 7484
Number of degrees of freedom: 29277 (by partition: 29277)
Solver converged in 54 iterations.
Peak virtual memory: 1143 MB, Peak resident memory: 416 MB
Cycle 5:
Number of active cells: 23220
Number of degrees of freedom: 82779 (by partition: 82779)
Solver converged in 76 iterations.
Peak virtual memory: 1801 MB, Peak resident memory: 1074 MB
Cycle 6:
Number of active cells: 71996
Number of degrees of freedom: 247185 (by partition: 247185)
Solver converged in 117 iterations.
Peak virtual memory: 3783 MB, Peak resident memory: 3055 MB
Cycle 7:
Number of active cells: 223140
Number of degrees of freedom: 743361 (by partition: 743361)
Solver converged in 166 iterations.
Peak virtual memory: 9739 MB, Peak resident memory: 9012 MB
==================================================================
Using PETScWrappers::PreconditionSOR preconditioner(system_matrix);
[100%] Switch CMAKE_BUILD_TYPE to Release
-- Autopilot invoked
-- Run $ make info to print a detailed help message
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/pgriffin/awork/dealii-work/examples/step-17
Scanning dependencies of target step-17
[ 50%] Building CXX object CMakeFiles/step-17.dir/step-17.cc.o
[100%] Linking CXX executable step-17
[100%] Built target step-17
Built target release
[ 66%] Built target step-17
[100%] Run step-17 with Release configuration
Cycle 0:
Number of active cells: 64
Number of degrees of freedom: 375 (by partition: 375)
Solver converged in 8 iterations.
Peak virtual memory: 786 MB, Peak resident memory: 56 MB
Cycle 1:
Number of active cells: 204
Number of degrees of freedom: 1059 (by partition: 1059)
Solver converged in 13 iterations.
Peak virtual memory: 798 MB, Peak resident memory: 71 MB
Cycle 2:
Number of active cells: 708
Number of degrees of freedom: 3231 (by partition: 3231)
Solver converged in 22 iterations.
Peak virtual memory: 824 MB, Peak resident memory: 97 MB
Cycle 3:
Number of active cells: 2388
Number of degrees of freedom: 9891 (by partition: 9891)
Solver converged in 35 iterations.
Peak virtual memory: 905 MB, Peak resident memory: 178 MB
Cycle 4:
Number of active cells: 7484
Number of degrees of freedom: 29277 (by partition: 29277)
Solver converged in 54 iterations.
Peak virtual memory: 1143 MB, Peak resident memory: 416 MB
Cycle 5:
Number of active cells: 23220
Number of degrees of freedom: 82779 (by partition: 82779)
Solver converged in 76 iterations.
Peak virtual memory: 1801 MB, Peak resident memory: 1074 MB
Cycle 6:
Number of active cells: 71996
Number of degrees of freedom: 247185 (by partition: 247185)
Solver converged in 117 iterations.
Peak virtual memory: 3783 MB, Peak resident memory: 3055 MB
Cycle 7:
Number of active cells: 223140
Number of degrees of freedom: 743361 (by partition: 743361)
Solver converged in 166 iterations.
Peak virtual memory: 9739 MB, Peak resident memory: 9012 MB
==================================================================
Using PETScWrappers::PreconditionNone preconditioner(system_matrix);
[100%] Switch CMAKE_BUILD_TYPE to Release
-- Autopilot invoked
-- Run $ make info to print a detailed help message
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/pgriffin/awork/dealii-work/examples/step-17
Scanning dependencies of target step-17
[ 50%] Building CXX object CMakeFiles/step-17.dir/step-17.cc.o
[100%] Linking CXX executable step-17
[100%] Built target step-17
Built target release
[ 66%] Built target step-17
[100%] Run step-17 with Release configuration
Cycle 0:
Number of active cells: 64
Number of degrees of freedom: 375 (by partition: 375)
Solver converged in 8 iterations.
Peak virtual memory: 786 MB, Peak resident memory: 55 MB
Cycle 1:
Number of active cells: 204
Number of degrees of freedom: 1059 (by partition: 1059)
Solver converged in 26 iterations.
Peak virtual memory: 798 MB, Peak resident memory: 71 MB
Cycle 2:
Number of active cells: 708
Number of degrees of freedom: 3231 (by partition: 3231)
Solver converged in 53 iterations.
Peak virtual memory: 824 MB, Peak resident memory: 97 MB
Cycle 3:
Number of active cells: 2388
Number of degrees of freedom: 9891 (by partition: 9891)
Solver converged in 105 iterations.
Peak virtual memory: 905 MB, Peak resident memory: 178 MB
Cycle 4:
Number of active cells: 7484
Number of degrees of freedom: 29277 (by partition: 29277)
Solver converged in 201 iterations.
Peak virtual memory: 1142 MB, Peak resident memory: 415 MB
Cycle 5:
Number of active cells: 23220
Number of degrees of freedom: 82779 (by partition: 82779)
Solver converged in 303 iterations.
Peak virtual memory: 1798 MB, Peak resident memory: 1070 MB
Cycle 6:
Number of active cells: 71996
Number of degrees of freedom: 247185 (by partition: 247185)
Solver converged in 557 iterations.
Peak virtual memory: 3772 MB, Peak resident memory: 3045 MB
Cycle 7:
Number of active cells: 223140
Number of degrees of freedom: 743361 (by partition: 743361)
Solver converged in 736 iterations.
Peak virtual memory: 9703 MB, Peak resident memory: 8975 MB
[100%] Built target run
==================================================================
Using PETScWrappers::PreconditionLU preconditioner(system_matrix);
[100%] Switch CMAKE_BUILD_TYPE to Release
-- Autopilot invoked
-- Run $ make info to print a detailed help message
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/pgriffin/awork/dealii-work/examples/step-17
Scanning dependencies of target step-17
[ 50%] Building CXX object CMakeFiles/step-17.dir/step-17.cc.o
[100%] Linking CXX executable step-17
[100%] Built target step-17
Built target release
[ 66%] Built target step-17
[100%] Run step-17 with Release configuration
Cycle 0:
Number of active cells: 64
Number of degrees of freedom: 375 (by partition: 375)
PETScWrappers::PreconditionLU preconditioner(system_matrix);
Solver converged in 1 iterations.
Peak virtual memory: 786 MB, Peak resident memory: 55 MB
Cycle 1:
Number of active cells: 204
Number of degrees of freedom: 1059 (by partition: 1059)
PETScWrappers::PreconditionLU preconditioner(system_matrix);
Solver converged in 1 iterations.
Peak virtual memory: 798 MB, Peak resident memory: 71 MB
Cycle 2:
Number of active cells: 708
Number of degrees of freedom: 3231 (by partition: 3231)
PETScWrappers::PreconditionLU preconditioner(system_matrix);
Solver converged in 1 iterations.
Peak virtual memory: 835 MB, Peak resident memory: 108 MB
Cycle 3:
Number of active cells: 2388
Number of degrees of freedom: 9891 (by partition: 9891)
PETScWrappers::PreconditionLU preconditioner(system_matrix);
Solver converged in 1 iterations.
Peak virtual memory: 914 MB, Peak resident memory: 187 MB
Cycle 4:
Number of active cells: 7484
Number of degrees of freedom: 29277 (by partition: 29277)
PETScWrappers::PreconditionLU preconditioner(system_matrix);
Solver converged in 1 iterations.
Peak virtual memory: 1144 MB, Peak resident memory: 417 MB
Cycle 5:
Number of active cells: 23220
Number of degrees of freedom: 82779 (by partition: 82779)
^CCMakeFiles/run.dir/build.make:57: recipe for target 'CMakeFiles/run' failed
make[3]: *** [CMakeFiles/run] Interrupt
^C'd very long runtime
==================================================================
Using PETScWrappers::PreconditionILU preconditioner(system_matrix);
[100%] Switch CMAKE_BUILD_TYPE to Release
-- Autopilot invoked
-- Run $ make info to print a detailed help message
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/pgriffin/awork/dealii-work/examples/step-17
Scanning dependencies of target step-17
[ 50%] Building CXX object CMakeFiles/step-17.dir/step-17.cc.o
[100%] Linking CXX executable step-17
[100%] Built target step-17
Built target release
[ 66%] Built target step-17
[100%] Run step-17 with Release configuration
Cycle 0:
Number of active cells: 64
Number of degrees of freedom: 375 (by partition: 375)
PETScWrappers::PreconditionILU preconditioner(system_matrix);
Solver converged in 6 iterations.
Peak virtual memory: 786 MB, Peak resident memory: 56 MB
Cycle 1:
Number of active cells: 204
Number of degrees of freedom: 1059 (by partition: 1059)
PETScWrappers::PreconditionILU preconditioner(system_matrix);
Solver converged in 10 iterations.
Peak virtual memory: 798 MB, Peak resident memory: 71 MB
Cycle 2:
Number of active cells: 708
Number of degrees of freedom: 3231 (by partition: 3231)
PETScWrappers::PreconditionILU preconditioner(system_matrix);
Solver converged in 17 iterations.
Peak virtual memory: 825 MB, Peak resident memory: 98 MB
Cycle 3:
Number of active cells: 2388
Number of degrees of freedom: 9891 (by partition: 9891)
PETScWrappers::PreconditionILU preconditioner(system_matrix);
Solver converged in 29 iterations.
Peak virtual memory: 908 MB, Peak resident memory: 181 MB
Cycle 4:
Number of active cells: 7484
Number of degrees of freedom: 29277 (by partition: 29277)
PETScWrappers::PreconditionILU preconditioner(system_matrix);
Solver converged in 48 iterations.
Peak virtual memory: 1142 MB, Peak resident memory: 415 MB
Cycle 5:
Number of active cells: 23220
Number of degrees of freedom: 82779 (by partition: 82779)
PETScWrappers::PreconditionILU preconditioner(system_matrix);
Solver converged in 74 iterations.
Peak virtual memory: 1798 MB, Peak resident memory: 1071 MB
Cycle 6:
Number of active cells: 71996
Number of degrees of freedom: 247185 (by partition: 247185)
PETScWrappers::PreconditionILU preconditioner(system_matrix);
Solver converged in 113 iterations.
Peak virtual memory: 3772 MB, Peak resident memory: 3045 MB
Cycle 7:
Number of active cells: 223140
Number of degrees of freedom: 743361 (by partition: 743361)
PETScWrappers::PreconditionILU preconditioner(system_matrix);
Solver converged in 156 iterations.
Peak virtual memory: 9705 MB, Peak resident memory: 8978 MB
[100%] Built target run
