Bonjour Jean-Paul, First, thank you for your detailed answer, it is immensively appreciated.
1. You are right, the documentation does not suggest using or not using openmp with Trilinos. I was just skeptical, because if you look here :https://dealii.org/9.0.0/external-libs/trilinos.html there is never a mention of openmp as a possible flag. It kind off gave me the impression that this was something that might not be supported. It could be a good idea to add a single line that says something like: Similarly, to enable OpenMP, commands should include -DTPL_ENABLE_OpenMP:BOOL=ON \ 2. Furthermore, when you compile Trilinos with OpenMP and you try to compile the latest version of DEAL.II, you get a compilation error when ".hpp" from Kokkos are included. The error reads something like: Kokkos was compiled with Openmp but the compiler did not pass an openmp flag . This can be easily fixed by manually adding -fopenmp to the CXX flags used by dealii. However, would it not be a better idea to add a DEALII_ENABLE_OpenMP Flag directly in the cmake to ensure that if you put that flag on, the -fopenmp flag is enabled? Maybe I missed such an option. It just made me unsure if I was doing something supported or not. 3. When compiled with OpenMP, I got deceptively poor performances, but maybe this is because of the relatively small size of my application. I would have (naively maybe) expected that the time to solver a linear system with GMRES using 1 MPI with 4 OMP threads would have been lower than the time it takes with 4 MPI and on my application this was not the case. I was surprised because I was expecting my ILU Preconditioning to work better on lower amount of cores, but maybe this is related to fill-in or other issues? Thank you for the rapid and very detailed answer. This makes this community so great. On Monday, 4 March 2019 03:05:06 UTC-5, Jean-Paul Pelteret wrote: > > Dear Bruno, > > The suggested compilation options for Trilinos do not suggest the use of > OpenMP and the flag is not enabled by default. > > > Can you please indicate where you read this? I’m looking at our > documentation for interfacing to external libraries, i.e. > https://dealii.org/9.0.0/readme.html > https://dealii.org/9.0.0/external-libs/trilinos.html > <https://www.google.com/url?q=https%3A%2F%2Fdealii.org%2F9.0.0%2Fexternal-libs%2Ftrilinos.html&sa=D&sntz=1&usg=AFQjCNH2OqPU-RSXffqy3kEVGSa0smsESQ> > and I don’t see any mention of OpenMP. > > Does this mean that the compilation of Trilinos with the suggested flag > enables shared memory parallelism when using the TrilinosWrappers function, > or is it necessary to compile Trilinos with openmp or threads enabled? > > > Trilinos does its own thread management, so irrespective of whether you've > built it with threading enabled (i.e. with the OpenMP or pthreads options > enabled) enabling/disabling threading in deal.II will have no effect on > this. What *might* be possible is that you could have scheduling > conflicts, e.g. what is mentioned here > > https://software.intel.com/en-us/forums/intel-threading-building-blocks/topic/281761 > if you use multiple threading models in one code base. But I don’t know > enough about the topic to say anything more than mention the potential > issue. > > What are the official guidelines for this? > > > I would say that there are none — how you’re wanting to run your > simulations (e.g. desktop, cluster computing, etc.) dictates what > parallelism model (i.e. multi-threading, MPI, or hybrid) would work best > for you. Would you care to elaborate more on this so that we could maybe > make some suggestions? > > Best, > Jean-Paul > > On 03 Mar 2019, at 21:40, Bruno Blais <[email protected] <javascript:>> > wrote: > > Hello everyone, > I have a quick question for which I have not found documentation. > The suggested compilation options for Trilinos do not suggest the use of > OpenMP and the flag is not enabled by default. > DEALII by default also compiles using TBB for shared memory parallelism. > Does this mean that the compilation of Trilinos with the suggested flag > enables shared memory parallelism when using the TrilinosWrappers function, > or is it necessary to compile Trilinos with openmp or threads enabled? > What are the official guidelines for this? > > Thanks a lot > Bruno > > -- > The deal.II project is located at http://www.dealii.org/ > For mailing list/forum options, see > https://groups.google.com/d/forum/dealii?hl=en > --- > You received this message because you are subscribed to the Google Groups > "deal.II User Group" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > For more options, visit https://groups.google.com/d/optout. > > > -- The deal.II project is located at http://www.dealii.org/ For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en --- You received this message because you are subscribed to the Google Groups "deal.II User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
