[deal.II] Re: Problems in solving two PDE systems using MPI+Thread parallel computing

[Jack]

Hi Bruno,

 

I appreciated it very much for your responses.

 

The Matrices and Vectors are of TrilinosWrappers, the solvers should use 
MPI, too. I used the OpenMPI 1.8.1. The release date is Apr 22 2014, later 
than the post on the Github. I initialized the MPI as following

try

{

 using namespace dealii;

 using namespace problem;

 Utilities::MPI::MPI_InitFinalize mpi_initialization (argc, argv,

numbers::invalid_unsigned_int);

 Problem<2> app;

 app.infofile = argv[1];

 app.run ();

}

So, It seems this error is because I called two functions which use MPI 
during solving thermal diffusion and stokes flow equations.

 According to this post https://github.com/open-mpi/ompi/issues/1081

 “It turns out that if one tries to use Open MPI with MPI THREAD MULTIPLE 
(at least master) and actually has multiple threads making concurrent MPI 
calls, one gets a flood of messages of the following ilk:

opal_libevent2022_event_base_loop: reentrant invocation. Only one 
event_base_loop can run on each event_base at once.

showing up in stderr.

This needs to be fixed before release of v2.x”

I should update my OpenMPI to the latest version and such problem will be 
avoided and I do not need to change my code for initialization of MPI and 
TBB myself?

 

Thanks for your kind help!

 

Regards,


Jack

在 2017年5月19日星期五 UTC+8下午8:25:55，Bruno Turcksin写道：
>
> Jack,
>
> are your solvers using MPI?  This looks similar to this problem 
> https://github.com/open-mpi/ompi/issues/1081 Which version of MPI are you 
> using? How do you initialize MPI? MPI_InitFinalize set 
> MPI_THREAD_SERIALIZED which "tells MPI that 
> we might use several threads but never call two MPI functions at the same 
> time." This is not want you want if your solvers use MPI. Then you need to 
> initialize MPI and TBB yourself and use MPI THREAD MULTIPLE.
>
> Best,
>
> Bruno
>
> On Friday, May 19, 2017 at 3:52:39 AM UTC-4, Jack wrote:
>>
>> Dear all,
>>
>>  
>>
>> I’m trying to solve the thermal diffusion and Stokes flow problem 
>> simultaneously, similar to the step-32. I opened two threads to solve 
>> thermal diffusion and Stokes equations during solving by linear solvers 
>> (the former is solved by CG solver and the latter is solved by GMRES 
>> solver) after assembling the matrices and RHS. I encountered a problem when 
>> I using parallel computing in combination with distributed and shared 
>> memory.
>>
>>  The code is like following:
>>
>>
>> assemble_thermal_diffusion_system();
>>
>> assemble_stokes_flow_system ();
>>
>> Threads::Task task =
>>
>> Threads::new_task (::solve_thermal_diffsuion, *this);
>>
>> solve_stokes_flow ();
>>
>> task.join();
>>
>>
>> The program ran successfully with single process and showed a high 
>> performance of thread parallel computing, unfortunately, when I ran the 
>> program using several processes I got following error. I have googled this 
>> error and can not find any solution. Could someone help me?
>>
>> Thanks in advance!
>>
>>  
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only 
>> one event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
>> event_base_loop can run on each event_base at once.
>>
>>
>> Thank you very much.
>>
>>  
>>
>> Best,
>>
>>  
>>
>> Jack
>>
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit 

[deal.II] Re: Getting error during cmake configuration step and later during make run

[Vikram Bhamidipati]

Hi Jean-Paul,

Thank you for that link. Glad to know I was not the only one to have that 
issue (illegal instruction 4). Hope the underlying cause is identified in 
some later version of the installer.

Thanks,
Vikram

On Thursday, May 18, 2017 at 3:44:43 PM UTC-5, Jean-Paul Pelteret wrote:
>
> Hi Vikram,
>
> I'm sorry that you've encountered trouble using the MacOS package. I see 
> that there might have been some issues 
>  with regards to the 
> bundling of the package, but I'm not actually sure if they're related. I 
> have no experience with this package, so there will be no guesses from me 
> as to where things might be going wrong. I'm glad to hear that you managed 
> to install it through Homebrew though.
>
> Regards,
> Jean-Paul
>
>
> On Thursday, May 18, 2017 at 8:07:13 PM UTC+2, Vikram Bhamidipati wrote:
>>
>> I finally abandoned using the terminal that comes with the installer and 
>> followed +Daniel Arndt's instructions for homebrew on mac os x (
>> https://github.com/dealii/dealii/wiki/deal.II-on-Homebrew---Linuxbrew) 
>> and was able to successfully configure, build and run the example.
>>
>> Thank you,
>> Vikram
>>
>> On Wednesday, May 17, 2017 at 1:09:42 PM UTC-5, Vikram Bhamidipati wrote:
>>>
>>> Hi,
>>>
>>> When googling I found the following info:
>>>
>>> https://github.com/NixOS/nixpkgs/issues/17372
>>>
>>> Could it be that the installer for v8.5.0 was not built for mac os x 
>>> Sierra? Anyone having a success running on Sierra?
>>>
>>> Thanks,
>>> Vikram
>>>
>>> On Tuesday, May 16, 2017 at 7:12:40 AM UTC-5, Vikram Bhamidipati wrote:

 Daniel,

 Thanks for your reply. I am attaching 2 files. I do not see 
 detailed.log file anywhere in the build directory.

 Vikram

 On Monday, May 15, 2017 at 6:25:38 AM UTC-5, Daniel Arndt wrote:
>
> Vikram,
>
>> I am new to dealii. I installed deal.II 8.5.0 using the dmg package 
>> from dealii website on mac OS Sierra. When I tried to configure I got 
>> this 
>> error:
>>
>> CMake Error at 
>> /Applications/deal.II-8.5-brew.app/Contents/Resources/brew/Cellar/cmake/3.7.2/share/cmake/Modules/CMakeTestCXXCompiler.cmake:44
>>  
>> (message):
>>   The C++ compiler
>>   
>> "/Applications/deal.II-8.5-brew.app/Contents/Resources/brew/bin/mpicxx" 
>> is
>>   not able to compile a simple test program.
>>
>>   It fails with the following output:
>>
>>Change Dir: 
>> /Users/vikramb/Documents/dealII/svn_co/deal.II/examples/step-1/CMakeFiles/CMakeTmp
>>
>>   
>>
>>   Run Build Command:"/usr/bin/make" "cmTC_5fce3/fast"
>>
>>   /Applications/Xcode.app/Contents/Developer/usr/bin/make -f
>>   CMakeFiles/cmTC_5fce3.dir/build.make CMakeFiles/cmTC_5fce3.dir/build
>>
>>   Building CXX object CMakeFiles/cmTC_5fce3.dir/testCXXCompiler.cxx.o
>>
>>   
>> /Applications/deal.II-8.5-brew.app/Contents/Resources/brew/bin/mpicxx -o
>>   CMakeFiles/cmTC_5fce3.dir/testCXXCompiler.cxx.o -c
>>   
>> /Users/vikramb/Documents/dealII/svn_co/deal.II/examples/step-1/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
>>
>>
>>   make[1]: *** [CMakeFiles/cmTC_5fce3.dir/testCXXCompiler.cxx.o] 
>> Illegal
>>   instruction: 4
>>
>>   make: *** [cmTC_5fce3/fast] Error 2
>>
>> To workaround this I changed the CXX compiler option though ccmake. I 
>> was able to successfully configure and build. However when I ran I got 
>> this 
>> error:
>>
>> make[3]: *** [CMakeFiles/run] Illegal instruction: 4
>> make[2]: *** [CMakeFiles/run.dir/all] Error 2
>> make[1]: *** [CMakeFiles/run.dir/rule] Error 2
>> make: *** [run] Error 2
>>
>> Can anyone tell what I am not doing right?
>>
> If CMake failed, there should exist
> CMakeFiles/CMakeError.log
> CMakeOutput.log
> detailed.log
> in the build directory. Can you send these files?
>
> Best,
> Daniel
>


-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Re: "operator+=" error in the assemble_system()

[Jean-Paul Pelteret]

Dear Kyusik,

Firstly (and just so that we're all clear on what you're trying to 
implement), was I correct that your comment that I linked contained the 
description of the integral that you are trying to compute?
 

> As far as I know fe_values2.shape_value(i,q_index) is vector-valued shape 
> function
>

No, this is not true. Here is this function's documentation 
.
 
This is different from what is returned 

 
if you use a FEValuesExtractor::Vector 

.
 

> sol_grad is also vector
>

Yes, since your original solution field is a scalar I think that this is 
correct. 

Is it now clear as to why you're seeing this error?

J-P

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] "operator+=" error in the assemble_system()

[hanks0227]

Dear All,

I'm trying to use two FE_Q in my assemble_system() to project the 
gradient of scalar solution onto the vector-valued FE space (that is, one 
is for scalar solution and the other is for vector)

my assemble_q() is as follow...

template 
void Step6::assemble_q ()
{
  const QGauss  quadrature_formula(Qp+1);
  const QGauss  quadrature_formula2(Qp+1);
  FEValues fe_values (fe, quadrature_formula,
   update_values|  update_gradients |
   update_quadrature_points  |  update_JxW_values);

  FEValues fe_values2 (fe2, quadrature_formula2,
   update_values|  update_gradients |
   update_quadrature_points  |  update_JxW_values);

  const unsigned int   dofs_per_cell = fe2.dofs_per_cell;
  const unsigned int   n_q_points= quadrature_formula.size();
  const unsigned int   n_q_points2= quadrature_formula2.size();

  Vector   cell_rhs (dofs_per_cell);
  FullMatrix   cell_matrix(dofs_per_cell,dofs_per_cell);

  std::vector local_dof_indices (dofs_per_cell);

  std::vector   sol_tmp(n_q_points);
  std::vector > sol_grad(n_q_points);

  typename Triangulation::active_cell_iterator
  cell = triangulation.begin_active(),
  endc = triangulation.end();
  for (; cell!=endc; ++cell)
{
 const typename DoFHandler::active_cell_iterator 
cell1(, cell->level(),

cell->index(),_handler);
 const typename DoFHandler::active_cell_iterator 
cell2(, cell->level(),

cell->index(),_handler2);
  cell_rhs = 0;

  fe_values.reinit (cell1);
  fe_values2.reinit (cell2);
  fe_values.get_function_values(solution,sol_tmp);
  fe_values.get_function_gradients(solution,sol_grad);
  for (unsigned int q_index=0; q_indexget_dof_indices (local_dof_indices);
  constraints_q.distribute_local_to_global (cell_matrix,
cell_rhs,
  local_dof_indices,
  system_matrix_q,
  system_rhs);

}
}

fe and fe2 are declared in

template 
Step6::Step6 ()
  :
  dof_handler (triangulation),
  dof_handler2 (triangulation),  //for q profile
  fe (Qp),
  fe2 (FE_Q(Qp),dim)   //for q profile
{}

However, the error occurs...

the error messages says that

[100%] Building CXX object CMakeFiles/step-6.dir/step-6.cc.o
/user2/hanks318/dealii/dealii_pre/examples/step-6/step-6.cc: In member 
function ‘void Step6::assemble_q() [with int dim = 2]’:
/user2/hanks318/dealii/dealii_pre/examples/step-6/step-6.cc:1490:   
instantiated from ‘void Step6::run() [with int dim = 2]’
/user2/hanks318/dealii/dealii_pre/examples/step-6/step-6.cc:1536:   
instantiated from here
/user2/hanks318/dealii/dealii_pre/examples/step-6/step-6.cc:665: error: no 
match for ‘operator+=’ in ‘cell_rhs.dealii::Vector::operator() 
[with Number = double](i) += dealii::operator*(const dealii::Tensor<1, dim, 
double>&, double) [with int dim = 2](fe_values2.dealii::FEValues<2, 
2>::.dealii::FEValuesBase::JxW [with int dim = 2, 
int spacedim = 2](q_index))’
/user2/hanks318/dealii/dealii_pre/include/deal.II/bundled/boost/system/error_code.hpp:
 
At global scope:
/user2/hanks318/dealii/dealii_pre/include/deal.II/bundled/boost/system/error_code.hpp:221:
 
warning: ‘boost::system::posix_category’ defined but not used
/user2/hanks318/dealii/dealii_pre/include/deal.II/bundled/boost/system/error_code.hpp:222:
 
warning: ‘boost::system::errno_ecat’ defined but not used
/user2/hanks318/dealii/dealii_pre/include/deal.II/bundled/boost/system/error_code.hpp:223:
 
warning: ‘boost::system::native_ecat’ defined but not used



What is wrong with "operator+=" in the calculation for cell_rhs(i)? (this 
part is marked as red in the above)

Could you please let me know how I can solve this problem...?

Thank you

[deal.II] Re: Problems in solving two PDE systems using MPI+Thread parallel computing

[Bruno Turcksin]

Jack,

are your solvers using MPI?  This looks similar to this problem 
https://github.com/open-mpi/ompi/issues/1081 Which version of MPI are you 
using? How do you initialize MPI? MPI_InitFinalize set 
MPI_THREAD_SERIALIZED which "tells MPI that 
we might use several threads but never call two MPI functions at the same 
time." This is not want you want if your solvers use MPI. Then you need to 
initialize MPI and TBB yourself and use MPI THREAD MULTIPLE.

Best,

Bruno

On Friday, May 19, 2017 at 3:52:39 AM UTC-4, Jack wrote:
>
> Dear all,
>
>  
>
> I’m trying to solve the thermal diffusion and Stokes flow problem 
> simultaneously, similar to the step-32. I opened two threads to solve 
> thermal diffusion and Stokes equations during solving by linear solvers 
> (the former is solved by CG solver and the latter is solved by GMRES 
> solver) after assembling the matrices and RHS. I encountered a problem when 
> I using parallel computing in combination with distributed and shared 
> memory.
>
>  The code is like following:
>
>
> assemble_thermal_diffusion_system();
>
> assemble_stokes_flow_system ();
>
> Threads::Task task =
>
> Threads::new_task (::solve_thermal_diffsuion, *this);
>
> solve_stokes_flow ();
>
> task.join();
>
>
> The program ran successfully with single process and showed a high 
> performance of thread parallel computing, unfortunately, when I ran the 
> program using several processes I got following error. I have googled this 
> error and can not find any solution. Could someone help me?
>
> Thanks in advance!
>
>  
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
> [warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
> event_base_loop can run on each event_base at once.
>
>
> Thank you very much.
>
>  
>
> Best,
>
>  
>
> Jack
>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Problems in solving two PDE systems using MPI+Thread parallel computing

[Jack]

Dear all,

 

I’m trying to solve the thermal diffusion and Stokes flow problem 
simultaneously, similar to the step-32. I opened two threads to solve 
thermal diffusion and Stokes equations during solving by linear solvers 
(the former is solved by CG solver and the latter is solved by GMRES 
solver) after assembling the matrices and RHS. I encountered a problem when 
I using parallel computing in combination with distributed and shared 
memory.

 The code is like following:


assemble_thermal_diffusion_system();

assemble_stokes_flow_system ();

Threads::Task task =

Threads::new_task (::solve_thermal_diffsuion, *this);

solve_stokes_flow ();

task.join();


The program ran successfully with single process and showed a high 
performance of thread parallel computing, unfortunately, when I ran the 
program using several processes I got following error. I have googled this 
error and can not find any solution. Could someone help me?

Thanks in advance!

 
[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.

[warn] opal_libevent2021_event_base_loop: reentrant invocation.  Only one 
event_base_loop can run on each event_base at once.


Thank you very much.

 

Best,

 

Jack

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.