Dear list members, I am facing a really weird problem, that I have been struggling with for a while now. I have written a problem class which, based on other objects, generates a system matrix, rhs and solution vector object. The datastructures are Trilinos Block distributed types. When I do this for the first time it all works perfectly. However the class is part of an optimization scheme and usually at the second time the object is used (randomly also later, but this has only happened once or twice) the Solver does not start. I am checking with MPI-barriers to see if all processes arrive at the GMRES::solve and they do but somehow not even my own preconditioners vmult method gets called anymore. The objects (the two vectors and the system matrix are exactly the same that they have been at the previous step (only slightly different numbers, but same vectors of IndexSets for the partition among processors)
I have debugged this code-segment with Eclipse and the parallel debugger but don't know what to do with the call stack: 18 ompi_request_default_wait_all() 7fffddd54b15 17 ompi_coll_tuned_barrier_intra_recursivedoubling() 7fffcf9abb5d 16 PMPI_Barrier() 7fffddd68a9c 15 Epetra_MpiDistributor::DoPosts() 7fffe4088b4f 14 Epetra_MpiDistributor::Do() 7fffe4089773 13 Epetra_DistObject::DoTransfer() 7fffe400a96a 12 Epetra_DistObject::Export() 7fffe400b7b7 11 int Epetra_FEVector::GlobalAssemble<int>() 7fffe4023d7f 10 Epetra_FEVector::GlobalAssemble() 7fffe40228e3 9 dealii::TrilinosWrappers::MPI::Vector::reinit() trilinos_vector.cc:261 7ffff52c937e 8 dealii::TrilinosWrappers::MPI::BlockVector::reinit() trilinos_block_vector.cc:191 7ffff4e43bd9 7 dealii::internal::SolverGMRES::TmpVectors<dealii::TrilinosWrappers::MPI::BlockVector>::operator() solver_gmres.h:535 4a847d 6 dealii::SolverGMRES<dealii::TrilinosWrappers::MPI::BlockVector>::solve<dealii::TrilinosWrappers::BlockSparseMatrix, PreconditionerSweeping>() solver_gmres.h:813 4d654a 5 Waveguide::solve() Waveguide.cpp:1279 48f150 The last line (5) here is a function I wrote which calls SolverGMRES<dealii::TrilinosWrappers::MPI::BlockVector>::solve with my preconditioner (which works perfecly fine during the previous run. I found some information online about MPI_Barrier being instable sometimes but I don't know enough about the inner workings of Trilinos (Epetra) and Dealii to make a judgment call here. If none can help I will try to provide a code fragment but I doubt that will be possible (if it really is a racing condition and I strip away the rather large ammout of code surrounding this segment, it is unlikely to be reproducible. Originally I had used two MPI communicatorsthat were only different in the numbering of the processes (one for the primal, one for the dual problem) and created two independend objects of my problem class hich only used their respective communicator. In that case, the solver had only worked whenever the numbering of processes was either equal to that of MPI_COMM_WORLD or exactly the opposite but not for say 1-2-3-4 -> 1-3-2-4 and gotten stuck in the exact same way. I had thought it might be some internal use of MPI_COMM_WORLD that was blocking somehow but it also happens now that I only use one communicator (MPI_COMM_WORLD). Thank you in advance for your time, Pascal Kraft -- The deal.II project is located at http://www.dealii.org/ For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en --- You received this message because you are subscribed to the Google Groups "deal.II User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
