Hello everyone,
We are using PRISMS-PF, a phase field framework finite element built upon
the deal.II library (with the MPI and p4est) dependencies.
The code uses the deal.II DataOut class (see code snippet at the end of the
email) to output simulation results into vtu files at certain time steps.
We have observed that for parallel runs (especially on multiple nodes) the
simulation gets stuck whenever a file of type (and extension) "lock" is
created.
For example, for the initial conditions, the code is supposed to output a
file with name:
solution-0000000.vtu
but then it also creates s file with name
solution-0000000.vtu-1364024289-50151.lock
This is what the support from the cluster told us about the issue:
*This seems to be an issue between dealii and openmpi working together. I
looks like dealii will lock a file when it it working on it across multiple
nodes so it doesn't get written to prematurely, but there might be an
instance of communications in mpi failing.*
We are complaining with gcc/8.2.0 and openmpi/3.1.4 but the issue still
happens with openMPI/ 3.1.6.
If we use openMPI/4.2.0 the "lock" doesn't happen but some of the files
appear to be corrupt.
We are using deal.II v 9.1.1
Has anyone encountered this issue? Any help is appreciated.
Thank you!
David
Output method (some sections have been ommited, output part in bold):
===
*//outputResults() method for MatrixFreePDE class*
#include "../../include/matrixFreePDE.h"
#include <deal.II/numerics/data_out.h>
*//output results*
*template* <*int* dim, *int* degree>
*void* MatrixFreePDE<dim,degree>::outputResults() {
*//log time*
computing_timer.enter_section("matrixFreePDE: output");
*//create DataOut object*
DataOut<dim> data_out;
*//loop over fields*
*for*(*unsigned* *int* fieldIndex=0; fieldIndex<fields.size();
fieldIndex++){
*//mark field as scalar/vector*
std::vector<DataComponentInterpretation::DataComponentInterpretation>
dataType \
(fields[fieldIndex].numComponents, \
(fields[fieldIndex].type==SCALAR ? \
DataComponentInterpretation::component_is_scalar: \
DataComponentInterpretation::component_is_part_of_vector));
*//add field to data_out*
std::vector<std::string> solutionNames
(fields[fieldIndex].numComponents, fields[fieldIndex].name.c_str());
data_out.add_data_vector(*dofHandlersSet[fieldIndex],
*solutionSet[fieldIndex], solutionNames, dataType);
}
data_out.build_patches (degree);
*//write to results file*
*//file name*
std::ostringstream cycleAsString;
cycleAsString <<
std::setw(std::floor(std::log10(userInputs.totalIncrements))+1) <<
std::setfill('0') << currentIncrement;
*char* baseFileName[100], vtuFileName[100];
sprintf(baseFileName, "%s-%s", userInputs.output_file_name.c_str(),
cycleAsString.str().c_str());
sprintf(vtuFileName, "%s.%u.%s",
baseFileName,Utilities::MPI::this_mpi_process(MPI_COMM_WORLD),userInputs.output_file_type.c_str());
*// Write to file in either vtu or vtk format*
*if* (userInputs.output_file_type == "vtu"){
*// Set flags to output the time and cycle number as part of the vtu
file*
dealii::DataOutBase::VtkFlags flags;
flags.time = currentTime;
flags.cycle = currentIncrement;
flags.print_date_and_time = *true*;
data_out.set_flags(flags);
*// Write the results to a file shared between all processes*
* char svtuFileName[100];*
* sprintf(svtuFileName, "%s.%s", baseFileName
,userInputs.output_file_type.c_str());*
* data_out.write_vtu_in_parallel(svtuFileName, MPI_COMM_WORLD);*
* pcout << "Output written to:" << svtuFileName << "\n\n";*
}
*//log time*
computing_timer.exit_section("matrixFreePDE: output");
}
===
--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see
https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to dealii+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/dealii/CAJdNbL9G_qNwUU-pAqFYxKbEdT%3DvO34uE58e78poQi63VyJCtQ%40mail.gmail.com.
