Good morning everyone,

I’ve recently started using parallel HDF5 for my company, as we wish to save 
analysed data on multiple files at a time. It would be an N:N case, with an 
output stream for each file.
The main program itself is written in C#, but we already have an API that 
allows us to make calls to hdf5 and MPI in C and C++. It retrieves data from an 
external device, executes some analysis and then saves the data, and 
parallelizing these three parts would speed up the process. However i’m not 
quite sure how to implement such parallelization on the third bit:
So far i’ve seen that parallelization is usually implemented right off the bat: 
the program is started with mpiexec (i’m on Windows), with a specified number 
of processes. (like “mpiexec -n x Program.exe). Unfortunately running multiple 
instances of the whole program in parallel would be problematic, but i’ve seen 
that one should be able to spawn processes later during runtime with 
MPI_Spawn(), indicating an executable as a target (provided that the “main” 
process, the program itself, has been started with “mpiexec -n 1 Program.exe” 
for example).
This second method could do it for us, but I was wondering if there is a more 
elegant way to achieve parallel output writing, like calling a function from my 
own program instead of an executable. 

Bonus question, just to make sure i’ve got the basics of PHDF5 right in the 
first place: I do need to have a process for each parallel action that I want 
to perform in parallel, be it writing N streams to N files, or writing N 
streams to a single file?

Thank you in advance


Sent from Mail for Windows 10

Hdf-forum is for HDF software users discussion.

Reply via email to