Good morning everyone,
I’ve recently started using parallel HDF5 for my company, as we wish to save
analysed data on multiple files at a time. It would be an N:N case, with an
output stream for each file.
The main program itself is written in C#, but we already have an API that
allows us to make calls to hdf5 and MPI in C and C++. It retrieves data from an
external device, executes some analysis and then saves the data, and
parallelizing these three parts would speed up the process. However i’m not
quite sure how to implement such parallelization on the third bit:
So far i’ve seen that parallelization is usually implemented right off the bat:
the program is started with mpiexec (i’m on Windows), with a specified number
of processes. (like “mpiexec -n x Program.exe). Unfortunately running multiple
instances of the whole program in parallel would be problematic, but i’ve seen
that one should be able to spawn processes later during runtime with
MPI_Spawn(), indicating an executable as a target (provided that the “main”
process, the program itself, has been started with “mpiexec -n 1 Program.exe”
This second method could do it for us, but I was wondering if there is a more
elegant way to achieve parallel output writing, like calling a function from my
own program instead of an executable.
Bonus question, just to make sure i’ve got the basics of PHDF5 right in the
first place: I do need to have a process for each parallel action that I want
to perform in parallel, be it writing N streams to N files, or writing N
streams to a single file?
Thank you in advance
Sent from Mail for Windows 10
Hdf-forum is for HDF software users discussion.