Good morning everyone, I’ve recently started using parallel HDF5 for my company, as we wish to save analysed data on multiple files at a time. It would be an N:N case, with an output stream for each file. The main program itself is written in C#, but we already have an API that allows us to make calls to hdf5 and MPI in C and C++. It retrieves data from an external device, executes some analysis and then saves the data, and parallelizing these three parts would speed up the process. However i’m not quite sure how to implement such parallelization on the third bit: So far i’ve seen that parallelization is usually implemented right off the bat: the program is started with mpiexec (i’m on Windows), with a specified number of processes. (like “mpiexec -n x Program.exe). Unfortunately running multiple instances of the whole program in parallel would be problematic, but i’ve seen that one should be able to spawn processes later during runtime with MPI_Spawn(), indicating an executable as a target (provided that the “main” process, the program itself, has been started with “mpiexec -n 1 Program.exe” for example). This second method could do it for us, but I was wondering if there is a more elegant way to achieve parallel output writing, like calling a function from my own program instead of an executable.
Bonus question, just to make sure i’ve got the basics of PHDF5 right in the first place: I do need to have a process for each parallel action that I want to perform in parallel, be it writing N streams to N files, or writing N streams to a single file? Thank you in advance Stefano Sent from Mail for Windows 10
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
