Hi, I'm starting looking into HDF5 for structured output files, and I have seen a lot of slides showing that collective I/O was far better than independent I/O. My application is a little bit different and doesn't fit the collective I/O pattern. I have an unstructured grid split on several nodes, and when I write a result, I need to reorder the data I want to write so that it follows growing node indices (requirement of the viewers, requirements of the post-processing...). As the grid is unstructured and split with parmetis, I can't directly write the data to disk, and I'm doing asynchronous gathers (using NBC) of all data to a subset of process that will actually write. Due to how the gathering is done (all ranks fill a chunk of indices with the data they have and they are reduced at the writing rank), I can't have buffers for all gathers at the same time, so I have a list of buffers that I'm using and as soon as a gather is done, I call a callback on the corresponding rank to write down everything. This works really nice when I want to write one piece of data in a binary file with pwrite, and even if the asynchronous gather process is not optimized yet, I have improvements in the used I/O bandwidth as soon as I use several blades (2 to 3 times better for using 2 or 4 blades).
Now I'd like to do the same with HDF5. So I will get new chunks with the offset in the file each time a gather is done. If the chunks are properly organized, I will have overlapping communication and overlapping I/O as each chunk will be at another offset in the HDF5. Will the HDF5 library behave properly, meaning will it directly write on the disk waiting for other processes? I can provide the test case by private email if someone has a clue! Regards, Matthieu Brucher -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
