-----Original Message----- From: Hdf-forum [mailto:[email protected]] On Behalf Of Rob Latham Sent: Wednesday, January 28, 2015 1:13 PM To: [email protected] Subject: Re: [Hdf-forum] File keeps being updated long after the dataset is closed
On 01/28/2015 10:24 AM, Mohamad Chaarawi wrote: > Hi Maxime, > > H5Dwrite is for writing raw data, and unlike HDF5 metadata operations the > library does not require them to be collective unless you ask for it. > For a list of HDF5 function calls which are required to be collective, look > here: > http://www.hdfgroup.org/HDF5/doc/RM/CollectiveCalls.html > > For raw data, we do not detect whether you are writing to the same position > in the file or not, and so we just pass the data down onto MPI to write > collectively or independently. You know, having struggled with etypes, ftypes, and datatype equivalence, it turns out that HDF5 (or pnetcdf or an application) has a much easier time determining if everyone is reading the same data. [msc] I agree. I have it in my bucket of things to do to, improve this, at least for the metadata HDF5 reads. For raw data, it is very easy to detect that but would introduce unnecessary communication in many cases to check for overlap. Maybe an additional property that tells HDF5 to look for overlaps for reads might be useful here.. Writing the same data is actually "undefined" (by the strict letter of the MPI standard), and while no one does this it would be fun to detect this condition and write out 0xDEADBEEF and see how many applications break.... [msc] yes! Mohamad ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5 _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
