On 05/27/2014 02:45 PM, Quincey Koziol wrote:
On May 27, 2014, at 2:43 PM, Dana Robinson <[email protected]
<mailto:[email protected]>> wrote:
Hi,
The HDF5 library does not support asynchronous I/O at this time. We
are looking into including async I/O support in a future release, however.
I've attached a document that describes our current ideas in this space.
Good read. Just how compute bound is HDF5, anyway? I'm always living
in a land of large datasets, where library overhead is dwarfed by the
I/O workload overhead.
you did not mention the multi-dataset I/O approach: it's a half-step
towards asynchronism -- or maybe a half-step backwards -- in that
instead of decoupling the description of the data with the execution of
the data, HDF5's multi-dataset routines will describe more data in a
single call.
I don't think the global HDF5 lock precludes an async approach.
Probably this async facility should exist on top of HDF5, though, and
can provide the caching, read-ahead, coalescing, and other benefits
while leaving the bulk of the 300k lines of C code untouched. In my
head it's MPI_THREAD_FUNELED for HDF5.
The various ways one can manage MPI progress are instructive here.
==rob
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
