Another piece of information that would be useful for the h5perf
benchmark/test would be the resulting file sizes for the write
operation. From this, one could get a clear idea of just how much
overhead a particular dataset layout was costing, both in the relative
sense (by comparing one HDF5 layout to another) and in an "absolute"
sense (by comparing an HDF5 layout to the raw posix file size).
Quincey Koziol wrote:
Hi John,
On May 9, 2011, at 3:17 PM, John Knutson wrote:
I've been using h5perf_serial in the process of testing the iRODS VFD that I've
been working on, and have ultimately found myself thinking it might be time to
revisit my choices for chunking and transfer sizes for our data. Eventually I
remembered that throughput wasn't the only or even the primary measure of
performance in our situation.
My suggestion is that there also be a performance benchmark that measures
latency, i.e., the amount of time it takes to do a single write or read. When
you're streaming data like we are, the ideal chunk size for latency may yield
very low throughput.
Additionally, while h5perf_serial supports the option of only doing write
tests, it would be really handy to have a similar option for only doing read
tests - that is, decouple reading and writing to have a better chance of
getting read throughput measurements that are really only showing the
throughput of the cache (disk, system, whatever it may be).
We're planning to update the parallel I/O performance benchmark and
will keep this in mind, trying to migrate the changes back to the serial
version also.
Thanks,
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
