Peter, I have similar questions as yours and would be interested in running benchmarks to define what would be the best chunk size to use to optimize performances of my application.
I also would like to compare performances with a bare binary file write operation in order to measure the overhead (if any) of the HDF5 API. I guess this is a common question and I'm wondering if the HDF Group has any plan to add hdf5 examples focused on benchmarking and optimizing applications ? I'm still in the learning curve and evaluation of HDF5 API and do not have answer to your question but will follow this thread with great interest ! Best regards Gerald > Message du 16/08/13 17:58 > De : "Peter Pichler" > A : "[email protected]" > Copie à : > Objet : [Hdf-forum] How to measure I/O speed? > > I am trying to measure the I/O speed programmatically from our application. > I've tried the naïve approach of periodically getting the file size and > calculating the growth like this (simplified): > > speed = (H5File::getFileSize() - H5File::getFreeSpace()) / time_period; > > This produces good clean results on Windows but running the same program on > Linux produces very "jerky" results. > The file size jumps by about 300MB every minute or so, but the reported free > space is always constant (and very small, about 2 KB), resulting in the > reported I/O speed of 0 most of the time, jumping to over 100MB/s once a > minute or so. > > I could understand that there may be some caching going on and the cache is > dumped from time to time. However running iotop on the same machine proves > that our application writes to the file all the time at a steady rate, > suggesting that the file size increase jump should be reflected as free space. > > So why does getFreeSpace() keep returning a constant value? > Is there a better way of measuring the I/O speed? > > > [ (pas de nom de fichier) (0.2 Ko) ] Une messagerie gratuite, garantie à vie et des services en plus, ça vous tente ? Je crée ma boîte mail www.laposte.net _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
