On 13/07/2012 14:31, Nicolas wrote:
I would like to know if hdf5 is suitable for real-time data logging or not ?
So, I am considering the possibility to use hdf5 as a back-end for data storage on disk (and numpy/pytable for internal representation). Do you think it is possible to update hdf5 file on at a regular interval from such python binding ?
Independently on the storage format, logging to disc from a real-time thread is not a very good idea: accessing to file system storage (and to mass storage in general) has not deterministic latencies.
What is recommended to do in this cases is to have a real-time process or thread collect the data and store it in an in memory ring-buffer, from where it is collected by a non real-time process and written to disc. It is fairly trivial to implement a thread safe single reader, single writer ring-buffer (an you can find many implementations online).
Cheers, Daniele _______________________________________________ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
