Greetings Daniele,
The HDF5 group has produced something called the Packet Table which was
designed for a similar purpose. See
http://www.hdfgroup.org/HDF5/doc/HL/H5PT_Intro.html
Perhaps this document will help to answer your question.
--dan
Daniele Nicolodi wrote:
Hello, I'm really new to HDF5 and I'm trying to understand what would be
the best structure for "real time" logging of experimental data.
More in detail, I have a real time acquisition system that collects data
from multiple sources at a constant rate. Data logging is now performed
putting the data in a ring buffer and having a consumer non real time
process that takes the data from the ring buffer and writes them into an
ASCII file. The file is rotated after a certain number of samples have
been written.
Loading the data from ASCII file is however slow, and ASCII files are
not the best option for storing metadata. There fore I would like to
switch to HDF5 as storage format. However, it is not clear to me what
would be the most suitable structure for my files.
I imagine I need a structure to which I can append data, but it is not
clear to me if I should handle my data as a collection of columns (one
for each channel, plus one for the timestamp) or as a collection of
records (one record for each acquisition cycle) or if a table like
structure would be better.
Thank you very much for any hint on this topic. Best regards,
--
Daniel Kahn
Science Systems and Applications Inc.
301-867-2162
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
