Hi Nicolas you wouldn't have so much trouble with the frequency as much as with the threads. There is no way you can write at that rate to an HDF5 file. I manage my locks pretty effectively and don't get any problems there but still under high frequency access, HDF5 collapses at some point. If your sources are separate processes, use one thread per process to write into separate files and merge periodically.
HTH -- dimitris 2012/7/13 Nicolas <nblouve...@gmail.com> > Dear list, > > I would like to know if hdf5 is suitable for real-time data logging or not > ? > > More precisely: I work on a project in which we want to continuously > (sampling rate ranging form 30 to 400Hz) mix a fair amount of data (several > hours) of different natures (telemetry, signals, videos). > > Data have to be written in real-time (or with a small delay) in order to > keep us from losing them on potential crash. > > Our first prototype is based on sqlite3, however we feel that some > limitations could rise from a long run usage: speed, one database == one > file, and difficulties for accessing database from several threads (Lock > exception when reading and writing at the same time). > > So, I am considering the possibility to use hdf5 as a back-end for data > storage on disk (and numpy/pytable for internal representation). Do you > think it is possible to update hdf5 file on at a regular interval from such > python binding ? > > Thank you very much ! > > Cheer, > Nicolas > > ______________________________**_________________ > Hdf-forum is for HDF software users discussion. > Hdf-forum@hdfgroup.org > http://mail.hdfgroup.org/**mailman/listinfo/hdf-forum_**hdfgroup.org<http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org> >
_______________________________________________ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
