Dear HDF5-Mailinglist-Subscribers,
I'm using HDF5 to record data received periodically from an EEG amplifier device to disk. The amplifier is able to sample with 38400Hz, where each sample consists of 256 float values (16-bit), one for each of the 256 channels. It sends blocks of 512 samples rather than sending each sample separately. So I get an average transfer rate of about 38400*256*4 =~ 40Megabytes/sec. The recording application is written in managed C++. So my question is what is the best configuration for the HDF5 library to handle such transfer rates (assuming that the disk itself can handle them), e.g. which dataset organization (contigues, chunked,.) would be the best solution, what chunk size (if using the chunked model) and so on. The data acquisition thread and other threads of the application should still be able to run. After experimenting a little bit I found that the DataSet.write method which writes data to the HDF5 dataset is blocking a very long time for bigger chunk sizes what makes me believing that the write-method always writes the data directly to disk. So I was wondering if it is possible to collect data in an internal buffer of the dataset which will be written to disk only after several seconds in a separate thread (so that new data can still be collected while the bunch of old collected data is being written to disk). Or maybe another solution will fit better? I would be glad if some more experienced users out there could give me some tips for tuning HDF5. Best regards, Matthias PS: Just to let you know about our upcoming events: <http://www.wcnr2010.org/> World Congress on NeuroRehabilitation from March 21 - 25, 2010 in Vienna, Austria <http://www.cnsmeeting.org/> Cognitive Neuroscience Society Meeting from April 17 - 20, 2010 with <http://www.gtec.at/profile/BCI_WS_CNS_2010.html> g.tec BCI workshop in Montreal, Canada <http://bcimeeting.org/2010/> BCI Meeting 2010 from May 31 - June 4, 2010 with <http://www.bci2000.org/BCI2000/Workshop.html> BCI2000 workshop on May 30 - 31, 2010 in Asilomar, California, USA <http://www.humanbrainmapping.org/i4a/pages/index.cfm?pageid=3342> 16th Annual Meeting of the Organization for Human Brain Mapping from June 6 - 10, 2010 with <http://www.gtec.at/profile/BCI_WS_Brainmapping_2010.html> g.tec BCI workshop in Barcelona, Spain <http://www.icchp.org/> ICCHP 2010 from July 14 - 16, 2010 with g.tec BCI workshop in Vienna, Austria <http://www.cnsorg.org/2010/> 19th Computational Neurosciences Meeting from July 24 - 30, 2010 in San Antonio, Texas, USA <http://www.icpr2010.org/> ICPR 2010 from August 23 - 26, 2010 in Istanbul, Turkey Would be nice to meet you there! cid:[email protected] _________________________________________________ cid:1A065B10873045C384A80A97969658DB@KrauszPC Matthias Zeintlinger, Dipl.-Ing. GUGER TECHNOLOGIES OG Herbersteinstr. 60, 8020 Graz, Austria phone: ++43 316 675106 - 22 fax: ++43 316 675106 - 39 e-mail: <mailto:[email protected]> [email protected] web: <http://www.gtec.at/> www.gtec.at _________________________________________________ This message and any attached files are confidential and intended solely for the addressee(s). Any publication, transmission or other use of the information by a person or entity other than the intended addressee is prohibited. If you receive this in error please contact the sender and delete the material. The sender does not accept liability for any errors or omissions as a result of the transmission.
<<image001.gif>>
<<image004.jpg>>
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
