> > I will be very interested in your performance results. Also, check the > > memory usage > > as you test the performance. > > I keep you informed ;-).
My test setup was to split up 488MB binary data to varying long portions and writing each as hex string (as I fail to write binary up to now) directly a) a zip file (on the fly inflation) b) to a hdf5 file (variable strings to packet table, chunk size 100, compression 9) Writing to the zip takes 66s, to hdf 432s. Resulting zip file is 28MB, hdf file is 961MB. As it is a java app the only thing I can tell concerning memory consumption is that I did not see increasing mem, does not look like eating the mem (garbage collecting returned needed mem to a size as used at startup). Both, output file size and performance, look like an issue to me: Concerning performance I found http://www.mail-archive.com/[email protected]/msg00024.html Therefore I expect hopefully some improvements with the mentioned 1.8.5 release. The compression of the packet table seems to take *no* effect: sum of sizes of all strings as written to the zip is 938M ... :-( Any hints of what I might have done the wrong here? My initialisation code: fH5fileId = Hdf5Hl.H5Fcreate( fHdf5File.toString(), HDF5Constants.H5F_ACC_TRUNC, HDF5Constants.H5P_DEFAULT, HDF5Constants.H5P_DEFAULT); fStrTypeId = Hdf5Hl.H5Tcopy (HDF5Constants.H5T_C_S1); status = Hdf5Hl.H5Tset_size (fStrTypeId, HDF5Constants.H5T_VARIABLE); fVlTableId = Hdf5Hl.H5PTcreate_fl(fH5fileId, "/Packets As Received", fStrTypeId, 100, 9); For writing the string H5PTappend is used with only 1 string being written each time (~1.100.000 calls). I'm preparing a test with writing blocks of 100 strings ... Any hints appreciated, Johannes Stamminger
signature.asc
Description: This is a digitally signed message part
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
