Hi Johannes,
22s is about right. Making your chunk size to be about 64kB or 1MB
will improve the compression ratio and I/O performance. You can try
different chunk size to get the best result. Compression level 6 should be
a good choice for i/o performance and compression ratio.
FYI, variable length strings (or any other variable length data) do not
compress well. In the example that I posted on March 23, I used
compression level 5, the compressed data is about 0.8GB and the uncompressed
data is about 1.2GB. The actual compression ratio also depends on the
contend of your strings.
Thanks
--pc
Stamminger, Johannes wrote:
On Fr, 2010-04-09 at 15:21 +0200, Johannes Stamminger wrote:
I'm preparing a test with writing blocks of 100 strings at one time ...
This improved things *a lot*, it took only 22s now! And yes, all strings
are still written to the hdf file ;-)
So it is remaining the file size issue. A played around with the chunk
size and compression level but did not succeed at all ... :-(
Have a nice weekend,
Johannes Stamminger
------------------------------------------------------------------------
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
